Image processing method, device, electronic device and computer readable storage medium
By acquiring and adjusting the guiding filter parameters of local areas in image processing, the problems of image noise and halo effects in traditional sharpening processing methods are solved, and image quality improvement and computational efficiency optimization are achieved.
Patent Information
- Application Number
- CN202210474111.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-29
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2042-04-29
AI Technical Summary
While traditional sharpening processing methods enhance image details and edges, they also enhance image noise and halo effects, affecting image quality.
By obtaining the guide filtering parameters of the target local area in the image to be processed and adjusting them based on preset conditions, the target guide filtering parameters are determined, and the range of filtering parameters is limited to balance sharpening effects, image noise and halo effects, thereby improving the image Sharpening quality.
The balance between sharpening effects, image noise and halo effects is achieved, improving the quality of image sharpening, reducing computational complexity and reducing hardware pressure.
Smart Images

Figure CN114913084B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and more specifically, to an image processing method, device, electronic device, and computer-readable storage medium. Background Art
[0002] Sharpening is an important step in image processing, which can enhance the detail information in the image, improve the imaging quality of the edge area of the image, and make the image visually clearer.
[0003] At present, the image is generally sharpened by filtering algorithms. From the perspective of information, sharpening the image by filtering algorithms does not increase the information contained in the image, it just processes the original information to make the image more in line with the response characteristics of human vision.
[0004] Traditional sharpening methods enhance image details and edges, but also enhance image noise and halo effect, affecting image quality. Summary of the invention
[0005] The purpose of the embodiments of the present application is to solve the problem of enhancing image noise and halo effect when sharpening an image.
[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, the method comprising:
[0007] Obtaining guided filtering parameters corresponding to the target local area in the image to be processed;
[0008] Adjusting the guided filter parameters based on the preset conditions, and determining target guided filter parameters corresponding to the guided filter parameters;
[0009] Filtering the pixel value of the central pixel point in the local area of the target according to the target-guided filtering parameter to obtain the processed pixel value of the central pixel point;
[0010] Based on the processed pixel values corresponding to the central pixel points of each target local area, a sharpened image corresponding to the image to be processed is obtained;
[0011] The preset conditions include guiding the filtering parameter to be not lower than a first threshold corresponding to the image noise and not higher than a second threshold corresponding to the halo effect.
[0012] Optionally, the guided filtering parameters include guided filtering parameters of at least one detail layer and guided filtering parameters of at least one edge layer;
[0013] The guided filter parameters are adjusted based on the preset conditions to determine the target guided filter parameters corresponding to the guided filter parameters, including:
[0014] If it is determined that the guided filter parameter of the detail layer is not lower than the first threshold, the guided filter parameter of the detail layer is used as the target guided filter parameter of the corresponding target local area;
[0015] If it is determined that the guided filter parameter of the detail layer is lower than the first threshold, the first threshold is used as the target guided filter parameter of the corresponding target local area.
[0016] Optionally, adjusting the guided filtering parameter based on a preset condition to determine a target guided filtering parameter corresponding to the guided filtering parameter further includes:
[0017] If it is determined that the guided filter parameter of the edge layer is not higher than the second threshold, the guided filter parameter of the edge layer is used as the target guided filter parameter of the corresponding target local area;
[0018] If it is determined that the guided filter parameter of the edge layer is higher than the second threshold, the target guided filter parameter of the corresponding target local area takes a value of 1.
[0019] Optionally, obtaining a guided filtering parameter corresponding to a target local area in the image to be processed includes:
[0020] Determine the infinity norm of the target local area;
[0021] The corresponding guided filter parameters are determined according to the infinity norm.
[0022] Optionally, determining corresponding guided filtering parameters according to the infinite norm includes:
[0023] According to the infinite norm, a mean value of the infinite norm in the target local area is determined, and a detail layer index and an edge layer index are determined according to the mean value of the infinite norm;
[0024] Among them, the mean of the infinite norm is used to replace the variance in the local area, and the variance in the target local area is the variance of the pixel values of all pixels in the target local area; there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect;
[0025] The corresponding guided filtering parameters are obtained from a preset lookup table according to the detail layer index and the edge layer index.
[0026] Optionally, before determining the corresponding guided filtering parameter according to the infinite norm, the method further includes:
[0027] Obtaining the mean of the infinite norm in the local target area in the sample image, and determining the corresponding guided filtering parameters according to the mean of the infinite norm;
[0028] A lookup table is created based on the association between the mean of the infinity norm and the guided filter parameters.
[0029] Optionally, filtering the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point includes:
[0030] Obtaining a sharpening intensity coefficient of a detail layer and a sharpening intensity coefficient of an edge layer;
[0031] According to the guided filtering parameters of the detail layer, the sharpening intensity coefficient of the detail layer, the guided filtering parameters of the edge layer and the sharpening intensity coefficient of the edge layer, the sharpening function of the filtering processing is determined, and the pixel value of the central pixel point in the target local area is introduced into the sharpening function to obtain the processed pixel value of the central pixel point.
[0032] According to another aspect of an embodiment of the present application, there is provided an image processing device, the device comprising:
[0033] An acquisition module is used to acquire the guided filtering parameters corresponding to the target local area in the image to be processed;
[0034] A determination module, used to adjust the guide filter parameter based on a preset condition and determine a target guide filter parameter corresponding to the guide filter parameter;
[0035] A processing module, used for filtering the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point;
[0036] A generation module, used for obtaining a sharpened image corresponding to the image to be processed based on the processed pixel values corresponding to the central pixel points of each target local area;
[0037] The preset conditions include guiding the filtering parameter to be not lower than a first threshold corresponding to the image noise and not higher than a second threshold corresponding to the halo effect.
[0038] Optionally, the guided filtering parameters include guided filtering parameters of at least one detail layer and guided filtering parameters of at least one edge layer;
[0039] Determine the module specifically for:
[0040] If it is determined that the guided filter parameter of the detail layer is not lower than the first threshold, the guided filter parameter of the detail layer is used as the target guided filter parameter of the corresponding target local area;
[0041] If it is determined that the guided filter parameter of the detail layer is lower than the first threshold, the first threshold is used as the target guided filter parameter of the corresponding target local area.
[0042] Optionally, the determination module is further configured to:
[0043] If it is determined that the guided filter parameter of the edge layer is not higher than the second threshold, the guided filter parameter of the edge layer is used as the target guided filter parameter of the corresponding target local area;
[0044] If it is determined that the guided filter parameter of the edge layer is higher than the second threshold, the target guided filter parameter of the corresponding target local area takes a value of 1.
[0045] Optionally, the acquisition module includes:
[0046] A first acquisition submodule is used to determine the infinite norm of the target local area;
[0047] The second acquisition submodule is used to determine the corresponding guided filtering parameters according to the infinite norm.
[0048] Optionally, the second acquisition submodule includes:
[0049] A first acquisition unit is used to determine a mean value of the infinite norm in the target local area according to the infinite norm, and determine a detail layer index and an edge layer index according to the mean value of the infinite norm;
[0050] Among them, the mean of the infinite norm is used to replace the variance in the local area, and the variance in the target local area is the variance of the pixel values of all pixels in the target local area; there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect;
[0051] The second acquisition unit is used to acquire corresponding guided filtering parameters from a preset lookup table according to the detail layer index and the edge layer index.
[0052] Optionally, the device further comprises:
[0053] A pre-acquisition module, used to obtain the mean of the infinite norm in the local target area in the sample image, and determine the corresponding guided filtering parameters according to the mean of the infinite norm;
[0054] A creation module is used to create a lookup table according to the correlation between the mean of the infinite norm and the guided filtering parameters.
[0055] Optionally, the processing module includes:
[0056] A first processing submodule is used to obtain a sharpening intensity coefficient of a detail layer and a sharpening intensity coefficient of an edge layer;
[0057] The second processing sub-module is used to determine the sharpening function of the filtering processing according to the guiding filtering parameters of the detail layer, the sharpening intensity coefficient of the detail layer, the guiding filtering parameters of the edge layer and the sharpening intensity coefficient of the edge layer, and introduce the pixel value of the central pixel point of the target local area into the sharpening function to obtain the processed pixel value of the central pixel point.
[0058] According to another aspect of the embodiments of the present application, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the steps of the image processing method of any one of the above aspects.
[0059] According to another aspect of an embodiment of the present application, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of the image processing method of any one of the above aspects.
[0060] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the image processing method of any one of the above aspects are implemented.
[0061] The beneficial effects of the technical solution provided by the embodiment of the present application are:
[0062] By limiting the range of guided filter parameters, a balance between sharpening effect, image noise and halo effect is achieved, thereby improving the quality of image sharpening. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.
[0064] Figure 1 One of the flowcharts of an image processing method provided in an embodiment of the present application;
[0065] Figure 2 A second flowchart of an image processing method provided in an embodiment of the present application;
[0066] Figure 3 A third flowchart of an image processing method provided in an embodiment of the present application;
[0067] Figure 4 A fourth flowchart of an image processing method provided in an embodiment of the present application;
[0068] Figure 5 A fifth flowchart of an image processing method provided in an embodiment of the present application;
[0069] Figure 6 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;
[0070] Figure 7 A schematic diagram of the structure of an electronic device for image processing provided in an embodiment of the present application. DETAILED DESCRIPTION
[0071] The embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the implementation methods described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.
[0072] It will be understood by those skilled in the art that, unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application refer to that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude the implementation as other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the one element may be directly connected or coupled to the other element, or it may refer to that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein indicates at least one of the items defined by the term, for example, "A and / or B" may be implemented as "A", or as "B", or as "A and B".
[0073] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0074] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on or combine with each other, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.
[0075] The present application provides an image processing method, such as Figure 1 As shown, including:
[0076] Step S101, obtaining guided filtering parameters corresponding to the target local area in the image to be processed.
[0077] The image to be processed is the original image that needs to be sharpened, and the local area is an area of a preset size centered on each pixel point determined when scanning the image to be processed. Image processing is performed in units of local areas. First, the target local area in the image to be processed is determined, and the guided filtering parameters corresponding to the target local area are obtained. For example, the image to be processed is a 100*100 image, and the local area is a 5*5 area. Then, there are 10,000 pixels in the image to be processed, and accordingly, there are 10,000 5*5 local areas centered on the pixels in the image to be processed. The size of the local area can be pre-set. The 5*5 size local area in the embodiment of the present application is only used as an example, and no limitation is imposed on this.
[0078] For an image to be processed, the image can be scanned by sliding a window, from top to bottom and from left to right, to determine the position of each pixel, and obtain the local area corresponding to each pixel in turn. The local area corresponding to the pixel at the boundary of the image to be processed can be obtained by mirror filling the image to be processed.
[0079] The image processing method provided in the embodiment of the present application can be implemented by a preset image processing model. For example, the image to be processed is input into a preset image processing model, and the image processing model executes the steps in the image processing method, and finally outputs a processed sharpened image.
[0080] The image processing model is designed based on the guided filter. The principle of the guided filter is to filter the input image through the guide map, so that the output image is roughly similar to the input image, and the texture part is similar to the guide map. The guide map can be the input image itself. There is a linear relationship between the pixels of the input image and the pixels of the output image, which can be represented by a linear function.
[0081] It can be understood that the image to be processed in the embodiment of the present application is the input image, and the sharpened image obtained after image processing is the output image. Correspondingly, the guided filtering parameters are linear coefficients in a linear function related to the image to be processed and the sharpened image. The guided filtering parameters are obtained by calculating the pixel values of the pixel points in each target local area. Therefore, by determining the pixel points in each target local area in the image to be processed, the guided filtering parameters corresponding to each target local area in the image to be processed can be obtained.
[0082] In a preferred embodiment of the present application, before executing step S101 on the image to be processed, the image to be processed may be firstly subjected to grayscale processing.
[0083] Step S102, adjusting the guided filtering parameter based on the preset condition, and determining the target guided filtering parameter corresponding to the guided filtering parameter;
[0084] The preset conditions include guiding the filtering parameter to be not lower than a first threshold corresponding to the image noise and not higher than a second threshold corresponding to the halo effect.
[0085] If it is determined that the guided filtering parameters meet the preset conditions, the guided filtering parameters are used as the target guided filtering parameters of the corresponding target local area; if it is determined that the guided filtering parameters do not meet the preset conditions, the preset guided filtering parameters are determined as the target guided filtering parameters corresponding to the target local area.
[0086] Image noise is unnecessary or redundant interference information existing in image data. The halo effect refers to the severe overshoot or undershoot on both sides of the edge caused by excessive edge enhancement in image data, which is intuitively represented by the generation of white or black edges at the edges.
[0087] Image sharpening includes enhancing the detail layer and edge layer in the image. Since the distinction between image noise and detail is small, when the detail layer in the image is enhanced, the image noise in the image will also be enhanced; when the edge layer in the image is enhanced too much, an obvious halo effect will also appear. In other words, image sharpening will enhance both image noise and halo effect at the same time.
[0088] The value of the guided filter parameter determines the sharpening effect after image processing. In the embodiment of the present application, the value of the guided filter parameter is restricted, and a first threshold corresponding to image noise and a second threshold corresponding to the halo effect are given, so as to balance the sharpening effect, image noise and halo effect and improve the quality of the sharpened image.
[0089] Specifically, for an image to be processed, the image processing model can automatically decompose it into different feature layers, including but not limited to detail layers and edge layers. For example, the pixel point area containing the target object in the image to be processed is decomposed into a detail layer, and the pixel point area containing the outer contour of the target object in the image to be processed is decomposed into an edge layer. The specific decomposition method is not limited in the embodiments of the present application.
[0090] If it is determined that the guided filter parameter meets the preset condition, that is, if it is determined that the guided filter parameter is not lower than the first threshold corresponding to the image noise, and is not higher than the second threshold corresponding to the halo effect, the guided filter parameter is used as the target guided filter parameter of the corresponding local area. Specifically, the image to be processed may include a detail layer and an edge layer, wherein the guided filter parameter meets the preset condition, which is manifested in that the guided filter parameter of the detail layer is not lower than the first threshold corresponding to the image noise, and the guided filter parameter of the edge layer is not higher than the second threshold corresponding to the halo effect.
[0091] If it is determined that the guided filter parameters do not meet the preset conditions, that is, the guided filter parameters of the detail layer are lower than the first threshold, the preset guided filter parameters are determined as the target guided filter parameters corresponding to the local area; or the guided filter parameters of the edge layer are higher than the second threshold, the preset guided filter parameters are determined as the target guided filter parameters corresponding to the local area.
[0092] It should be noted that the preset guide filter parameters of the detail layer are different from the guide filter parameters of the edge layer.
[0093] Step S103 , filtering the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point.
[0094] After determining the target guided filtering parameters of each target local area, the sharpening function corresponding to the central pixel point of each target local area can be determined. The pixel value of each pixel point is introduced into the corresponding sharpening function, and the filtering process is performed to obtain the processed pixel value.
[0095] Step S104, obtaining a sharpened image corresponding to the image to be processed based on the processed pixel values corresponding to the central pixel points of each target local area.
[0096] When the processed pixel value of each pixel in the image to be processed is determined, the pixel value of each pixel in the sharpened image corresponding to the image to be processed can be determined to generate the sharpened image.
[0097] Apply the image processing method provided in the embodiment of the present application to obtain guided filtering parameters corresponding to each target local area in the image to be processed; adjust the guided filtering parameters based on preset conditions to determine the target guided filtering parameters corresponding to the guided filtering parameters; filter the pixel values of the central pixel points of the target local area according to the target guided filtering parameters to obtain the processed pixel values of the central pixel points; obtain a sharpened image corresponding to the image to be processed based on the processed pixel values corresponding to the central pixel points of each target local area; wherein the preset conditions include that the guided filtering parameters are not lower than a first threshold corresponding to the image noise, and are not higher than a second threshold corresponding to the halo effect.
[0098] The embodiment of the present application achieves a balance between sharpening effect, image noise and halo effect by limiting the range of guided filter parameters, thereby improving the quality of image sharpening.
[0099] The present application provides an image processing method, such as Figure 2 As shown, including:
[0100] Step S201, obtaining guided filtering parameters corresponding to the target local area in the image to be processed.
[0101] The image to be processed is the original image that needs to be sharpened, and the local area is the area of a preset size centered on each pixel point determined when scanning the image to be processed. For a pair of images to be processed, the position of each pixel point is determined, and the target local area corresponding to each pixel point is obtained in turn.
[0102] The image processing method provided in the embodiment of the present application can be implemented by a preset image processing model. For example, the image to be processed is input into a preset image processing model, and the image processing model executes the steps in the image processing method, and finally outputs a processed sharpened image.
[0103] The image processing model is designed based on the guided filter. It can be understood that the image to be processed in the embodiment of the present application is the input image of the guided filter, and the sharpened image obtained after image processing is the output image of the guided filter. Correspondingly, the guided filter parameters are the linear coefficients in the linear function related to the image to be processed and the sharpened image. The guided filter parameters are obtained by calculating the pixel values of the pixel points in each target local area.
[0104] Therefore, by determining the pixel points in each target local area in the image to be processed, the guided filtering parameters corresponding to each target local area in the image to be processed can be obtained.
[0105] In a preferred embodiment of the present application, Figure 3 As shown, step S201 includes:
[0106] Step S2011, determining the infinite norm of the target local area.
[0107] In the traditional method of sharpening an image based on a guided filter, the corresponding guided filter parameters are obtained by calculating the variance in the local area of each pixel. When calculating the variance, multiple multiplication operations are involved, and the calculation complexity is high, resulting in high hardware pressure and low processing efficiency. Therefore, in the embodiment of the present application, the variance in the local area is not calculated, but the corresponding guided filter parameters are determined according to the infinite norm.
[0108] The method of calculating the variance in the local area also involves other pixel values except the maximum pixel value and the minimum pixel value in the mean operation. In the non-flat area of the image, when the noise is small, the other pixel values except the maximum pixel value and the minimum pixel value are mostly symmetrically distributed on both sides of the central pixel value. In other words, using the infinite norm to estimate here can reduce the computational complexity under the condition of very small error.
[0109] The formula for calculating the variance in a local area is:
[0110]
[0111] Where N is the number of pixels in the local area, X i is the difference between each pixel and the mean of all pixels in the local area;
[0112] The calculation formula of the second norm is:
[0113]
[0114] It can be seen that the variance can be expressed using the second norm:
[0115]
[0116] The infinity norm includes the positive infinity norm and the negative infinity norm. The calculation formula of the positive infinity norm is:
[0117] ||X|| ∞ =max i |X i |;
[0118] The calculation formula for negative infinity norm is:
[0119] ||X|| -∞ =min i |X i |;
[0120] From the calculation formula of the two norms and the positive and negative infinity norms, we can get the relationship between the variance and the positive and negative infinity norms as follows:
[0121] ||X|| -∞ 2 ≤var≤||X|| ∞ 2 ;
[0122] According to the pixel value of each pixel point in the local area, the infinite norm in the local area is determined, and the variance in the local area is approximately determined.
[0123] Step S2012, determining corresponding guided filtering parameters according to the infinite norm.
[0124] According to the relationship between the variance in the local area and the guided filtering parameters, the approximate relationship between the infinite norm and the guided filtering parameters can be obtained, and the guided filtering parameters corresponding to the infinite norm can be determined.
[0125] Generally speaking, when calculating the guided filter parameters based on the variance of the local area, the guided filter parameters are mean filtered. In the embodiment of the present application, this step is performed as an omitted step to facilitate implementation in ASIC hardware.
[0126] In a preferred embodiment of the present application, Figure 4As shown, step S2012 includes:
[0127] Step S20121, determining the mean of the infinite norm in the target local area according to the infinite norm, and determining the detail layer index and the edge layer index according to the mean of the infinite norm;
[0128] Among them, the mean of the infinite norm is used to replace the variance in the local area, and the variance in the target local area is the variance of the pixel values of all pixels in the target local area; there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect.
[0129] The difference between the pixel value of each pixel in the local area and the mean of the pixel value of each pixel is small. Therefore, the mean of the infinite norm in the local area can be used to replace the variance in the local area:
[0130] inf=(||X|| -∞ 2 +||X|| ∞ 2 ) / 2≈var
[0131] Furthermore, the relationship between the variance and the guided filter parameters is:
[0132] a=var / (var+∈)
[0133] Among them, ∈ is a value representing the filtering strength of the guided filter;
[0134] The relationship between the mean of the infinite norm and the guided filter parameter is:
[0135] a=inf / (inf+∈)
[0136] The local areas related to details in the image to be processed are divided into detail layers, and the local areas related to edges are divided into edge layers. The detail layer corresponds to image noise, and the edge layer corresponds to the halo effect.
[0137] According to the infinite norm, the mean of the infinite norm in the local area of the target is determined, including: according to the infinite norm of the local area of the detail layer, the mean of the infinite norm in the local area of the detail layer is determined, and according to the infinite norm of the local area of the edge layer, the mean of the infinite norm in the local area of the edge layer is determined. Then, according to the mean of the infinite norm in the local area of the detail layer and the mean of the infinite norm of the variance in the local area of the edge layer, the corresponding detail layer index and edge layer index are determined respectively. It can be seen that there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect.
[0138] Specifically, the detail layer index and the edge layer index refer to the index information of the lookup table. In the embodiment of the present application, in order to reduce the amount of calculation during image processing, the corresponding guide filter parameters are calculated in advance according to the mean of the possible infinite norm, and the guide filter parameters are stored in a preset lookup table. In this way, as long as the index information is determined, the guide filter parameters can be found in the corresponding lookup table, avoiding the need to recalculate the guide filter parameters according to the mean of the infinite norm every time the image to be processed is sharpened.
[0139] In a preferred embodiment of the present application, Figure 5 As shown, before step S2012, the method further includes:
[0140] Step S501, obtaining the mean of the infinite norm in the target local area in the sample image, and determining the corresponding guided filtering parameters according to the mean of the infinite norm.
[0141] Step S502: Create a lookup table according to the association between the mean of the infinite norm and the guided filtering parameters.
[0142] Before sharpening the image to be processed, a sample image is obtained in advance, and the mean of the infinite norm in each target local area in the sample image is determined, thereby determining the corresponding guided filter parameters. A lookup table of guided filter parameters is created, and based on the correlation between the mean of the infinite norm and the guided filter parameters, the guided filter parameters are stored in the lookup table, and corresponding index information is generated so that the guided filter parameters can be subsequently searched according to the index information.
[0143] The index information of the lookup table is:
[0144] id=(inf>>rshift)-offset
[0145] Among them, rshift is used to limit the value range of index information, and offset is used to limit the value range of guided filtering parameters.
[0146] Step S20122: Obtain corresponding guided filtering parameters from a preset lookup table according to the detail layer index and the edge layer index.
[0147] According to the detail layer index and the edge layer index, corresponding guided filtering parameters are obtained from a preset detail layer lookup table and an edge layer lookup table respectively.
[0148] It should be noted that, in the embodiment of the present application, in addition to the detail layer and the edge layer, other feature layers in the image to be processed may also be divided, and a corresponding lookup table may be pre-established.
[0149] Adjusting the guided filter parameters based on the preset conditions, and determining target guided filter parameters corresponding to the guided filter parameters;
[0150] The preset conditions include guiding the filtering parameter to be not lower than a first threshold corresponding to the image noise and not higher than a second threshold corresponding to the halo effect.
[0151] Image noise is unnecessary or redundant interference information in image data. The halo effect refers to the sharp overshoot or undershoot on both sides of the edge caused by excessive edge enhancement in image data, which is intuitively represented by white or black edges on the edge. Image sharpening includes enhancing the detail layer and edge layer in the image. Image sharpening will enhance both image noise and halo effect.
[0152] Therefore, in the embodiment of the present application, the value of the guided filter parameter is restricted, and a first threshold corresponding to the image noise and a second threshold corresponding to the halo effect are given, so as to balance the sharpening effect, image noise and halo effect and improve the quality of the sharpened image.
[0153] Step S202: If it is determined that the guided filtering parameter meets the preset condition, the guided filtering parameter is used as the target guided filtering parameter of the corresponding local area.
[0154] If it is determined that the guided filter parameters meet the preset conditions, that is, if it is determined that the guided filter parameters of the detail layer are not lower than the first threshold corresponding to the image noise, then the guided filter parameters are used as the target guided filter parameters of the corresponding local area; and if it is determined that the guided filter parameters of the edge layer are not higher than the second threshold corresponding to the halo effect, then the guided filter parameters are used as the target guided filter parameters of the corresponding local area. The first threshold corresponding to the image noise and the second threshold corresponding to the halo effect are obtained by analyzing the sharpened images corresponding to a large number of sample images.
[0155] For example, in one application scenario, a sharpened image that meets the requirements of the sharpening effect is determined, and then the guided filter parameters of each local area in the sharpened image are determined. By analyzing the guided filter parameters of a large number of sharpened images, a suitable first threshold and a suitable second threshold are determined. Specifically, a first threshold and a third threshold corresponding to image noise exist in the detail layer, wherein the first threshold is less than the third threshold; a second threshold and a fourth threshold corresponding to the halo effect exist in the edge layer, wherein the fourth threshold is less than the second threshold.
[0156] In another application scenario, different guided filter parameters are substituted into the sharpening functions of the sample image and the sharpened image to obtain different sharpening effects, and then the guided filter parameters are continuously adjusted according to the sharpening effect until a sharpened image that meets the sharpening effect requirements is obtained, and a suitable first threshold and a suitable second threshold are determined, which will not be repeated.
[0157] The steps of determining the first threshold and the second threshold can be implemented through a model, or the default values can be directly used. The first threshold can be set to 0.25 by default, and the second threshold can be set to 0.95 by default. The embodiments of the present application do not impose any restrictions on this.
[0158] It can be understood that the local area of the image to be processed in the embodiment of the present application includes the local area of the detail layer and the local area of the edge layer, and accordingly, the guided filtering parameters corresponding to the local area can also be divided into the guided filtering parameters of the detail layer and the guided filtering parameters of the edge layer. Then, when determining whether the guided filtering parameters meet the preset requirements, the guided filtering parameters of the detail layer can be compared with the first threshold corresponding to the detail layer, and the guided filtering parameters of the edge layer can be compared with the second threshold corresponding to the edge layer.
[0159] Step S203: If it is determined that the guided filtering parameter does not meet the preset condition, the preset guided filtering parameter is determined as the target guided filtering parameter corresponding to the target local area.
[0160] In a preferred embodiment of the present application, the guided filtering parameters include guided filtering parameters of at least one detail layer and guided filtering parameters of at least one edge layer;
[0161] Step S202 and step S203 include:
[0162] If it is determined that the guided filter parameters of the detail layer are not lower than the first threshold, the guided filter parameters of the detail layer are used as the target guided filter parameters of the corresponding target local area; if it is determined that the guided filter parameters of the detail layer are lower than the first threshold, the first threshold is used as the target guided filter parameters of the corresponding target local area.
[0163] If it is determined that the guided filtering parameters of the edge layer are not higher than the second threshold, the guided filtering parameters of the edge layer are used as the target guided filtering parameters of the corresponding target local area; if it is determined that the guided filtering parameters of the edge layer are higher than the second threshold, the target guided filtering parameters of the corresponding target local area are taken as 1.
[0164] If it is determined that the guided filter parameters do not meet the preset conditions, that is, the guided filter parameters of the detail layer are lower than the first threshold, then the preset guided filter parameters, that is, the value of the first threshold, are determined as the target guided filter parameters corresponding to the target local area; or if the guided filter parameters of the edge layer are higher than the second threshold, then the value of the target guided filter parameters corresponding to the target local area is determined to be 1.
[0165] It should be noted that the preset guide filter parameters of the detail layer are different from the guide filter parameters of the edge layer.
[0166] According to the relationship between the mean of the infinite norm and the guided filter parameter, it can be determined that: if the value of ∈ is reduced, the guided filter parameter will also be reduced when the mean of the infinite norm is smaller, and the guided filter parameter will remain unchanged when the mean of the infinite norm is larger, that is, the image noise will be amplified when the mean of the infinite norm is smaller; if the value of ∈ is increased, the guided filter parameter will remain unchanged when the mean of the infinite norm is smaller, and the halo effect will be enhanced when the mean of the infinite norm is larger.
[0167] Therefore, when the guided filter parameter of the detail layer is lower than the first threshold, the preset first threshold is used as the target guided filter parameter, so that the image noise when the mean of the infinite norm is small will not be amplified; when the guided filter parameter of the edge layer is higher than the second threshold, the target guided filter parameter is set to 1, so that the halo effect when the mean of the infinite norm is large will not be enhanced.
[0168] Step S204, filtering the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point.
[0169] After determining the target guided filtering parameters of each target local area, the sharpening function corresponding to the central pixel point of each target local area can be determined. The pixel value of each pixel point is introduced into the corresponding sharpening function, and the filtering process is performed to obtain the processed pixel value.
[0170] In a preferred embodiment of the present application, Figure 7 As shown, step S204 includes:
[0171] Step S2041, obtaining the sharpening intensity coefficient of the detail layer and the sharpening intensity coefficient of the edge layer.
[0172] Step S2042, determines the sharpening function of the filtering processing according to the guiding filtering parameters of the detail layer, the sharpening intensity coefficient of the detail layer, the guiding filtering parameters of the edge layer and the sharpening intensity coefficient of the edge layer, introduces the pixel value of the central pixel point of the target local area into the sharpening function, and obtains the processed pixel value of the central pixel point.
[0173] Obtain the sharpening strength coefficient of the detail layer and the sharpening strength coefficient of the edge layer, wherein the sharpening strength coefficient can be set according to the needs of different users. In the embodiment of the present application, the guided filtering parameters are determined based on the mean of the infinite norm, eliminating the step of mean filtering the guided filtering parameters. Therefore, based on the relationship between the input image and the output image in the guided filtering method: The sharpening function can be determined as:
[0174]
[0175] Where I is the image to be processed, For the detail layer, is the edge layer, k1 is the sharpening strength coefficient of the detail layer, k2 is the sharpening strength coefficient of the edge layer, is the mean value of each pixel in the local area.
[0176] Step S205 , obtaining a sharpened image corresponding to the image to be processed based on the processed pixel values corresponding to the central pixel points of each target local area.
[0177] When the processed pixel value of each pixel in the image to be processed is determined, the pixel value of each pixel in the sharpened image corresponding to the image to be processed can be determined to generate the sharpened image.
[0178] Apply the image processing method provided in the embodiment of the present application to obtain guided filtering parameters corresponding to the target local area in the image to be processed; if it is determined that the guided filtering parameters meet the preset conditions, the guided filtering parameters are used as the target guided filtering parameters of the corresponding target local area; if it is determined that the guided filtering parameters do not meet the preset conditions, the preset guided filtering parameters are determined as the target guided filtering parameters corresponding to the target local area; the pixel values of the central pixel points of the target local area are filtered according to the target guided filtering parameters to obtain the processed pixel values of the central pixel points; based on the processed pixel values corresponding to the central pixel points of each target local area, a sharpened image corresponding to the image to be processed is obtained; wherein the preset conditions include that the guided filtering parameters are not lower than a first threshold corresponding to the image noise, and are not higher than a second threshold corresponding to the halo effect.
[0179] The embodiment of the present application performs sharpening processing on the processed image based on the guided filter, limits the range of the guided filter parameters in the sharpening function of the sharpening processing, and directly performs sharpening processing on the processed image according to the guided filter parameters when the guided filter parameters do not exceed the threshold range. When the guided filter parameters exceed the threshold range, the preset target guided filter parameters are used to sharpen the processed image, thereby achieving a balance between the sharpening effect, image noise and halo effect, and improving the quality of image sharpening. At the same time, the embodiment of the present application uses an infinite norm to replace the variance in the local area, reduces the computational complexity in the image processing process, and uses a lookup table to replace the calculation process of the guided filter parameters, which greatly reduces the amount of calculation, reduces the hardware pressure, and improves the efficiency of image processing.
[0180] An image processing device is provided in an embodiment of the present application. Figure 6 As shown, including:
[0181] An acquisition module 601 is used to acquire guided filtering parameters corresponding to a target local area in an image to be processed;
[0182] A determination module 602 is used to adjust the guided filter parameter based on a preset condition and determine a target guided filter parameter corresponding to the guided filter parameter;
[0183] A processing module 603 is used to filter the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point;
[0184] A generating module 604, configured to obtain a sharpened image corresponding to the image to be processed based on the processed pixel values corresponding to the central pixel points of each target local area;
[0185] The preset conditions include guiding the filtering parameter to be not lower than a first threshold corresponding to the image noise and not higher than a second threshold corresponding to the halo effect.
[0186] Optionally, the guided filtering parameters include guided filtering parameters of at least one detail layer and guided filtering parameters of at least one edge layer;
[0187] The determination module 602 is specifically used for:
[0188] If it is determined that the guided filter parameter of the detail layer is not lower than the first threshold, the guided filter parameter of the detail layer is used as the target guided filter parameter of the corresponding target local area;
[0189] If it is determined that the guided filter parameter of the detail layer is lower than the first threshold, the first threshold is used as the target guided filter parameter of the corresponding target local area.
[0190] Optionally, the determination module 602 is further configured to:
[0191] If it is determined that the guided filter parameter of the edge layer is not higher than the second threshold, the guided filter parameter of the edge layer is used as the target guided filter parameter of the corresponding target local area;
[0192] If it is determined that the guided filter parameter of the edge layer is higher than the second threshold, the target guided filter parameter of the corresponding target local area takes a value of 1.
[0193] Optionally, the acquisition module 601 includes:
[0194] A first acquisition submodule is used to determine the infinite norm of the target local area;
[0195] The second acquisition submodule is used to determine the corresponding guided filtering parameters according to the infinite norm.
[0196] Optionally, the second acquisition submodule includes:
[0197] A first acquisition unit is used to determine a mean value of the infinite norm in the target local area according to the infinite norm, and determine a detail layer index and an edge layer index according to the mean value of the infinite norm;
[0198] Among them, the mean of the infinite norm is used to replace the variance in the local area, and the variance in the target local area is the variance of the pixel values of all pixels in the target local area; there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect;
[0199] The second acquisition unit is used to acquire corresponding guided filtering parameters from a preset lookup table according to the detail layer index and the edge layer index.
[0200] Optionally, the device further comprises:
[0201] A pre-acquisition module, used to obtain the mean of the infinite norm in the local target area in the sample image, and determine the corresponding guided filtering parameters according to the mean of the infinite norm;
[0202] A creation module is used to create a lookup table according to the correlation between the mean of the infinite norm and the guided filtering parameters.
[0203] Optionally, the processing module 603 includes:
[0204] A first processing submodule is used to obtain a sharpening intensity coefficient of a detail layer and a sharpening intensity coefficient of an edge layer;
[0205] The second processing sub-module is used to determine the sharpening function of the filtering processing according to the guiding filtering parameters of the detail layer, the sharpening intensity coefficient of the detail layer, the guiding filtering parameters of the edge layer and the sharpening intensity coefficient of the edge layer, and introduce the pixel value of the central pixel point of the target local area into the sharpening function to obtain the processed pixel value of the central pixel point.
[0206] An image processing device provided in an embodiment of the present application is used to obtain guided filtering parameters corresponding to a target local area in an image to be processed; the guided filtering parameters are adjusted based on preset conditions to determine target guided filtering parameters corresponding to each guided filtering parameter; pixel values of central pixels of the target local area are filtered according to the target guided filtering parameters to obtain processed pixel values of the central pixels; a sharpened image corresponding to the image to be processed is obtained based on the processed pixel values corresponding to the central pixels of each target local area; wherein the preset conditions include that the guided filtering parameters are not lower than a first threshold corresponding to image noise, and not higher than a second threshold corresponding to the halo effect.
[0207] The embodiment of the present application performs sharpening processing on the processed image based on the guided filter, limits the range of the guided filter parameters in the sharpening function of the sharpening processing, and directly performs sharpening processing on the processed image according to the guided filter parameters when the guided filter parameters do not exceed the threshold range. When the guided filter parameters exceed the threshold range, the preset target guided filter parameters are used to sharpen the processed image, thereby achieving a balance between the sharpening effect, image noise and halo effect, and improving the quality of image sharpening. At the same time, the embodiment of the present application uses an infinite norm to replace the variance in the local area, reduces the computational complexity in the image processing process, and uses a lookup table to replace the calculation process of the guided filter parameters, which greatly reduces the amount of calculation, reduces the hardware pressure, and improves the efficiency of image processing.
[0208] An electronic device (computer device / equipment / system) is provided in an embodiment of the present application, including a memory, a processor and a computer program stored in the memory, and the processor executes the above-mentioned computer program to implement the steps of the image processing method. Compared with the related art, it can achieve: by limiting the range of the guided filter parameters, a balance between the sharpening effect, image noise and the halo effect is achieved, thereby improving the quality of image sharpening.
[0209] In an alternative embodiment, an electronic device is provided, such as Figure 7 As shown, Figure 7The electronic device 7000 shown includes: a processor 7001 and a memory 7003. The processor 7001 and the memory 7003 are connected, such as through a bus 7002. Optionally, the electronic device 7000 may also include a transceiver 7004, which may be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception. It should be noted that in actual applications, the transceiver 7004 is not limited to one, and the structure of the electronic device 7000 does not constitute a limitation on the embodiments of the present application.
[0210] Processor 7001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 7001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0211] The bus 7002 may include a path to transmit information between the above components. The bus 7002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 7002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0212] The memory 7003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, without limitation herein.
[0213] The memory 7003 is used to store the computer program for executing the embodiment of the present application, and the execution is controlled by the processor 7001. The processor 7001 is used to execute the computer program stored in the memory 7003 to implement the steps shown in the above method embodiment.
[0214] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps and corresponding contents of the aforementioned method embodiment can be implemented.
[0215] The embodiment of the present application also provides a computer program product, including a computer program, which can implement the steps and corresponding contents of the aforementioned method embodiment when executed by a processor.
[0216] The terms "first", "second", "third", "fourth", "1", "2", etc. (if any) in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described in the drawings.
[0217] It should be understood that, although each operation step is indicated by arrows in the flowchart of the embodiment of the present application, the implementation order of these steps is not limited to the order indicated by the arrows. Unless clearly stated herein, in some implementation scenarios of the embodiment of the present application, the implementation steps in each flowchart can be performed in other orders according to demand. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on actual implementation scenarios. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage in these sub-steps or stages may also be executed at different times respectively. In different scenarios of execution time, the execution order of these sub-steps or stages may be flexibly configured according to demand, and the embodiment of the present application does not limit this.
[0218] The above is only an optional implementation method for some implementation scenarios of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the solution of the present application, other similar implementation methods based on the technical ideas of the present application are also within the protection scope of the embodiments of the present application.
Claims
1. An image processing method, characterized in that: include: Obtaining guided filtering parameters corresponding to the target local area in the image to be processed; Adjusting the guided filtering parameter based on a preset condition to determine a target guided filtering parameter corresponding to the guided filtering parameter; Performing filtering processing on the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point; Based on the processed pixel values corresponding to the central pixel points of each target local area, a sharpened image corresponding to the image to be processed is obtained; wherein the preset conditions include that the guided filter parameter is not lower than a first threshold corresponding to the image noise, and is not higher than a second threshold corresponding to the halo effect; The step of obtaining the guided filtering parameters corresponding to the target local area in the image to be processed includes: Determining the infinity norm of the target local area; Determining a mean value of the infinite norm in the target local area according to the infinite norm, and determining a detail layer index and an edge layer index according to the mean value of the infinite norm; Among them, the mean of the infinite norm is used to replace the variance in the local area, and the variance in the target local area is the variance of the pixel values of all pixels in the target local area; there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect; The corresponding guided filtering parameters are obtained from a preset lookup table according to the detail layer index and the edge layer index.
2. The image processing method according to claim 1, characterized in that: The guided filtering parameters include guided filtering parameters of at least one detail layer and guided filtering parameters of at least one edge layer; The step of adjusting the guided filtering parameter based on the preset condition and determining the target guided filtering parameter corresponding to the guided filtering parameter includes: If it is determined that the guided filter parameter of the detail layer is not lower than the first threshold, taking the guided filter parameter of the detail layer as the target guided filter parameter of the corresponding target local area; If it is determined that the guided filtering parameter of the detail layer is lower than the first threshold, the first threshold is used as the target guided filtering parameter of the corresponding target local area.
3. The image processing method according to claim 2, characterized in that: The adjusting the guided filtering parameter based on the preset condition to determine the target guided filtering parameter corresponding to the guided filtering parameter also includes: If it is determined that the guided filtering parameter of the edge layer is not higher than the second threshold, taking the guided filtering parameter of the edge layer as the target guided filtering parameter of the corresponding target local area; If it is determined that the guided filtering parameter of the edge layer is higher than the second threshold, the target guided filtering parameter of the corresponding target local area takes a value of 1.
4. The image processing method according to claim 1, characterized in that: The lookup table is created in the following way: Obtaining a mean value of the infinite norm in the target local area in the sample image, and determining a corresponding guided filtering parameter according to the mean value of the infinite norm; The lookup table is created according to the association between the mean of the infinity norm and the guided filtering parameter.
5. The image processing method according to claim 4, characterized in that: The filtering process is performed on the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point, including: Obtaining a sharpening intensity coefficient of the detail layer and a sharpening intensity coefficient of the edge layer; According to the guiding filter parameters of the detail layer, the sharpening intensity coefficient of the detail layer, the guiding filter parameters of the edge layer and the sharpening intensity coefficient of the edge layer, the sharpening function of the filtering process is determined, and the pixel value of the central pixel point of the target local area is introduced into the sharpening function to obtain the processed pixel value of the central pixel point.
6. An image processing device, characterized in that: include: An acquisition module is used to acquire the guided filtering parameters corresponding to the target local area in the image to be processed; A determination module, configured to adjust the guide filter parameter based on a preset condition and determine a target guide filter parameter corresponding to the guide filter parameter; A processing module, used for filtering the pixel value of the central pixel point of the target local area according to the target guided filtering parameter to obtain the processed pixel value of the central pixel point; A generating module, configured to obtain a sharpened image corresponding to the image to be processed based on the processed pixel values corresponding to the central pixel points of each target local area; wherein the preset conditions include that the guided filtering parameter is not lower than a first threshold corresponding to the image noise, and is not higher than a second threshold corresponding to the halo effect; Wherein, the acquisition module is specifically used for: Determining the infinity norm of the target local area; Determining a mean value of the infinite norm in the target local area according to the infinite norm, and determining a detail layer index and an edge layer index according to the mean value of the infinite norm; Among them, the mean of the infinite norm is used to replace the variance in the local area, and the variance in the target local area is the variance of the pixel values of all pixels in the target local area; there is a mapping relationship between the detail layer index and the detail layer corresponding to the image noise; there is a mapping relationship between the edge layer index and the edge layer corresponding to the halo effect; The corresponding guided filtering parameters are obtained from a preset lookup table according to the detail layer index and the edge layer index.
7. A computer program product, characterized in that The method comprises computer program instructions, wherein the computer program instructions enable a computer to execute the image processing method according to any one of claims 1 to 5.
8. An electronic device, characterized in that: The electronic device comprises: one or more processors; Memory; One or more application programs, wherein the one or more application programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs are configured to: execute the image processing method according to any one of claims 1 to 5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 5 is implemented.
