Image processing method, device and medium

Abstract

The application discloses an image processing method, device and medium, wherein the method comprises the following steps: taking an original image to be processed as a guide image, performing guide filtering on the original image to obtain a first denoising image; obtaining an enhanced image based on high-frequency information in the first denoising image, wherein the enhanced image contains enhanced texture information of the first denoising image; updating the first denoising image by using the enhanced image to obtain a second denoising image; taking the second denoising image as an updated guide image, and returning to perform guide filtering on the original image until a loop ending condition is reached to obtain a target image. The high-frequency information is iteratively added to the guide image, so that the result of the guide filtering retains sufficient sharpness.

Description

Technical Field

[0001] This disclosure generally relates to the field of image processing technology, and specifically to an image processing method, apparatus, and medium. Background Technology

[0002] Guided filtering is an image filtering technique that uses a guide map to filter the input image, making the final output image largely similar to the input image, but with textures similar to the guide map.

[0003] In related technologies, since a suitable guide map is often unavailable when performing guided filtering, the input image itself is typically used as the guide map for filtering. However, the input image itself usually contains a lot of noise, making the filtered image still relatively blurry, i.e., lacking clear texture information. Summary of the Invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an image processing method, device and medium that can maintain sufficient sharpness in the result of guided filtering by iteratively adding high-frequency information to the guide image.

[0005] On one hand, embodiments of this application provide an image processing method, including:

[0006] The original image to be processed is used as a guide image, and guide filtering is performed on the original image to obtain the first denoised image;

[0007] Based on the high-frequency information in the first denoised image, an enhanced image is obtained, wherein the enhanced image contains texture information of the first denoised image after enhancement.

[0008] The enhanced image is used to update the first denoised image to obtain the second denoised image;

[0009] The second denoised image is used as the updated guide image, and the process of performing guide filtering on the original image is repeated until the loop ends, thus obtaining the target image.

[0010] On one hand, embodiments of this application provide an image processing apparatus, including:

[0011] The filtering module is used to use the original image to be processed as a guide image, and to perform guided filtering on the original image to obtain a first denoised image;

[0012] The acquisition module is used to acquire an enhanced image based on the high-frequency information in the first denoised image, wherein the enhanced image contains texture information of the first denoised image after enhancement;

[0013] The update module is used to update the first denoised image using the enhanced image to obtain a second denoised image;

[0014] The loop module is used to take the second denoised image as the updated guide map, return to perform guide filtering on the original image, until the condition for the loop to end is met, and obtain the target image.

[0015] On one hand, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in embodiments of this application.

[0016] On one hand, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in embodiments of this application.

[0017] The image processing method of this application determines an enhanced image for texture information enhancement based on the guided filtering result of the original image itself. Then, a second denoised image for further guided filtering is obtained based on the enhanced image. The noise in the second denoised image is effectively eliminated by recursion, so that the second denoised image, which is the updated guided map, has stronger high-frequency information and effectively retains richer edge information. As a result, the first denoised image obtained based on the updated guided map can have higher sharpness and retain more edge information.

[0018] When acquiring the target image, the system determines whether to output the current first denoised image as the final filtering result of the original image based on the sharpness of the first denoised image obtained by guided filtering using the guided map. If the sharpness of the first denoised image does not reach a preset threshold, the system continues to construct a guided map based on the high-frequency information of the current first denoised image. That is, the high-frequency information of the guided map is gradually enhanced through iterative operations, so that the guided filtering result retains sufficient sharpness as the high-frequency information of the guided map is enhanced, or even achieves sharpness enhancement, thereby achieving detail preservation and detail enhancement of the filtering result.

[0019] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0021] Figure 1 A schematic diagram of the structure of a computer system suitable for implementing electronic devices or servers according to embodiments of this application is shown;

[0022] Figure 2 A flowchart of an image processing method according to an embodiment of this application is shown;

[0023] Figure 3 A flowchart of an image processing method according to another embodiment of this application is shown;

[0024] Figure 4 A flowchart of an image processing method according to another embodiment of this application is shown;

[0025] Figure 5 A flowchart of an image processing method according to another embodiment of this application is shown;

[0026] Figure 6 A flowchart of an image processing method according to another embodiment of this application is shown;

[0027] Figure 7 A block diagram of an image processing apparatus according to an embodiment of this application is shown. Detailed Implementation

[0028] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security and prevention, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial recognition payment, facial recognition unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beauty filters, cosmetics, medical aesthetics, and intelligent temperature measurement.

[0029] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0030] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] Guided filtering is a typical spatial denoising algorithm guided by additional prior information. Its denoising performance largely depends on the usefulness of this prior information. When additional information is lacking, guided filtering often uses the image itself as its guide map. Due to noise in the image, when guided filtering uses the image itself as its guide map, the denoised image is usually blurry and fails to preserve image details.

[0032] Based on this, this application proposes an image processing method, device, and medium that can construct a guide map containing more texture information by using the image to be filtered itself as a guide map in a denoised image, thereby enabling the filtering result based on the constructed guide map to retain more texture details.

[0033] The image processing method provided in this application can be applied to, for example... Figure 1 The computer device 10 shown can be illustrated as follows: The internal structure of the computer device 10 can be shown as follows: Figure 1 As shown. The computer system includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 102 or programs loaded from storage section 108 into random access memory (RAM) 103. RAM 103 also stores various programs and data required for the system's operating instructions. The CPU 101, ROM 102, and RAM 103 are interconnected via bus 104. Input / output (I / O) interface 105 is also connected to bus 104.

[0034] The following components are connected to I / O interface 105: an input section 106 including a keyboard, mouse, etc.; an output section 107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN card, modem, etc. The communication section 109 performs communication processing via a network such as the Internet. Drive 110 is also connected to I / O interface 105 as needed. Removable media 111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 110 as needed so that computer programs read from them can be installed into storage section 108 as needed.

[0035] In one embodiment of this application, computer device 10 can construct a new guide map based on the original image using its own denoised image for image filtering processing.

[0036] Figure 2 A flowchart of an image processing method according to an embodiment of this application is shown. This method can be applied to... Figure 1 The computer device 10 shown. (e.g.) Figure 2 As shown, the method includes the following steps:

[0037] Step 201: Using the original image to be processed as a guide image, guide filtering is performed on the original image to obtain the first denoised image.

[0038] It should be noted that the original image can be an image or a video image. In the embodiments of this application, the original image can be an image acquired by an image acquisition device or an image acquired from a storage device. This application does not specify the source and format of the original image.

[0039] Guided filtering is an image denoising method that uses a guide image to build a filter. Specifically, it determines a local linear model between the guide image I and the filtered output image (the first denoised image Q), and then uses a local window to filter the original image so that the edge information in the output image comes from and only from the guide image I.

[0040] As a feasible embodiment, such as Figure 3 As shown, the original image to be processed is used as the guide image, and guided filtering is performed on the original image to obtain the first denoised image, including:

[0041] Step 301: Perform mean filtering on the original image according to the preset spatial filtering radius to obtain the first image after mean filtering.

[0042] It should be noted that the spatial filtering radius *r* is the size of the local window used to filter the original image. The local window is a rectangular area with a radius equal to *r*. The spatial filtering radius *r* can be set according to the content of the original image. The larger the value of the spatial filtering radius *r*, the higher the degree of image blurring; the smaller the spatial filtering radius *r*, the lower the image model performance. For example, when performing mean filtering on the original image according to the preset spatial filtering radius *r*, the pixels in the original image *P* can be used as the center point. Then, the points within the rectangular area with a radius equal to *r* are obtained, and the pixel values ​​of the points within the rectangular area are averaged. The result is used as the filtering result for the center point, and the image formed by the filtering result for the center point is used as the first image *mean_p*, where *p* is a pixel in the original image *P*.

[0043] Step 302: Determine the second image corresponding to the guide image based on the first image.

[0044] It should be understood that when the guide image is the original image, the mean-filtered image corresponding to guide image I is the same as the filtered image of the original image. Therefore, the second image of the mean-filtered image corresponding to guide image I is mean_i = mean_p, where i is a pixel in guide image I.

[0045] Step 303: Calculate the variance diagram corresponding to the guide map, and calculate the covariance diagram between the first image and the guide map based on the first image and the second image.

[0046] As one possible implementation, before calculating the variance of the guide map, it is necessary to calculate the correlation matrix corr_i of the filtered image (second image) of the guide map, corr_i = boxfilter(i.*i), where boxfilter is the mean filter and * indicates that the corresponding pixels are multiplied. Then, based on the correlation matrix corr_i of the guide map, the atrial difference map var_i corresponding to the guide map is calculated, var_i = corr_i - mean_i.*mean_1.

[0047] Accordingly, we first need to obtain the correlation matrix corr_ip between the guide map I and the original image P, corr_ip = boxfilter(i.*i). Then, we calculate the covariance map cov_ip between the first image and the second image based on the mean-filtered image mean_i of the guide map I and the mean-filtered image mean_p of the original image P, cov_ip = corr_ip - mean_i.*mean_p.

[0048] Step 304: Based on the variance map and covariance map, the first denoised image is obtained.

[0049] It should be noted that the variance map and covariance map are used to characterize the discreteness between each pixel in the original image P and the guide map I. In order to calculate and obtain the first denoised image, it is also necessary to calculate the parameters in the specific local linear model based on the variance map and covariance map, namely, the first correlation factor a and the second correlation factor b.

[0050] As one possible implementation, such as Figure 4 As shown, based on the variance map and covariance map, the first denoised image is obtained, including:

[0051] Step 401: Determine the first correlation factor based on the variance plot, covariance plot, and spatial filtering intensity.

[0052] It should be noted that the first correlation factor 'a' is used to determine the texture intensity of a pixel, a = cov_ip. / (var_i + eps), where '. / ' represents element-wise division, and eps is the spatial filtering intensity, used to adjust the intensity of information enhancement. The setting of the spatial filtering intensity eps can be set according to the graphic characteristics of the original image. The larger the value of the spatial filtering intensity eps, the greater the image blurring, and the smaller the value of the spatial filtering intensity eps, the less the image blurring.

[0053] It should be noted that for smooth regions in an image, that is, regions without abrupt changes or edge regions, the variance (cov) will be very small, causing the first correlation factor a to approach 0. Conversely, edge regions or regions with abrupt changes in the image will cause the first correlation factor a to approach 1.

[0054] Step 402: Determine the second correlation factor based on the first image and the first correlation factor.

[0055] In other words, based on the first correlation factor a and the first image mean_p, the second correlation factor b is obtained, where b = mean_p - a.*mean_i.

[0056] In some embodiments, the first correlation factor a can be considered as high-frequency information related to the original image P and the guide map I, and the second correlation factor b is low-frequency information related to the original image P and the guide map I.

[0057] Step 403: Based on the first correlation factor and the second correlation factor, the first denoised image is obtained.

[0058] As one possible implementation, the mean values ​​of the first correlation factor a and the second correlation factor b are calculated separately to obtain the mean value of the first correlation factor a, mean_a, and the mean value of the second correlation factor b, mean_b. The mean value of the first correlation factor a, mean_a, is fused with the guide map I to obtain the high-frequency information of the first denoised image. Then, the high-frequency information of the first denoised image is superimposed with the mean value of the second correlation factor b, mean_b, to obtain the final first denoised image Q, q = mean_a.*i + mean_b.

[0059] Step 202: Based on the high-frequency information in the first denoised image, obtain an enhanced image, which contains the enhanced texture information of the first denoised image.

[0060] It should be noted that, for images, high-frequency information is usually not directly obtainable, but rather obtained through low-frequency information and the image itself.

[0061] As one possible embodiment, obtaining an enhanced image based on a first denoised image includes: performing mean filtering on the first denoised image to obtain low-frequency information of the first denoised image; performing difference calculation on the low-frequency information of the first denoised image and the first denoised image to obtain high-frequency information of the first denoised image; and using intensity control parameters to perform intensity correction on the high-frequency information to obtain an enhanced image, wherein the intensity control parameters are used to characterize the multiple by which the high-frequency information needs to be enhanced.

[0062] For example, the first denoised image Q can be first subjected to mean filtering to obtain the low-frequency information lp of the first denoised image Q, where lp = boxfilter(q). Then, the difference between the first denoised image Q and the low-frequency information lp is calculated to obtain the high-frequency information hp, where hp = q - lp. The intensity of the high-frequency information hp is then corrected using the intensity control parameter alpha to obtain the enhanced image, that is, alpha.*hp is calculated.

[0063] It should be noted that the intensity control parameter is positively correlated with the preset threshold used to determine whether the first denoised image meets the filtering requirements. That is, when the required sharpness of the first denoised image is high, a higher intensity control parameter can be set, and when the required sharpness of the first denoised image is low, a lower intensity control parameter can be set.

[0064] The intensity control parameter is a value greater than 1. That is, the intensity control parameter is used to improve the expression of high-frequency information in order to improve the sharpness of the enhanced image, that is, to improve the texture expression ability of the enhanced image.

[0065] Step 203: Update the first denoised image using the enhanced image to obtain the second denoised image.

[0066] As one possible implementation, the first denoised image can be updated by using the enhanced image, which can be done by superimposing the enhanced image and the first denoised image Q to obtain the second denoised image S, where s = q + alpha.*hp.

[0067] Step 204: Use the second denoised image as the updated guide image, return to perform guide filtering on the original image until the loop ends, and obtain the target image.

[0068] The target image is the final image obtained by applying guided filtering to the original image.

[0069] In other words, after each guided filtering of the original image to obtain the first denoised image, it is further determined whether the loop termination condition has been met. If the loop termination condition is met, the first denoised image is output as the target image. If the loop termination condition is not met, the enhanced image is calculated based on the current first denoised image, and the second denoised image is obtained. The new second denoised image is then used as the guide image to continue guided filtering of the original image until the loop termination condition is met, and the target image is output.

[0070] One possible implementation is to assign the second denoised image Q to the guide map, i.e., let i = s. Then, control the original image to perform guided filtering based on the new guide map I to obtain a new first denoised image, thus realizing a new guided filtering cycle.

[0071] One possible implementation is that the loop terminates when the target number of guided filtering operations on the original image is reached. In other words, when performing guided filtering on the original image, the number of times guided filtering is performed needs to be counted. After acquiring the first denoised image each time, it is determined whether the current count of guided filtering has reached the target number. If the target number is reached, the first denoised image is output as the target image. If the target number is not reached, the enhanced image is calculated based on the current first denoised image, and a second denoised image is acquired. This new second denoised image is then used as the guide image to continue guided filtering on the original image until the target number of iterations is reached, at which point the target image is output.

[0072] It should be understood that the target number of the guided filtering can be a threshold obtained through a finite number of experiments using related or similar images, or a threshold obtained through a finite number of computer simulations. This application does not make any specific limitation.

[0073] As one possible implementation, the loop can end when the sharpness of the first denoised image reaches the target threshold.

[0074] It should be noted that the sharpness requirements in the first denoised image vary depending on the content represented by the image. For example, in Van Gogh's "Starry Night", the texture of the entire image is relatively rich and evenly distributed, so more texture needs to be retained, making it suitable for sharpness judgment of the entire image. In contrast, medical images have less texture and are concentrated in the central area of ​​the image, so less texture needs to be retained, making it suitable for sharpness judgment of specific areas.

[0075] After obtaining the new first denoised image, it is necessary to further determine whether the sharpness of the first denoised image reaches the target threshold. If the sharpness of the first denoised image reaches the target threshold, the loop stops and the first denoised image is output. That is, the currently obtained first denoised image is used as the target image for output. If the sharpness of the first denoised image does not reach the target threshold, the enhanced image is calculated based on the current first denoised image, and the second denoised image is obtained. The new second denoised image is used as the guide image to continue to perform guided filtering on the original image until the sharpness reaches the target threshold, and the target image is obtained for output.

[0076] As one possible implementation, when judging the sharpness of the first denoised image as a whole, the energy distribution statistics corresponding to the first denoised image can be obtained. If the energy distribution statistics are greater than or equal to the target threshold, it is determined that the sharpness of the first denoised image has reached the preset threshold; if the energy distribution statistics are less than the target threshold, it is determined that the sharpness of the first denoised image has not reached the target threshold.

[0077] The sharpness of the first denoised image can be represented by point sharpness, and the calculation formula is as follows:

[0078]

[0079] Where m and n are the length and width of the first denoised image, df is the grayscale change amplitude, and dx is the distance increment between pixels. The formula can be described as follows: for each pixel in the image, subtract 8 neighboring points from it, first calculate the weighted sum of the 8 differences (the weight depends on the distance; the greater the distance, the smaller the weight), then add the values ​​obtained from all points and divide by the total number of pixels. The formula can be understood as a statistical analysis of the grayscale diffusion degree around each pixel in the image; that is, the more intense the diffusion, the larger the value, and the clearer the image. From another perspective, this algorithm can be approximately equivalent to a statistical analysis of the energy distribution of the point spread function of the first denoised image.

[0080] For example, after obtaining the first denoised image, the energy distribution statistics of the point expansion of the first denoised image are calculated using the point sharpness formula. If the statistical result is greater than or equal to the target threshold, it means that the overall sharpness of the first denoised image meets the enhancement requirements and can be output as the target image. If the statistical result is less than the target threshold, it means that the overall sharpness of the first denoised image does not meet the enhancement requirements. Then, based on the current first denoised image, the enhancement image is calculated and the second denoised image is obtained. The new second denoised image is used as a guide image to continue to perform guided filtering on the original image until the sharpness reaches the target threshold, and the target image is output.

[0081] As one possible implementation, when judging the sharpness of the first denoised image portion, target pixels can be determined based on the original image. These target pixels are the pixels used for sharpness judgment. For each target pixel, a target threshold is set. The sharpness value corresponding to each target pixel is calculated. If the sharpness value corresponding to each target pixel is greater than or equal to the target threshold, it is determined that the sharpness of the first denoised image has reached the target threshold. If the sharpness value corresponding to any target pixel is less than the target threshold, it is determined that the sharpness of the first denoised image has not reached the target threshold.

[0082] In other words, the target threshold can be set independently based on the area in the original image where details need to be preserved. That is, you can first select at least one target pixel that needs to be preserved or enhanced based on the image content recorded in the original image, and then set the target threshold based on the at least one target pixel. For example, you can set a corresponding target threshold for each target pixel, or you can set a uniform target threshold for all target pixels.

[0083] For example, before filtering the original image, target pixels for sharpness assessment can be selected based on the original image, and a target threshold can be set for each target pixel. After obtaining the first denoised image, the target pixels are located on the first denoised image, and the sharpness value of each target pixel is calculated based on the first denoised image. Then, for each target pixel, the relationship between the sharpness value and the target threshold is determined. If the sharpness value of each target pixel is greater than or equal to the target threshold, it means that the sharpness of the target pixels in the first denoised image meets the requirements, and the first denoised image can be output as the target image. If the sharpness value of at least one target pixel is less than the target threshold, the enhancement image is calculated based on the current first denoised image, and a second denoised image is obtained. The new second denoised image is then used as a guide image to continue guiding filtering of the original image until the sharpness reaches the target threshold, and the target image is output.

[0084] The image processing method of this application determines an enhanced image for texture information enhancement based on the guided filtering result of the original image itself. Then, a second denoised image for further guided filtering is obtained based on the enhanced image. The noise in the second denoised image is effectively eliminated by recursion, so that the second denoised image, which is the updated guided map, has stronger high-frequency information and effectively retains richer edge information. As a result, the first denoised image obtained based on the updated guided map can have higher sharpness and retain more edge information.

[0085] When obtaining the final filtering result of the original image, the system determines whether to output the current first denoised image as the final filtering result of the original image based on the sharpness of the first denoised image obtained by guided filtering based on the guided map. If the sharpness of the first denoised image does not reach the target threshold, the system continues to construct a guided map based on the high-frequency information of the current first denoised image. That is, the high-frequency information of the guided map is gradually enhanced through iterative operations, so that the guided filtering result retains sufficient sharpness as the high-frequency information of the guided map is enhanced, or even achieves sharpness enhancement, thereby achieving detail preservation and detail enhancement of the filtering result.

[0086] In one possible implementation, such as Figure 5 As shown, taking the target number of iterations as the loop termination condition as an example, the image processing method proposed in this application includes the following steps:

[0087] Step 501: Obtain the original image p.

[0088] Step 502: Perform mean filtering on the original image to obtain the first image mean_p.

[0089] Among them, the original image can be mean filtered using a mean filter with a mean filtering radius r, mean_p = boxfilter(p).

[0090] Step 503, initialize the guide graph i, i = p.

[0091] Step 504: Initialize the mean filtering result of the guide graph i, mean_i = mean_p.

[0092] In other words, during the initialization of the guide map, the original image p is assigned to the guide map i, so that the initial guide map i is the original image p itself. Since the guide map i is the original image p itself, the mean filtering result of the guide map i is also the mean filtering result of the original image p.

[0093] Step 505: Determine whether the number of guided filtering operations has reached the target number.

[0094] If yes, proceed to step 506; otherwise, proceed to step 5051.

[0095] Step 505.1, calculate corr_i = boxfilter(i.*i).

[0096] Step 505.2, calculate corr_ip = boxfilter(i.*i).

[0097] Step 505.3, calculate var_i = corr_i - mean_i.*mean_1.

[0098] It should be noted that steps 5051 and 5053 are for calculating the mean and variance var_i of the guide graph i in the integral graph.

[0099] Step 505.4, calculate cov_ip = corr_ip - mean_i * mean_p.

[0100] Further, obtain the covariance map cov_ip between the guide image i and the original image p.

[0101] Step 505.5, calculate a = cov_ip / (var_i + eps).

[0102] Among them, the first correlation factor 'a' is used to determine the texture intensity of the pixel, and eps is the spatial filtering intensity, which is used to adjust the intensity of information enhancement. The setting of the spatial filtering intensity eps can be set according to the graphic features of the original image. The larger the value of the spatial filtering intensity eps, the greater the degree of image blurring, and the smaller the value of the spatial filtering intensity eps, the less the degree of image blurring.

[0103] Step 505.6, calculate b = mean_p - a.*mean_i.

[0104] Step 505.7, calculate mean_a = boxfilter(a).

[0105] Step 505.8, calculate mean_b = boxfilter(b).

[0106] Step 505.9: Calculate the result of the guided filter q = mean_a.*i + mean_b.

[0107] Where q is the first denoised image obtained in the current loop.

[0108] Step 505.10: Calculate the low-frequency information lp = boxfilter(q) of the guided filter result.

[0109] Step 505.11: Calculate the high-frequency information hp = q - lp of the guided filter result.

[0110] Step 505.12: Calculate the second denoised image s = q + alpha * hp after enhancing the high-frequency information.

[0111] Step 505.13: Use the second denoised image as the new guide map i = s, and return to execution 505.

[0112] Step 506: Loop ends, output target image.

[0113] In one possible implementation, such as Figure 6 As shown, taking the sharpness of the first denoised image as the loop termination condition as an example, the image processing method proposed in this application includes the following steps:

[0114] Step 601: Obtain the original image p.

[0115] Step 602: Perform mean filtering on the original image to obtain the first image mean_p.

[0116] Among them, the original image can be mean filtered using a mean filter with a mean filtering radius r, mean_p = boxfilter(p).

[0117] Step 603, initialize the guide graph i, i = p.

[0118] Step 604: Initialize the mean filtering result of the guide graph i, mean_i = mean_p.

[0119] In other words, during the initialization of the guide map, the original image p is assigned to the guide map i, so that the initial guide map i is the original image p itself. Since the guide map i is the original image p itself, the mean filtering result of the guide map i is also the mean filtering result of the original image p.

[0120] Step 605: Calculate corr_i = boxfilter(i.*i).

[0121] Step 606: Calculate corr_ip = boxfilter(i.*i).

[0122] Step 607: Calculate var_i = corr_i - mean_i.*mean_1.

[0123] It should be noted that steps 6051 and 6053 are for calculating the mean and variance var_i of the guide graph i in the integral graph.

[0124] Step 608: Calculate cov_ip = corr_ip - mean_i * mean_p.

[0125] Further, obtain the covariance map cov_ip between the guide image i and the original image p.

[0126] Step 609, calculate a = cov_ip / (var_i + eps).

[0127] Among them, the first correlation factor 'a' is used to determine the texture intensity of the pixel, and eps is the spatial filtering intensity, which is used to adjust the intensity of information enhancement. The setting of the spatial filtering intensity eps can be set according to the graphic features of the original image. The larger the value of the spatial filtering intensity eps, the greater the degree of image blurring, and the smaller the value of the spatial filtering intensity eps, the less the degree of image blurring.

[0128] Step 610, calculate b = mean_p - a.*mean_i.

[0129] Step 611, calculate mean_a = boxfilter(a).

[0130] Step 612, calculate mean_b = boxfilter(b).

[0131] Step 613: Calculate the result of the guided filtering: q = mean_a.*i + mean_b.

[0132] Where q is the first denoised image obtained in the current loop.

[0133] Step 614: Determine whether the sharpness of the guided filter result q reaches the target threshold.

[0134] If yes, proceed to step 619; otherwise, proceed to step 615.

[0135] Step 615: Calculate the low-frequency information lp = boxfilter(q) of the guided filter result.

[0136] Step 616: Calculate the high-frequency information hp = q - lp of the guided filtering result.

[0137] Step 617: Calculate the second denoised image s = q + alpha * hp after enhancing the high-frequency information.

[0138] Step 618: Use the second denoised image as the new guide map i = s, and return to execution 605.

[0139] Step 619: Output the first denoised image as the target image.

[0140] It should be noted that although the operation of the method of the present invention is described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed in order to achieve the desired result.

[0141] The image processing method of this application determines an enhanced image for texture information enhancement based on the guided filtering result of the original image itself. Then, a second denoised image for further guided filtering is obtained based on the enhanced image. The noise in the second denoised image is effectively eliminated by recursion, so that the second denoised image, which is the updated guided map, has stronger high-frequency information and effectively retains richer edge information. As a result, the first denoised image obtained based on the updated guided map can have higher sharpness and retain more edge information.

[0142] When obtaining the final filtering result of the original image, the system determines whether to output the current first denoised image as the final filtering result of the original image based on the sharpness of the first denoised image obtained by guided filtering based on the guided map. If the sharpness of the first denoised image does not reach a preset threshold, the system continues to construct a guided map based on the high-frequency information of the current first denoised image. That is, the high-frequency information of the guided map is gradually enhanced through iterative operations, so that the guided filtering result retains sufficient sharpness as the high-frequency information of the guided map is enhanced, or even achieves sharpness enhancement, thereby achieving detail preservation and detail enhancement of the filtering result.

[0143] Figure 7 A block diagram of an image processing apparatus according to an embodiment of this application is shown.

[0144] like Figure 7 As shown, the image processing apparatus 10 proposed in this application embodiment includes:

[0145] The filtering module 11 is used to use the original image to be processed as a guide image to perform guided filtering on the original image to obtain a first denoised image;

[0146] The acquisition module 12 is used to acquire an enhanced image based on the high-frequency information in the first denoised image, wherein the enhanced image contains texture information of the first denoised image after enhancement;

[0147] Enhancement module 13 is used to update the first denoised image using the enhanced image to obtain a second denoised image;

[0148] The loop module 14 is used to take the second denoised image as the updated guide map and return to perform guide filtering on the original image until the loop ends and the target image is obtained.

[0149] In some embodiments, the acquisition module 12 is specifically used for:

[0150] The first denoised image is subjected to mean filtering to obtain the low-frequency information of the first denoised image;

[0151] The high-frequency information of the first denoised image is obtained by performing a difference operation on the low-frequency information of the first denoised image and the first denoised image.

[0152] The intensity of the high-frequency information is corrected using intensity control parameters to obtain the enhanced image. The intensity control parameters are used to characterize the multiple by which the high-frequency information needs to be enhanced.

[0153] In some embodiments, the intensity control parameter is positively correlated with the preset threshold, wherein the intensity control parameter is a value greater than 1.

[0154] In some embodiments, the enhancement module 13 is specifically used for:

[0155] The enhanced image is fused with the first denoised image to obtain the second denoised image.

[0156] In some embodiments, the condition for reaching the end of the loop includes: the number of times the guided filtering of the original image is returned reaches a target number.

[0157] In some embodiments, the condition for reaching the end of the loop includes: the sharpness of the first denoised image reaches a preset threshold.

[0158] In some embodiments, the loop module 14 is specifically used for:

[0159] Obtain the energy distribution statistics corresponding to the first denoised image;

[0160] If the energy distribution statistics are greater than or equal to the target threshold, it is determined that the sharpness of the first denoised image has reached the target threshold;

[0161] If the energy distribution statistics are less than the target threshold, it is determined that the sharpness of the first denoised image has not reached the target threshold.

[0162] In some embodiments, the loop module 14 is specifically used for:

[0163] Target pixels are determined based on the original image, and the target pixels are those used for sharpness assessment.

[0164] For each target pixel, a target threshold corresponding to the target pixel is set;

[0165] Calculate the sharpness value corresponding to each target pixel;

[0166] If the sharpness value corresponding to each target pixel is greater than or equal to the target threshold corresponding to the target pixel, it is determined that the sharpness of the first denoised image has reached the target threshold.

[0167] If the sharpness value corresponding to any of the target pixels is less than the target threshold corresponding to the target pixel, it is determined that the sharpness of the first denoised image has not reached the target threshold.

[0168] In some embodiments, the filtering module 11 is specifically used for:

[0169] According to the preset spatial filtering radius, the original image is subjected to mean filtering to obtain the first image after mean filtering.

[0170] Determine the second image corresponding to the guide map based on the first image;

[0171] Calculate the variance plot corresponding to the guide map, and calculate the covariance plot between the first image and the guide map based on the first image and the second image;

[0172] The first denoised map is obtained based on the variance map and the covariance map.

[0173] In some embodiments, the filtering module 11 is further configured to:

[0174] Based on the variance plot, the covariance plot, and the spatial filtering intensity, the first correlation factor is determined;

[0175] Based on the first image and the first correlation factor, a second correlation factor is determined;

[0176] The first denoised image is obtained based on the first correlation factor and the second correlation factor.

[0177] In some embodiments, the filtering module 11 is further configured to:

[0178] Based on the spatial filtering radius, the mean values ​​of the first correlation factor and the second correlation factor are calculated respectively to obtain the first filtered image and the second filtered image.

[0179] The first denoised image is obtained based on the first filtered image and the second filtered image.

[0180] It should be understood that the units or modules described in the image processing apparatus 10 are the same as those in the reference. Figure 2The steps in the described method correspond to each other. Therefore, the operations and features described above for the method also apply to the image processing apparatus 10 and the units contained therein, and will not be repeated here. The image processing apparatus 10 can be pre-implemented in the browser or other security applications of an electronic device, or it can be loaded into the browser or its security applications of an electronic device by means of downloading, etc. The corresponding units in the image processing apparatus 10 can cooperate with the units in the electronic device to implement the solutions of the embodiments of this application.

[0181] The division of modules or units mentioned in the detailed description above is not mandatory. In fact, according to the embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0182] The image processing apparatus of this application determines an enhanced image for texture information enhancement based on the guided filtering result of the original image itself. Then, a second denoised image for further guided filtering is obtained based on the enhanced image. The noise in the second denoised image is effectively eliminated in a recursive manner, so that the second denoised image, which is the updated guided map, has stronger high-frequency information and effectively retains richer edge information. As a result, the first denoised image obtained based on the updated guided map can have higher sharpness and retain more edge information.

[0183] When obtaining the final filtering result of the original image, the system determines whether to output the current first denoised image as the final filtering result of the original image based on the sharpness of the first denoised image obtained by guided filtering based on the guided map. If the sharpness of the first denoised image does not reach a preset threshold, the system continues to construct a guided map based on the high-frequency information of the current first denoised image. That is, the high-frequency information of the guided map is gradually enhanced through iterative operations, so that the guided filtering result retains sufficient sharpness as the high-frequency information of the guided map is enhanced, or even achieves sharpness enhancement, thereby achieving detail preservation and detail enhancement of the filtering result.

[0184] Specifically, according to embodiments of this application, the flowchart above refers to... Figure 2The described process can be implemented as a computer software program. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program contains program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 109, and / or installed from removable medium 111. When the computer program is executed by central processing unit (CPU) 101, it performs the functions defined in the system of this application.

[0185] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0186] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operational instructions of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two connected blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified functions or operational instructions, or using a combination of dedicated hardware and computer instructions.

[0187] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be housed in a processor; for example, a processor can be described as including a filtering module, an acquisition module, an enhancement module, and a loop module. The names of these units or modules do not necessarily limit the specific unit or module itself. For example, a filtering module can also be described as "using the original image to be processed as a guide image, performing guided filtering on the original image to obtain a first denoised image."

[0188] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments, or may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium stores one or more programs that, when used by one or more processors, execute the image processing method described in this application.

[0189] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An image processing method, characterized in that, include: The original image to be processed is used as a guide image, and guide filtering is performed on the original image to obtain the first denoised image; Based on the high-frequency information in the first denoised image, an enhanced image is obtained, wherein the enhanced image contains texture information of the first denoised image after enhancement. The enhanced image is used to update the first denoised image to obtain the second denoised image; The second denoised image is used as the updated guide image, and the guide filtering of the original image is performed again until the condition for the end of the loop is met, and the target image is obtained. The conditions for reaching the end of the loop include: the sharpness of the first denoised image reaches the target threshold. Wherein, the sharpness of the first denoised image reaches the target threshold, including: Target pixels are determined based on the original image, and the target pixels are those used for sharpness assessment. For each target pixel, a target threshold corresponding to the target pixel is set; Calculate the sharpness value corresponding to each target pixel; If the sharpness value corresponding to each target pixel is greater than or equal to the target threshold corresponding to the target pixel, it is determined that the sharpness of the first denoised image has reached the target threshold.

2. The method according to claim 1, characterized in that, The step of obtaining an enhanced image based on the first denoised image includes: The first denoised image is subjected to mean filtering to obtain the low-frequency information of the first denoised image; The high-frequency information of the first denoised image is obtained by performing a difference operation on the low-frequency information of the first denoised image and the first denoised image. The intensity of the high-frequency information is corrected using intensity control parameters to obtain the enhanced image. The intensity control parameters are used to characterize the multiple by which the high-frequency information needs to be enhanced.

3. The method according to claim 1, characterized in that, The condition for reaching the end of the loop also includes: the number of times the guided filtering of the original image is returned reaches the target number.

4. The method according to claim 1, characterized in that, The sharpness of the first denoised image reaches the target threshold, including: Obtain the energy distribution statistics corresponding to the first denoised image; If the energy distribution statistics are greater than or equal to the target threshold, it is determined that the sharpness of the first denoised image has reached the target threshold.

5. The method according to claim 1, characterized in that, The step of using the original image to be processed as a guide image and performing guide filtering on the original image to obtain a first denoised image includes: According to the preset spatial filtering radius, the original image is subjected to mean filtering to obtain the first image after mean filtering. Determine the second image corresponding to the guide map based on the first image; Calculate the variance plot corresponding to the guide map, and calculate the covariance plot between the first image and the guide map based on the first image and the second image; The first denoised map is obtained based on the variance map and the covariance map.

6. The method according to claim 5, characterized in that, The process of obtaining the first denoised image based on the variance map and the covariance map includes: Based on the variance plot, the covariance plot, and the spatial filtering intensity, the first correlation factor is determined; Based on the first image and the first correlation factor, a second correlation factor is determined; The first denoised image is obtained based on the first correlation factor and the second correlation factor.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image processing method as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the image processing method as described in any one of claims 1-6.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the image processing method according to any one of claims 1-6.

Images