Image optimization method

Abstract

The present disclosure relates to an image optimization method. The image optimization method comprises: obtaining an original image and determining a size of the original image; down-sampling the original image to obtain a down-sampled image; performing noise reduction processing on the down-sampled image to obtain a noise-reduced image; performing sharpening processing based on the noise-reduced image to obtain a sharpened image; fusing the down-sampled image and the sharpened image to obtain a fused image; up-sampling the fused image so that the size of the up-sampled fused image is the same as that of the original image; and fusing the original image and the up-sampled fused image to obtain a target image. The present disclosure reduces the loss of image details of the target image relative to the original image, and reduces the distortion of the image after sharpening processing, thereby improving the image quality of the target image.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to an image optimization method. Background Technology

[0002] In today's digital age, image applications are widely used in numerous fields, such as semiconductor manufacturing. The ever-increasing demands for image quality across these fields are driving the continuous development of image optimization technologies.

[0003] In the semiconductor manufacturing industry, as chip manufacturing processes continue to shrink, the precision requirements for wafer surface defect detection are becoming increasingly stringent. Traditional image optimization methods typically include noise reduction or sharpening. After acquiring wafer images using a scanning electron microscope, traditional image optimization methods employ noise reduction and sharpening algorithms sequentially to optimize the image. However, it is difficult to retain sufficient image details, such as minute defect features, while simultaneously reducing noise. For example, conventional noise reduction algorithms, when used to reduce image noise to obtain a denoised image, can easily lead to the loss of image details, such as smoothing out the edges of minute defects. If the denoised image is used for defect detection, it may result in missed defects. Conversely, sharpening algorithms, when used to enhance image details to obtain a sharpened image, may amplify noise and increase image distortion. If the sharpened image is used for defect detection, it may increase the false alarm rate.

[0004] Therefore, the image optimization methods involved in related technologies result in images with a high degree of loss of image details and high degree of image distortion. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides an image optimization method.

[0006] This disclosure provides an image optimization method, the method comprising: acquiring an original image and determining the size of the original image; downsampling the original image to obtain a downsampled image; performing noise reduction processing on the downsampled image to obtain a denoised image; performing sharpening processing on the denoised image to obtain a sharpened image; fusing the downsampled image and the sharpened image to obtain a fused image; upsampling the fused image such that the size of the upsampled fused image is the same as that of the original image; and fusing the original image and the upsampled fused image to obtain a target image.

[0007] In some embodiments, before sharpening the denoised image to obtain a sharpened image, the method further includes: obtaining a preset first weight map; determining a second weight map based on the first weight map, such that the sum of each second weight value in the second weight map and the first weight value at the same position in the first weight map is one; obtaining a first image obtained by multiplying the first weight map and the values ​​in the downsampled image at the same position; obtaining a second image obtained by multiplying the second weight map and the values ​​in the denoised image at the same position; and summing the values ​​at the same position in the first image and the second image to obtain the processed denoised image; wherein, sharpening the denoised image to obtain a sharpened image includes: sharpening the processed denoised image to obtain a sharpened image.

[0008] In some embodiments, before fusing the downsampled image and the sharpened image to obtain a fused image, the method further includes: obtaining a third image obtained by multiplying the values ​​in the second weight image and the processed denoised image at the same positions; obtaining a fourth image obtained by multiplying the values ​​in the first weight image and the sharpened image at the same positions; and summing the values ​​in the third image and the fourth image at the same positions to obtain a processed sharpened image; wherein, fusing the downsampled image and the sharpened image to obtain a fused image includes: fusing the downsampled image and the processed sharpened image to obtain a fused image.

[0009] In some embodiments, obtaining a preset first weight map includes: dividing the downsampled image into multiple downsampled sub-images, and determining the variance of pixel values ​​corresponding to pixels in each downsampled sub-image; normalizing the variance of each downsampled sub-image to obtain a normalized variance; using the normalized variance of the downsampled sub-image as the first weight value corresponding to each pixel in the downsampled sub-image, and obtaining the first weight map based on the first weight value corresponding to each pixel in the downsampled image.

[0010] In some embodiments, dividing the downsampled image into multiple downsampled sub-images includes: dividing the downsampled image horizontally, and / or dividing the downsampled image vertically to obtain multiple rectangular downsampled sub-images.

[0011] In some embodiments, obtaining a preset first weight map includes: performing edge detection on the downsampled image to obtain an edge image; and performing normalization processing on the edge image to obtain the first weight map.

[0012] In some embodiments, before fusing the downsampled image and the sharpened image to obtain a fused image, the method further includes: obtaining a preset third weight map; determining a fourth weight map based on the third weight map, such that the sum of each fourth weight value in the fourth weight map and the third weight value at the same position in the third weight map is one; obtaining a fifth image obtained by multiplying the values ​​in the third weight map and the sharpened image at the same position; obtaining a sixth image obtained by multiplying the values ​​in the fourth weight map and the processed denoised image at the same position; and summing the values ​​at the same position in the fifth image and the sixth image to obtain a processed sharpened image; wherein, fusing the downsampled image and the sharpened image to obtain a fused image includes: fusing the downsampled image and the processed sharpened image to obtain a fused image.

[0013] In some embodiments, obtaining the preset third weight map includes: dividing the processed denoised image into multiple sub-images and determining the variance of the pixel values ​​corresponding to pixels in each sub-image; normalizing the variances corresponding to each sub-image to obtain normalized variances; using the normalized variances corresponding to the sub-images as the third weight values ​​corresponding to each pixel in the sub-images, and obtaining the third weight map based on the third weight values ​​corresponding to each pixel in the processed denoised image.

[0014] In some embodiments, dividing the processed denoised image into multiple sub-images includes: dividing the processed denoised image horizontally, and / or dividing the processed denoised image vertically to obtain multiple rectangular sub-images.

[0015] In some embodiments, obtaining a preset third weight map includes: performing edge detection on the processed denoised image to obtain an edge image; and performing normalization processing on the edge image to obtain the third weight map.

[0016] This disclosure provides an image optimization method that downsamples the original image. Compared to directly processing the original image, using the downsampled image reduces computational complexity in subsequent processing and typically filters out high-frequency noise from the original image. Fusing the sharpened image (obtained after noise reduction and sharpening) with the downsampled image yields a fused image, which reduces the loss of image details and image distortion, thus improving the image quality of the target image. Fusing the upsampled fused image with the original image yields the target image, which reduces the loss of image details in the target image relative to the original image and also reduces the distortion after sharpening, thereby improving the image quality of the target image.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0019] Figure 1 This is a flowchart illustrating an image optimization method according to an exemplary embodiment.

[0020] Figure 2 This is a flowchart illustrating an exemplary embodiment for obtaining a preset first weight map.

[0021] Figure 3 This is a flowchart illustrating another method for obtaining a preset first weight map, according to an exemplary embodiment.

[0022] Figure 4 This is a flowchart illustrating a processed, denoised image according to an exemplary embodiment.

[0023] Figures 5A-5C These are exemplary diagrams illustrating the use of various noise reduction algorithms to denoise downsampled images.

[0024] Figure 6 This is a flowchart illustrating an example of acquiring a processed sharpened image.

[0025] Figure 7 This is a flowchart illustrating another method for acquiring a processed sharpened image, according to an exemplary embodiment.

[0026] Figure 8 This is a flowchart illustrating an exemplary embodiment for obtaining a preset third weight map.

[0027] Figure 9 This is a flowchart illustrating another method for obtaining a preset third weight map, according to an exemplary embodiment.

[0028] Figures 10A-10C These are exemplary schematic diagrams illustrating the sharpening process of denoised images using various sharpening algorithms.

[0029] Figure 11 This is a schematic diagram illustrating an acquisition of a target image according to an exemplary embodiment.

[0030] Figure 12 This is a schematic diagram illustrating another method for acquiring a target image according to an exemplary embodiment.

[0031] Figures 13A-13C These are schematic diagrams of the original image, the downsampled image, and the first weight map shown in an exemplary embodiment. Detailed Implementation

[0032] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0033] The image optimization method provided in this disclosure is applicable to image processing scenarios, such as optimizing images of samples to be tested in the semiconductor field.

[0034] In related technologies, when optimizing images, noise reduction algorithms are used to denoise the original image to obtain a denoised image, and then the denoised image is sharpened to obtain an optimized image, also known as the target image. However, noise reduction algorithms often lead to the loss of image details, and sharpening algorithms often lead to image distortion, resulting in the target image having problems such as a high degree of loss of image details and a high degree of image distortion.

[0035] In view of this, this disclosure provides an image optimization method, which is based on fusing an image that has undergone noise reduction and sharpening (i.e., a sharpened image) with the original image to obtain an optimized image, also known as a target image. This method reduces the degree of image detail loss in the target image relative to the original image while simultaneously reducing the distortion of the target image, thereby improving image quality.

[0036] Figure 1 This is a flowchart illustrating an image optimization method according to an exemplary embodiment. Figure 1 As shown, it includes steps S11-S17.

[0037] Step S11: Obtain the original image and determine the size of the original image.

[0038] In this embodiment of the disclosure, the original image to be optimized is obtained, and the image size of the original image is determined.

[0039] In this embodiment of the disclosure, the type of the original image is not limited. For example, it can be an image acquired using a scanning electron microscope, an image acquired using an optical microscope, or an image acquired using a camera. The image can be an image of a sample to be inspected in the semiconductor field, such as an image of a wafer. The shape of the original image can be rectangular.

[0040] Step S12: Downsample the original image to obtain a downsampled image.

[0041] In this embodiment of the disclosure, a downsampling algorithm can be used to reduce the size of the original image to obtain a downsampled image. The image size of the downsampled image can be determined as needed. For example, the downsampling algorithm can be used to reduce the image size of the original image to half of the original image size to obtain a downsampled image.

[0042] Step S13: Perform noise reduction processing on the downsampled image to obtain a noise-reduced image.

[0043] In this embodiment of the disclosure, one or more denoising algorithms can be used to denoise the downsampled image to obtain a denoised image. For example, the denoising algorithm can be a Gaussian blur algorithm, a nonlocal mean denoising algorithm, etc. This disclosure does not limit the denoising algorithm.

[0044] Step S14: Sharpen the image based on the denoised image to obtain a sharpened image.

[0045] In this embodiment of the disclosure, one or more sharpening algorithms can be used to sharpen the denoised image to obtain a sharpened image. For example, unsharpening masking algorithms, frequency domain filtering algorithms, etc., can be used.

[0046] If the sharpened image is used directly as the target image, the image quality of the target image will be low because noise reduction processing can easily lead to the loss of image details and sharpening processing can easily lead to image distortion. Therefore, it is necessary to perform image fusion on the sharpened image to improve the image quality of the target image.

[0047] Step S15: The downsampled image and the sharpened image are fused to obtain a fused image.

[0048] In this embodiment of the disclosure, although the downsampled image loses some image details compared to the original image after downsampling, the downsampled image still retains some image details of the original image, and the image distortion of the downsampled image is usually small. Therefore, after obtaining the fused image in step S15, it is beneficial to retain the image details from the original image and reduce the image distortion. Subsequently, the fused image is upsampled and fused with the original image to obtain the target image, which can reduce the degree of image detail loss and thus improve the image quality of the target image.

[0049] In this embodiment of the disclosure, an image fusion algorithm can be used to fuse the downsampled image and the sharpened image to obtain a fused image.

[0050] The image fusion algorithm can be a weighted summation algorithm, a wavelet transform algorithm, a Laplacian pyramid algorithm, etc. This disclosure does not limit the image fusion algorithm.

[0051] Step S16: Upsample the fused image so that the size of the upsampled fused image is the same as that of the original image.

[0052] In this embodiment of the disclosure, any upsampling algorithm can be used to ensure that the size of the upsampled fused image is the same as that of the original image. For example, the upsampling algorithm can be nearest neighbor interpolation, bicubic interpolation, bilinear interpolation, etc. This disclosure does not limit the upsampling algorithm.

[0053] Step S17: Fuse the original image and the upsampled fused image to obtain the target image.

[0054] In this embodiment of the disclosure, when further fusing the original image and the upsampled fused image, the image fusion algorithm used may be the same as or different from the image fusion algorithm involved in step S15, and this disclosure does not limit it.

[0055] According to exemplary embodiments of this disclosure, by downsampling the original image, the computational complexity can be reduced in subsequent processing compared to directly processing the original image. It can also typically filter out high-frequency noise in the original image initially. Fusing the sharpened image (obtained after noise reduction and sharpening) with the downsampled image to obtain a fused image reduces the loss of image details and image distortion, thus improving the image quality of the target image. Fusing the upsampled fused image with the original image to obtain the target image reduces the loss of image details in the target image relative to the original image, while also reducing the distortion of the sharpened image, thereby improving the image quality of the target image.

[0056] In this embodiment of the disclosure, the following method will be used to target Figure 1 The steps involved are further explained below.

[0057] In this embodiment of the disclosure, after obtaining the downsampled image and before obtaining the sharpened image, a first weight map and a denoised image can be obtained based on the downsampled image, and the downsampled image and the denoised image can be processed based on the first weight map to obtain the processed denoised image.

[0058] In this embodiment of the disclosure, the first weight map can be understood as a map used to characterize the richness of image details in the downsampled image, and each first weight value is used to characterize the richness of details in the surrounding area of ​​the pixel corresponding to the first weight value in the downsampled image. This can be achieved using methods such as... Figure 2 or Figure 3 The first weight map is obtained using any of the following methods, utilizing the downsampled image.

[0059] Figure 2This is a flowchart illustrating an exemplary embodiment for obtaining a preset first weight map. For example... Figure 2 As shown, it includes steps S21-S23.

[0060] Step S21: Divide the downsampled image into multiple downsampled sub-images, and determine the variance of the pixel values ​​corresponding to the pixels in each downsampled sub-image.

[0061] Step S22: Normalize the variance of each downsampled sub-image to obtain the normalized variance.

[0062] Step S23: The normalized variance corresponding to the downsampled sub-image is used as the first weight value corresponding to each pixel in the downsampled sub-image, and the first weight map is obtained based on the first weight value corresponding to each pixel in the downsampled image.

[0063] In one example, the downsampled image is segmented into multiple downsampled sub-images of the same size (where a downsampled sub-image can also be understood as a local region of the downsampled image), but this is not limited to this. For each downsampled sub-image, the variance corresponding to each downsampled sub-image is calculated, and all variances are normalized to obtain the normalized variance corresponding to each variance. The normalization method is based on existing techniques; for example, dividing all variances by the largest variance among all variances yields the normalized variance corresponding to each variance. The normalized variance corresponding to each pixel in the downsampled sub-image is used as a weight value, with the weight value ranging from 0 to 1. Each pixel in the downsampled image has a corresponding weight value, so a first weight map is obtained based on these weight values. Figure 13C Taking the first weighted graph as an example, the weight value of the pure black position in the first weighted graph is 0, the weight value of the pure white position is 1, and the weight value of the other positions is greater than 0 and less than 1.

[0064] According to an exemplary embodiment of this disclosure, based on the downsampled image, the variance of each downsampled sub-image is normalized by calculating the variance of the pixel value corresponding to each pixel in each downsampled sub-image, so as to facilitate the determination of the weight value corresponding to each position in the first weight image. The variance of the pixel reflects the richness of image details in the surrounding area (which is the downsampled sub-image at this time) where the pixel is located.

[0065] In this embodiment of the disclosure, the downsampled image is divided into multiple downsampled sub-images, including: dividing the downsampled image horizontally, and / or dividing the downsampled image vertically, to obtain multiple rectangular downsampled sub-images.

[0066] In one example, the original image can be a rectangular image. Downsampling the original rectangular image yields a downsampled rectangular image. When dividing the downsampled image, it can be divided horizontally, vertically, or a combination of horizontal and vertical divisions to obtain downsampled sub-images. In this embodiment, the horizontal axis is parallel to one rectangular side of the downsampled image, and the vertical axis is parallel to another rectangular side; the horizontal and vertical axes are perpendicular to each other.

[0067] Figure 3 This is a flowchart illustrating another method for obtaining a preset first weight map, according to an exemplary embodiment. For example... Figure 3 As shown, it includes steps S31-S32.

[0068] Step S31: Perform edge detection on the downsampled image to obtain an edge image.

[0069] In this embodiment of the disclosure, an edge detection algorithm is used to perform edge detection on the downsampled image.

[0070] Step S32: Normalize the edge image to obtain the first weight map.

[0071] In one exemplary embodiment, an edge detection algorithm can be used to extract the edges of a downsampled image to obtain an edge image, and the edge image can be normalized to obtain a first weight map. The edge detection algorithm can be the Sobel algorithm, the Canny algorithm, or the Prewitt algorithm, etc. This disclosure does not limit the edge detection algorithm used for edge detection.

[0072] In this regard, the existing technology for normalization is to divide all pixel values ​​in the edge image by the maximum pixel value among all pixel values ​​to obtain the normalized pixel value corresponding to each pixel value.

[0073] According to an exemplary embodiment of this disclosure, an edge image is obtained by performing edge detection on the downsampled image, which highlights the edge portion in the downsampled image. The pixel values ​​in the edge image reflect the richness of detail in the surrounding area of ​​the pixel and can be used as a first weight map.

[0074] In this embodiment of the disclosure, the following methods can be used: Figure 4 As shown, the first weight map obtained above is used to process the denoised image to obtain the processed denoised image.

[0075] Figure 4 This is a flowchart illustrating a processed, denoised image according to an exemplary embodiment. For example... Figure 4As shown, before sharpening the image based on the denoised image to obtain the sharpened image, the image optimization method also includes steps S41-S43.

[0076] Step S41: Obtain the preset first weight map and determine the second weight map based on the first weight map.

[0077] In this embodiment of the disclosure, the method for obtaining the first weight map has been described above and will not be repeated here; using the weight values ​​in the first weight map, the second weight value at the same position in the second weight map as in the first weight map is determined.

[0078] In this context, the sum of each second weight value in the second weight graph and the first weight value in the first weight graph at the same position is one. For example, at the same position (x, y) in both the first and second weight graphs, with w... x,y Given the first weight value at position (x, y) in the first weight graph, the second weight value at position (x, y) in the second weight graph is: 1 - w x,y .

[0079] Step S42: Multiply the values ​​in the first weight map and the downsampled image at the same positions to obtain the first image; multiply the values ​​in the second weight map and the denoised image at the same positions to obtain the second image.

[0080] In this embodiment, a first weight value in a first weight map at the same location is multiplied by a pixel value in a downsampled image to obtain a first image. Then, a second weight value in a second weight map at the same location is multiplied by a pixel value in a denoised image to obtain a second image.

[0081] In one example, for instance, with The pixel value at position (x, y) in the downsampled image I1 is represented by... The pixel value at position (x, y) in the denoised image I2 is represented by the first weight map and the second weight map obtained in step S41, as well as the content of step S42.

[0082] The pixel value at position (x, y) in the first image is:

[0083] The pixel value at position (x, y) in the second image is:

[0084] Step S43: Summing the values ​​at the same positions in the first image and the second image to obtain the processed denoised image.

[0085] In this embodiment of the disclosure, the values ​​at the same positions of the first image and the second image are summed to obtain the processed denoised image I2′.

[0086] In one example, for instance, with Let I2' represent the pixel value at position (x, y) in the processed denoised image. Based on the pixel values ​​at position (x, y) in the first and second images obtained in step S42, we can conclude that:

[0087]

[0088] According to an exemplary embodiment of this disclosure, a processed denoised image I2 is obtained based on a first image and a second image. For regions with rich details in the downsampled image, the first weight value w corresponding to each pixel is... x,y The value is relatively large. According to the formula above, when the first weight value w... x,y When the first weight value w is large, the processed denoised image I2′ retains more pixel values ​​from the downsampled image I1, while the downsampled image I1 retains some image details of the original image. Therefore, the processed denoised image I2′ has relatively rich image details. x,y When the pixel size is small, the processed denoised image I2′ retains more pixel values ​​from the denoised image I2. Therefore, the processed denoised image I2′ can reduce more noise. It can be seen that the pixel values ​​in the processed denoised image I2′ are obtained by weighted summation of the downsampled image I1 and the denoised image I2, which balances the degree of noise reduction and the degree of image detail preservation. Compared with the denoised image, the processed denoised image improves the image quality.

[0089] When image optimization involves using multiple denoising algorithms to denoise downsampled images, for example, algorithms such as... Figures 5A-5C The layout shown illustrates the arrangement of various noise reduction algorithms.

[0090] Figures 5A-5C These are exemplary diagrams illustrating the use of various noise reduction algorithms to denoise downsampled images.

[0091] In one example, multiple denoising algorithms can be executed in parallel as shown in 5A to denoise the downsampled image, resulting in a denoised image. For instance, the downsampled image can be simultaneously input into different denoising algorithms, and the downsampled image can be denoised using different algorithms at the same time. The average of all images processed by different denoising algorithms can then be calculated to obtain the denoised image.

[0092] In another example, it can also be used as follows Figure 5BThe method shown serially executes multiple denoising algorithms to denoise the downsampled image, resulting in a denoised image. For example, the downsampled image is input into any one denoising algorithm for denoising, and then the denoised image is input into other denoising algorithms for further denoising. By sequentially performing denoising processing in different denoising algorithms, a denoised image is obtained.

[0093] In another example, the following can also be used: Figure 5C The method shown involves executing a first number (e.g., three) of denoising algorithms in parallel on the downsampled image. The images processed by a second number (e.g., two) of denoising algorithms are then averaged to obtain the second number of denoised images. Here, the second number is a value less than the first number. The second number of denoised images is then averaged again with all images processed by the other numbers of denoising algorithms in the first number (excluding the second number) to obtain the first number of denoised images. The images processed by the first number of denoising algorithms are then sequentially input into a third number (e.g., two) of denoising algorithms, where the first number of denoised images are sequentially denoised to obtain the denoised image.

[0094] In this embodiment of the disclosure, for the denoised image obtained by using the preset first weight map, the sharpened image can also be processed by the second weight map to obtain the processed sharpened image.

[0095] The second weighted graph can be adopted. Figure 4 The content retrieval shown is not described in detail here.

[0096] For example, it can be adopted Figure 6 or Figure 7 The image can be processed using any of the methods described above to obtain the processed sharpened image.

[0097] Figure 6 This is a flowchart illustrating an exemplary embodiment for acquiring a processed, sharpened image. For example... Figure 6 As shown, before fusing the downsampled image and the sharpened image to obtain the fused image, the image optimization method also includes steps S51-S53.

[0098] Step S51: Obtain the third image obtained by multiplying the values ​​in the second weight map and the processed denoised image at the same positions.

[0099] In this embodiment of the disclosure, the second weight value in the second weight map at the same position is multiplied with the pixel value in the processed denoised image to obtain the third image.

[0100] The second weight value of the second weighted graph at position (x, y) is: 1 - w x,y Furthermore, the pixel value at position (x,y) in the processed denoised image I2′ is... It can be seen that the pixel value at position (x, y) in the third image is:

[0101]

[0102] Step S52: Obtain the fourth image obtained by multiplying the values ​​in the first weighted image and the sharpened image at the same positions.

[0103] In this embodiment, the method for obtaining the first weight map is the same as described above, and will not be repeated here.

[0104] For example, with The pixel value at position (x, y) in the sharpened image I3 is represented by the first weight value of the first weight map at position (x, y), which is w. x,y We can see that the pixel value at position (x, y) in the fourth image is:

[0105]

[0106] Step S53: Summate the values ​​at the same positions in the third and fourth images to obtain the processed sharpened image.

[0107] In this embodiment of the disclosure, the values ​​of the third image and the fourth image at the same positions are summed to obtain the processed sharpened image.

[0108] In one example, for instance, with Let represent the pixel value at position (x, y) in the sharpened image I3′. Then, the pixel value at position (x, y) in the sharpened image I3′ can be expressed as:

[0109]

[0110] According to an exemplary embodiment of this disclosure, for regions with rich details in a downsampled image, the first weight value w corresponding to each pixel is... x,y The value is relatively large. According to the formula above, when the first weight value w... x,y When the value is large, the processed sharpened image I3′ retains more pixel values ​​from the sharpened image I3, and the sharpened image I3 enhances the image details of the processed denoised image I2′, thus making the processed sharpened image I3′ richer in image details. When the first weight value w x,yWhen the image size is small, the sharpened image I3′ retains more pixel values ​​from the denoised image I2′. Therefore, the sharpened image I3′ reduces the enhancement of noise in the sharpened image I3. Thus, the pixel values ​​in the sharpened image I3′ are obtained by weighted summation of the sharpened image I3 and the denoised image I2′. This enhances more details in the denoised image I2′ while reducing the enhancement of noise in the denoised image I2′, balancing the enhancement of details and the amplification of noise. Compared with the sharpened image I3, the sharpened image I3′ improves image quality.

[0111] In this embodiment of the present disclosure, before obtaining the fused image based on the sharpened image and the downsampled image, in addition to obtaining a preset first weight map using the downsampled image and processing the sharpened image to obtain the processed sharpened image as described above, a preset third weight map can also be obtained based on the denoised image, the sharpened image can be processed using the third weight map to obtain the processed sharpened image, and the fused image can be obtained using the processed sharpened image and the downsampled image.

[0112] Figure 7 This is a flowchart illustrating another method for acquiring a processed, sharpened image, according to an exemplary embodiment. For example... Figure 7 As shown, the image optimization method includes steps S61-S63.

[0113] Step S61: Obtain the preset third weight map, and determine the fourth weight map based on the third weight map.

[0114] In this embodiment of the disclosure, the weight values ​​in the third weight graph are used to determine the fourth weight value at the same position in the fourth weight graph as in the third weight graph.

[0115] In particular, the sum of each fourth weight value in the fourth weight graph and the third weight value in the third weight graph at the same position is one.

[0116] For example, at the same position (x) in the third weight map and the fourth weight map ′ ,y ′ ) at, with w ′ x′,y′ Represents the position (x) in the third weighted graph ′ ,y ′ The third weight value at position (x) in the fourth weight graph is then used to determine the position (x) in the fourth weight graph. ′ ,y ′ The fourth weight at position ) is: 1-w ′ x′,y′ .

[0117] Step S62: Multiply the values ​​in the third weighted image and the sharpened image in the same positions to obtain the fifth image; multiply the values ​​in the fourth weighted image and the processed denoised image in the same positions to obtain the sixth image.

[0118] In this embodiment, the third weight value corresponding to the third weight map at the same location is multiplied by the pixel value in the sharpened image to obtain the fifth image. Then, the fourth weight value corresponding to the fourth weight map at the same location is multiplied by the pixel value in the processed denoised image to obtain the sixth image.

[0119] In one example, for instance, with Characterizing the position (x) of pixels in the sharpened image I3 ′ ,y ′ The pixel value at position ) is In the denoised image I2′ after characterization, the pixel at position (x ′ ,y ′ The pixel value at position () can be determined from the third and fourth weight maps obtained in step S61, and the content of step S62:

[0120] In the fifth image, the pixel is at position (x) ′ ,y ′ The pixel value at position ) is:

[0121] In the sixth image, the pixel is at position (x ′ ,y ′ The pixel value at position ) is:

[0122] Step S63: Summate the values ​​at the same positions in the fifth and sixth images to obtain the processed sharpened image.

[0123] In this embodiment of the disclosure, the values ​​of the fifth and sixth images at the same positions are summed to obtain the processed sharpened image I3. ′ .

[0124] In one example, for instance, with This indicates the position (x) of a pixel in the sharpened image after processing. ′ ,y ′ The pixel value at position (x) in the sharpened image is then the pixel value at position (x) in the processed image. ′ ,y ′ The pixel value at position ) can be represented as:

[0125]

[0126] According to an exemplary embodiment of this disclosure, a processed sharpened image is obtained based on the fifth and sixth images. For regions with rich details in the denoised image, the third weight value w corresponding to each pixel is...′ x′,y′ It is relatively large. According to the above formula, when the third weight value w... ′ x′,y′ When the value is large, the processed sharpened image I3′ retains more pixel values ​​from the sharpened image I3, and the sharpened image I3 enhances the image details of the processed denoised image I2′, thus making the processed sharpened image I3′ richer in image details. When the third weight value w ′ x′,y′ When the image size is small, the sharpened image I3′ retains more pixel values ​​from the denoised image I2′. Therefore, the sharpened image I3′ can reduce the enhancement of noise in the sharpened image I3. Thus, the pixel values ​​in the sharpened image I3′ are obtained by weighted summation of the denoised image I2′ and the sharpened image I3′. This not only enhances more details in the denoised image I2′, but also reduces the enhancement of noise in the denoised image I2′, balancing the enhancement of details and the amplification of noise. Compared with the sharpened image I3, the sharpened image I3′ improves the image quality.

[0127] Specifically, for the pre-defined third weighted graph, the following can be used: Figure 8 or Figure 9 The image is obtained by processing it using any of the methods shown.

[0128] Figure 8 This is a flowchart illustrating an exemplary embodiment for obtaining a preset third weight map. It includes steps S71-S73.

[0129] Step S71: Divide the processed denoised image into multiple sub-images and determine the variance of the pixel values ​​corresponding to the pixels in each sub-image.

[0130] Step S72: Normalize the variance of each sub-image to obtain the normalized variance.

[0131] Step S73: The normalized variance corresponding to the sub-image is used as the third weight value corresponding to each pixel in the sub-image, and the third weight map is obtained based on the third weight value corresponding to each pixel in the processed denoised image.

[0132] In one example, the processed denoised image is segmented into multiple sub-images of the same size (where a sub-image can also be understood as a local region of the processed denoised image), but this is not limited to this. For each sub-image, the variance corresponding to each sub-image is calculated, and all variances are normalized to obtain the normalized variance corresponding to each variance. The normalization method has been explained above and will not be repeated here. The normalized variance corresponding to each pixel in the sub-image is used as the weight value, and the weight value is between 0 and 1. Each pixel in the processed denoised image has a corresponding weight value, so a third weight map is obtained based on these weight values.

[0133] In this embodiment of the disclosure, the processed denoised image is divided into multiple sub-images, including: dividing the processed denoised image horizontally, and / or dividing the processed denoised image vertically to obtain multiple rectangular sub-images.

[0134] In one example, the processed denoised image can be a rectangular image. This rectangular denoised image can be divided horizontally, vertically, or a combination of both. Regardless of whether the division is horizontal or vertical, the direction of the division is parallel to the rectangular edges of the processed denoised image, and the horizontal and vertical directions are perpendicular to each other.

[0135] Figure 9 This is a flowchart illustrating another method for obtaining a preset third weight map, according to an exemplary embodiment. For example... Figure 9 As shown, it includes the following steps S81-S82.

[0136] Step S81: Perform edge detection on the processed denoised image to obtain an edge image.

[0137] In this embodiment of the disclosure, an edge detection algorithm is used to perform edge detection on the processed denoised image.

[0138] Step S82: Normalize the edge image to obtain the third weight map.

[0139] In one exemplary embodiment, an edge detection algorithm can be used to extract the edges of the processed denoised image, obtaining the corresponding edge image of the processed denoised image. The edge image of the processed denoised image is then normalized to obtain a third weight map. The edge detection process and normalization method can be found in [reference needed]. Figure 3 This will not be elaborated upon here.

[0140] In this embodiment of the disclosure, regarding the image optimization method described above, when the image optimization process involves using multiple sharpening algorithms for sharpening processing, for example, a method such as... Figures 10A-10C The layout shown illustrates how various sharpening algorithms are applied.

[0141] Figures 10A-10C These are exemplary schematic diagrams illustrating the sharpening process of denoised images using various sharpening algorithms.

[0142] In one example, the following can be used: Figure 10A The method shown executes multiple sharpening algorithms in parallel to sharpen the denoised image or the processed denoised image, resulting in a sharpened image. For example, the denoised image or the processed denoised image is simultaneously input into different sharpening algorithms, and the different sharpening algorithms are used to sharpen the denoised image or the processed denoised image at the same time. The average of all images processed by different sharpening algorithms is then calculated to obtain the sharpened image.

[0143] In another example, it can also be used as follows Figure 10B The method shown serially executes multiple sharpening algorithms to sharpen a denoised image or a processed denoised image, resulting in a sharpened image. For example, the denoised image or the processed denoised image is input into any one sharpening algorithm for sharpening, and then the sharpened image is input again into other sharpening algorithms for further sharpening. By sequentially performing sharpening processes in different sharpening algorithms, a sharpened image is obtained.

[0144] In another example, the following can also be used: Figure 10C The method shown involves executing a first number (e.g., three) of sharpening algorithms in parallel on the denoised image. The images processed by a second number (e.g., two) of sharpening algorithms are then averaged to obtain the second number of sharpened images. Here, the second number is a value less than the first number. The image processed by the second number of sharpening algorithms is then averaged again with all images processed by the other numbers of sharpening algorithms in the first number (excluding the second number) to obtain the second number of sharpened images. The images processed by the first number of sharpening algorithms are then sequentially input into a third number (e.g., two) of sharpening algorithms, where the first number of sharpened images are processed sequentially to obtain the sharpened image.

[0145] In this embodiment of the disclosure, the downsampled image and the processed sharpened image are fused to obtain a fused image.

[0146] In this embodiment of the disclosure, the above-described implementation method can be adopted, according to... Figure 11 The diagram shown illustrates the acquisition of the target image. Figure 11 This is a schematic diagram illustrating an embodiment of acquiring a target image. Figure 11The original image I is downsampled to obtain a downsampled image I1. Noise reduction is then performed on the downsampled image I1 to obtain a denoised image I2. The denoised image I2 is further sharpened to obtain a sharpened image I3. The downsampled image I1 and the sharpened image I3 are then fused to obtain a fused image I4. The fused image is then upsampled to obtain an upsampled fused image I4′ with the same size as the original image. Finally, the upsampled fused image I4′ is further fused with the original image to obtain the target image I5.

[0147] In this embodiment of the disclosure, the above-described implementation method can be adopted, according to... Figure 12 The diagram shown illustrates the acquisition of the target image. Figure 12 This is a schematic diagram illustrating another method for acquiring a target image according to an exemplary embodiment.

[0148] exist Figure 12 A downsampled image I1 corresponding to the original image I is obtained, and a weight map (including a first weight map and a second weight map) is obtained using the downsampled image I1. A denoising algorithm is applied to the downsampled image I1 to perform denoising processing, resulting in a denoised image. Based on the downsampled image I1, the weight map, and the denoised image I2, the above-described... Figure 4 The denoised image I2′ is obtained by processing the image as shown. A sharpening algorithm is then applied to the denoised image I2′ to obtain a sharpened image I3. Based on the denoised image I2′, the weight map, and the sharpened image I3, the above-described method of this disclosure is used to... Figure 6 The sharpened image I3′ is obtained by processing the image as shown. Based on the downsampled image I1 and the sharpened image I3′, a fused image I4 is obtained. The fused image I4 is upsampled to obtain an upsampled fused image I4′ with the same size as the original image I. The upsampled fused image I4′ is further fused with the original image to obtain the target image I5.

[0149] in, Figure 12 The weight map mentioned in the section about "obtaining the processed sharpened image I3′" can also be a weight map obtained from the processed denoised image. Specifically, the weight map obtained from the processed denoised image includes a third weight map and a fourth weight map.

[0150] In this embodiment of the disclosure, the process of obtaining the weight map from the original image can also be illustrated by the diagram. Figures 13A-13C To further explain the above, Figures 13A-13C These are schematic diagrams of the original image, downsampled image, and first weight map shown in an exemplary embodiment. The second weight map is not shown in this embodiment. Figure 13A For the raw image obtained directly, targeting Figure 13AThe original image in the image is downsampled according to actual usage requirements to obtain a result compared to... Figure 13A The downsampled image after downsampling the original image, such as Figure 13B .against Figure 13B Determine the corresponding Figure 13B The first weighted graph is obtained Figure 13C For example, in Figure 13C In the weight map, if a region appears pure black, it means that the information in that region in the processed denoised image depends entirely on the information in the corresponding region of the denoised image. Conversely, if a region appears pure white, it means that the information in that region depends entirely on the information in the corresponding region of the downsampled image.

[0151] In this disclosure, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0152] The methods and apparatus disclosed herein can be implemented using standard programming techniques, utilizing rule-based logic or other logic to implement various method steps. It should also be noted that the terms "apparatus" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0153] Any step, operation, or procedure described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product comprising a computer-readable medium containing computer program code, which is executable by a computer processor to perform any or all of the described steps, operations, or procedures.

[0154] The foregoing description of embodiments of this disclosure has been provided for purposes of illustration and description. The foregoing description is not exhaustive and is not intended to limit this disclosure to the exact form disclosed; various modifications and variations may be made in accordance with the foregoing teachings, or may be derived from practice of this disclosure. These embodiments were chosen and described to illustrate the principles of this disclosure and its practical application, enabling those skilled in the art to utilize this disclosure in various implementations and modifications suitable for the particular purpose conceived.

[0155] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0156] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0157] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0158] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0159] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0160] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0161] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image optimization method, characterized in that, The method includes: Acquire the original image and determine the size of the original image; The original image is downsampled to obtain a downsampled image; The downsampled image is then subjected to noise reduction processing to obtain a noise-reduced image; The denoised image is then sharpened to obtain a sharpened image. The downsampled image and the sharpened image are fused together to obtain a fused image; The fused image is upsampled so that the upsampled fused image and the original image have the same size; The target image is obtained by fusing the original image and the upsampled fused image.

2. The method according to claim 1, characterized in that, Before performing sharpening processing on the denoised image to obtain the sharpened image, the method further includes: Obtain a preset first weight map, and determine a second weight map based on the first weight map, such that the sum of each second weight value in the second weight map and the first weight value at the same position in the first weight map is one; Obtain a first image by multiplying the values ​​in the first weighted image and the downsampled image at the same positions; obtain a second image by multiplying the values ​​in the second weighted image and the denoised image at the same positions; and sum the values ​​in the first image and the second image at the same positions to obtain the processed denoised image. The step of sharpening the denoised image to obtain a sharpened image includes: The denoised image is then sharpened to obtain a sharpened image.

3. The method according to claim 2, characterized in that, Before fusing the downsampled image and the sharpened image to obtain the fused image, the method further includes: Obtain a third image by multiplying the values ​​in the second weight image and the processed denoised image at the same positions; obtain a fourth image by multiplying the values ​​in the first weight image and the sharpened image at the same positions; and sum the values ​​in the third image and the fourth image at the same positions to obtain the processed sharpened image. The step of fusing the downsampled image and the sharpened image to obtain the fused image includes: The downsampled image and the processed sharpened image are fused together to obtain a fused image.

4. The method according to claim 2 or 3, characterized in that, The process of obtaining the preset first weighted graph includes: The downsampled image is divided into multiple downsampled sub-images, and the variance of the pixel values ​​corresponding to the pixels in each downsampled sub-image is determined. The variances corresponding to each downsampled sub-image are normalized to obtain the normalized variances. The first weight map is obtained based on the normalized variance corresponding to the downsampled sub-image as the first weight value corresponding to each pixel in the downsampled sub-image.

5. The method according to claim 4, characterized in that, The step of dividing the downsampled image into multiple downsampled sub-images includes: The downsampled image is divided horizontally, and / or vertically, to obtain multiple rectangular downsampled sub-images.

6. The method according to claim 2 or 3, characterized in that, The process of obtaining the preset first weighted graph includes: Edge detection is performed on the downsampled image to obtain an edge image; The edge image is normalized to obtain the first weight map.

7. The method according to claim 2, characterized in that, Before fusing the downsampled image and the sharpened image to obtain the fused image, the method further includes: Obtain a preset third weight map, and determine a fourth weight map based on the third weight map, such that the sum of each fourth weight value in the fourth weight map and the third weight value at the same position in the third weight map is one; Obtain the fifth image obtained by multiplying the values ​​in the third weight image and the sharpened image at the same positions; obtain the sixth image obtained by multiplying the values ​​in the fourth weight image and the processed denoised image at the same positions; and sum the values ​​in the fifth and sixth images at the same positions to obtain the processed sharpened image. The step of fusing the downsampled image and the sharpened image to obtain the fused image includes: The downsampled image and the processed sharpened image are fused together to obtain a fused image.

8. The method according to claim 7, characterized in that, The process of obtaining the preset third weight map includes: The processed denoised image is divided into multiple sub-images, and the variance of the pixel values ​​corresponding to the pixels in each sub-image is determined. The variances corresponding to each of the sub-images are normalized to obtain the normalized variances. The normalized variance corresponding to the sub-image is used as the third weight value corresponding to each pixel in the sub-image, and the third weight map is obtained based on the third weight value corresponding to each pixel in the processed denoised image.

9. The method according to claim 8, characterized in that, The step of dividing the processed, denoised image into multiple sub-images includes: The processed denoised image is divided horizontally, and / or vertically, to obtain multiple rectangular sub-images.

10. The method according to claim 7, characterized in that, The process of obtaining the preset third weight map includes: Edge detection is performed on the processed denoised image to obtain an edge image; The edge image is normalized to obtain the third weight map.

Images