Image brightness adjusting method and related product

Abstract

The invention provides an image brightness adjusting method and a related product. The method comprises the following steps: determining a gradual change area from an image to be processed; calculating a derivative of a pixel point in the gradual change area, and recording the derivative as a pixel point derivative; determining a brightness adjustment area from the gradual change area according to the pixel point derivative; and carrying out gray value stretching on the pixel points in the brightness adjustment area. According to the method, a high-brightness area and a low-brightness image area are identified, and further gray value stretching processing is performed, so that a gradient image of the high-brightness area and a gradient image of the low-brightness area are realized, and a convex area and a concave area in a tested sample are better identified; noisy point information appearing in the image can be further filtered out, and the signal-to-noise ratio of the image is improved. Therefore, the grayscale image realized by the method can meet the identification of more detail information of the image to a greater extent.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor image processing technology, and in particular to an image brightness adjustment method and related products. Background Art

[0002] In the current semiconductor imaging field, in the analysis and research of microscopic morphology, there are often recessed or raised areas in the inspected samples. With the development of high-tech technologies such as semiconductor devices, it is often required to have a deeper observation and understanding of the surface structures. Therefore, obtaining a clear, high-resolution photo requires not only a high-performance, high-resolution scanning electron microscope, but also a more complete image processing algorithm.

[0003] In a scanning electron microscope, the uneven structure of the sample surface is presented as a difference in light and dark contrast in the electronic image on the display screen. The sunken parts of the sample being tested are reflected in the image as a high-grayscale image, while the protruding parts of the image are low-grayscale images. Moreover, the light and dark contrast also comes from the difference in the strength of the secondary electron signal on the surface of the sample. Due to the limitations of the scanning electron microscope in terms of process and technology, the image quality on the display screen is reduced, which puts higher requirements on the image processing algorithm. The details of the sample being tested are sufficient and fine, and the contrast in the key areas is high to better observe the structural problems on the surface of the sample. In order to observe more image details, it is necessary to appropriately adjust the local image brightness of the sunken or protruding parts of the sample.

[0004] In current scanning electron microscopes, the method for adjusting brightness and contrast is to adjust the level of the input signal of the preamplifier and the high voltage value of the photomultiplier tube. On this basis, the methods for adaptively adjusting image brightness are:

[0005] 1. Histogram equalization processing uses the cumulative function to adjust the grayscale value so that a relatively concentrated grayscale interval is evenly distributed in the entire grayscale space, realizing nonlinear stretching of the image. However, this method reduces the grayscale level of the pixel value of the redistributed image, resulting in a reduction in some details.

[0006] 2. Based on the nonlinear processing of gamma transform, the product operation is performed on each pixel of the original image, and then the image is corrected by enhancing the details of low grayscale or high grayscale. However, this method is suitable for the situation where the image contrast is low and the overall brightness value is high, and it is not suitable for the dark brightness value of the scanning electron microscope image. Summary of the invention

[0007] In view of the above problems, the present invention is proposed to provide an image brightness adjustment method that overcomes the above problems or at least partially solves the above problems.

[0008] One purpose of the present invention is to achieve more reasonable brightness adjustment for the gradient area of ​​an image, so as to identify more detailed information in the image.

[0009] A further object of the present invention is to be applicable to brightness adjustment of a scanned image whose overall brightness is dim but has local high brightness areas.

[0010] Another further object of the present invention is to filter the noise information that appears during image recognition to further improve the image signal-to-noise ratio.

[0011] In particular, the present invention provides an image brightness adjustment method, which comprises:

[0012] Determining a gradient region from the image to be processed;

[0013] Calculate the derivative of the pixel point in the gradient area, recorded as the pixel point derivative;

[0014] Determine a brightness adjustment area from the gradient area according to the pixel point derivative;

[0015] Grayscale values ​​of pixels within the brightness adjustment area are stretched.

[0016] Optionally, the pixel point derivative includes the derivative of the pixel point in the X coordinate direction and the Y coordinate direction respectively;

[0017] The step of determining the brightness adjustment area from the gradient area according to the pixel point derivative comprises:

[0018] Selecting an adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction, and selecting an adjustment interval of the gradient region in the Y-coordinate direction according to the derivative in the Y-coordinate direction;

[0019] The brightness adjustment area is determined according to the adjustment interval of the gradient area in the X-coordinate direction and the adjustment interval in the Y-coordinate direction.

[0020] Optionally, the image brightness adjustment method further includes:

[0021] When executing the step of selecting the adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction, selecting the noise point interval of the gradient region in the X-coordinate direction;

[0022] When executing the step of selecting the adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction, selecting the noise point interval of the gradient region in the Y coordinate direction;

[0023] Determining an image noise area from the gradient area according to the noise area in the X-coordinate direction and the noise area in the Y-coordinate direction of the gradient area;

[0024] Perform filtering processing on the image noise area.

[0025] Optionally, the step of selecting an adjustment interval of the gradual change area in the X-coordinate direction according to the derivative in the X-coordinate direction includes:

[0026] According to the X-coordinate direction, selecting a candidate interval in the X-coordinate direction from the gradient area according to the derivative in the X-coordinate direction;

[0027] Determine whether the candidate interval in the X-coordinate direction is the adjustment interval in the X-coordinate direction according to the derivative in the X-coordinate direction within the candidate interval in the X-coordinate direction.

[0028] Optionally, according to the X-coordinate direction, the two ends of each candidate interval in the X-coordinate direction are a first pixel point and a second pixel point respectively; the derivative in the X-coordinate direction of each pixel point between the first pixel point and the second pixel point is a non-zero value; the derivative in the X-coordinate direction of the first pixel point is a non-zero value; the derivative in the X-coordinate direction of the second pixel point is a zero value, or the derivative in the X-coordinate direction of the second pixel point is a non-zero value and the second pixel point is at the edge of the gradient area.

[0029] Optionally, the step of selecting a candidate interval in the X-coordinate direction from the gradient area according to the X-coordinate direction and according to a derivative in the X-coordinate direction includes:

[0030] According to the X-coordinate direction, determine whether the first non-zero derivative in the X-coordinate direction appears;

[0031] If yes, the pixel point corresponding to the derivative in the X-coordinate direction is taken as the first pixel point; and whether the first derivative in the X-coordinate direction with a value of 0 appears is further determined according to the X-coordinate direction;

[0032] When the first derivative in the X-coordinate direction with a value of 0 is determined, the pixel point corresponding to the derivative in the X-coordinate direction is used as the second pixel point, and the first derivative in the X-coordinate direction with a value other than 0 is determined according to the X-coordinate direction until the last pixel point is determined;

[0033] When the derivative in the X-coordinate direction in which the first 0 value does not appear continues to be determined according to the X-coordinate direction, the last pixel point is used as the second pixel point to obtain at least one candidate interval.

[0034] Optionally, the step of determining whether the candidate interval of the X-coordinate direction is the adjustment interval of the X-coordinate direction according to the derivative of the X-coordinate direction within the candidate interval of the X-coordinate direction comprises:

[0035] Obtaining the number of non-zero derivatives in the X-coordinate direction within the candidate interval in the X-coordinate direction, including the derivative in the X-coordinate direction of the first pixel point and the derivative in the X-coordinate direction of the second pixel point;

[0036] When the number is greater than or equal to the number threshold, the candidate interval in the X-coordinate direction is determined as the adjustment interval in the X-coordinate direction.

[0037] Optionally, the image brightness adjustment method further includes:

[0038] When the number is less than the number threshold, the candidate interval in the X-coordinate direction is determined to be the noise point interval in the X-coordinate direction, so that when the step of selecting the adjustment interval of the gradient area in the X-coordinate direction according to the derivative in the X-coordinate direction is performed, the noise point interval of the gradient area in the X-coordinate direction is selected.

[0039] Optionally, the step of selecting an adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction includes:

[0040] According to the Y coordinate direction, selecting a candidate interval in the Y coordinate direction from the gradient area according to the derivative in the Y coordinate direction;

[0041] Determine whether the candidate interval in the Y-coordinate direction is the adjustment interval in the Y-coordinate direction according to the derivative in the Y-coordinate direction within the candidate interval in the Y-coordinate direction.

[0042] Optionally, according to the Y-coordinate direction, the two ends of each alternative interval in the Y-coordinate direction are the third pixel point and the fourth pixel point respectively; the derivative of the Y-coordinate direction of each pixel point between the third pixel point and the fourth pixel point is a non-zero value; the derivative of the Y-coordinate direction of the third pixel point is a non-zero value; the derivative of the Y-coordinate direction of the fourth pixel point is 0, or the derivative of the Y-coordinate direction of the fourth pixel point is a non-zero value and the fourth pixel point is at the edge of the gradient area.

[0043] Optionally, the step of selecting a candidate interval in the Y coordinate direction from the gradient area according to the Y coordinate direction and a derivative in the Y coordinate direction includes:

[0044] According to the Y coordinate direction, determine whether the first non-zero derivative in the Y coordinate direction appears;

[0045] If yes, the pixel point corresponding to the derivative in the Y coordinate direction is used as the third pixel point; and whether the first derivative in the Y coordinate direction with a value of 0 appears is further determined according to the Y coordinate direction;

[0046] When the first derivative in the Y coordinate direction of 0 is determined, the pixel point corresponding to the derivative in the Y coordinate direction is taken as the fourth pixel point, and the first derivative in the Y coordinate direction of a non-zero value is determined according to the Y coordinate direction until the last pixel point is determined;

[0047] When the derivative in the Y coordinate direction without the first 0 value continuing to be determined in the Y coordinate direction, the last pixel point is taken as the fourth pixel point to obtain at least one candidate interval.

[0048] Optionally, the step of determining whether the candidate interval in the Y-coordinate direction is the adjustment interval in the Y-coordinate direction according to the derivative in the Y-coordinate direction within the candidate interval in the Y-coordinate direction includes:

[0049] Obtaining the number of non-zero derivatives in the Y-coordinate direction within the candidate interval in the Y-coordinate direction, including the derivative in the Y-coordinate direction of the third pixel point and the derivative in the Y-coordinate direction of the fourth pixel point;

[0050] When the number is greater than or equal to the number threshold, the candidate interval in the Y coordinate direction is determined as the adjustment interval in the Y coordinate direction.

[0051] Optionally, the image brightness adjustment method further includes:

[0052] When the number is less than the number threshold, the alternative interval in the Y-coordinate direction is determined to be the noise point interval in the Y-coordinate direction, so that when executing the step of selecting the adjustment interval of the gradient area in the Y-coordinate direction according to the derivative in the Y-coordinate direction, the noise point interval of the gradient area in the Y-coordinate direction is selected.

[0053] Optionally, determining the gradient region from the image to be processed specifically comprises: performing edge detection on the image using an edge detection algorithm to obtain the gradient region;

[0054] The graphics to be processed are grayscale images; the step of determining the gradient area from the image to be processed comprises:

[0055] Calculate the gradient of each pixel in the image;

[0056] Comparing each of the gradient magnitudes with a preset dual threshold;

[0057] The gradient region is determined according to the comparison result.

[0058] Optionally, the grayscale values ​​of the pixels in the brightness adjustment area are stretched by nonlinear transformation.

[0059] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for adjusting image brightness are implemented.

[0060] According to yet another aspect of the present invention, a computer program product is provided, which includes a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned methods for adjusting image brightness are implemented.

[0061] According to another aspect of the present invention, a computer device is provided, which includes a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of any of the above-mentioned image brightness adjustment methods.

[0062] In the image brightness adjustment method of the present invention, since it is necessary to determine the gradient area from the image and determine the brightness adjustment area from the gradient area, a more reasonable brightness adjustment can be achieved for the gradient area of ​​the image, so as to identify more detail information in the image, prevent the grayscale level of the pixel value of the redistributed image from being reduced, and prevent the reduction of certain detail parts.

[0063] Furthermore, the present invention does not need to perform a product operation on each pixel of the original image, and then corrects the image by enhancing the low grayscale or high grayscale details. The present invention only adjusts the brightness of the brightness adjustment area in the gradient area, which is not only suitable for images with low image contrast and high overall brightness values, but also particularly suitable for images with dark brightness values ​​of scanning electron microscopes.

[0064] Furthermore, in the image brightness adjustment method of the present invention, the brightness adjustment area in the image can be well selected through the pixel point derivative, especially the first-order derivative of the pixel point. Moreover, since the gradient area is pre-selected, it is not necessary to perform pixel point derivative on the entire image, which is more targeted, and the amount of calculation when adjusting the image brightness is reduced, and the efficiency is higher.

[0065] Furthermore, in the image brightness adjustment method of the present invention, there may be image noise areas in the gradient area, that is, these areas may have gradients, but cannot meet the gradient requirements, so they can be called noise points. The image noise areas are filtered and the noise information that appears is filtered to further improve the image signal-to-noise ratio.

[0066] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0068] Figure 1 is a flow chart of an image brightness adjustment method according to an embodiment of the present invention;

[0069] Figure 2 is a partial flow chart of an image brightness adjustment method according to an embodiment of the present invention;

[0070] Figure 3 is a partial flow chart of an image brightness adjustment method according to an embodiment of the present invention;

[0071] Figure 4 is a partial flow chart of an image brightness adjustment method according to an embodiment of the present invention;

[0072] Figure 5 is a partial flow chart of an image brightness adjustment method according to an embodiment of the present invention;

[0073] Figure 6 is a partial flow chart of an image brightness adjustment method according to an embodiment of the present invention;

[0074] Figure 7 1 is a schematic diagram of pixel values ​​in the X-coordinate direction in a method for adjusting image brightness according to an embodiment of the present invention, wherein there are an adjustment interval and a noise interval;

[0075] Figure 8 It is a schematic diagram of the relationship between the pixel value and the initial pixel value after the grayscale value of the pixel points in the brightness adjustment area is stretched by nonlinear transformation in an image brightness adjustment method according to an embodiment of the present invention;

[0076] Fig. 9 is a schematic diagram of a computer program product according to an embodiment of the present invention;

[0077] Fig.10 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention; and

[0078] Fig.11 is a schematic diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0079] This embodiment provides a solution for a method for adjusting image brightness. Figure 1FIG. 1 is a flow chart of a method for adjusting image brightness according to an embodiment of the present invention. The method may generally include:

[0080] Step S100, determine the gradient area from the image to be processed. The gradient area is the key area of ​​the image, usually a local image of a concave part or a convex part of the sample. By adjusting part or all of the brightness in the key area, more image details can be observed, so that the details of the sample to be tested are sufficient and fine. Therefore, it is necessary to determine the gradient area.

[0081] Step S200, calculate the derivative of the pixel points in the gradient area, which is recorded as the pixel point derivative. The pixel point derivative is an important concept in image processing. Its first-order derivative represents the rate of change at a certain pixel point. Since the pixel points are discrete, the first-order derivative can be calculated by the difference between adjacent pixel values. The pixel point derivative can reflect whether the image has a gradient.

[0082] Step S300, determine the brightness adjustment area from the gradient area according to the pixel derivative. For a pixel, although it is in the gradient area, the values ​​of the pixels around it are consistent with the value of the pixel, which means that the brightness around the pixel is consistent, and there is no need to adjust the image brightness. Therefore, it is necessary to determine the brightness adjustment area from the gradient area, not to adjust the brightness of all the pixels in the gradient area.

[0083] Step S400: grayscale value stretching is performed on the pixels within the brightness adjustment area.

[0084] In the embodiment of the present invention, since it is necessary to determine the gradient area from the image and determine the brightness adjustment area from the gradient area, it is possible to achieve more reasonable brightness adjustment for the gradient area of ​​the image, thereby identifying more detail information in the image, preventing the grayscale level of the pixel value of the redistributed image from being reduced, and preventing certain detail parts from being reduced. Moreover, the embodiment of the present invention does not need to perform a product operation on each pixel point of the original image, and then corrects the image by enhancing the details of low grayscale or high grayscale.

[0085] The embodiment of the present invention only performs brightness adjustment on the brightness adjustment area in the gradient area, which is not only applicable to images with low image contrast and high overall brightness values, but also particularly applicable to images with darker brightness values ​​of scanning electron microscopes, that is, particularly applicable to images with darker overall brightness but with local high brightness areas. In the embodiment of the present invention, the brightness adjustment area in the image can be well selected through pixel derivatives, especially first-order derivatives of pixels. Moreover, since the gradient area is pre-selected, it is not necessary to perform pixel derivatives on the entire image, which is more targeted, and the amount of calculations during image brightness adjustment is reduced, which is more efficient.

[0086] In some embodiments of the present invention, the pixel point derivative includes the derivative of the pixel point in the X coordinate direction and the Y coordinate direction. Figure 2 As shown, the above step S300, the step of determining the brightness adjustment area from the gradient area according to the pixel point derivative, includes:

[0087] Step S310 , selecting an adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction.

[0088] Step S320, selecting an adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction.

[0089] Step S330: determining a brightness adjustment region according to the adjustment range of the gradient region in the X-coordinate direction and the adjustment range in the Y-coordinate direction.

[0090] The pixels in the image are usually arranged in a matrix along the X-coordinate direction and the Y-coordinate direction. Therefore, the adjustment interval in the X-coordinate direction and the adjustment interval in the Y-coordinate direction can be determined. The area formed based on these two intervals is the brightness adjustment area. This also takes into account the gradient in both directions, making the brightness adjustment area obtained more accurate.

[0091] In some embodiments of the present invention, Figures 3 to 5 As shown, the image brightness adjustment method also includes:

[0092] When executing the step of selecting the adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction, the noise point interval of the gradient region in the X-coordinate direction is selected. That is, when selecting the adjustment interval in the X-coordinate direction, the noise point interval in the X-coordinate direction is also selected.

[0093] When executing the step of selecting the adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction, the noise point interval of the gradient region in the Y coordinate direction is selected. That is, when selecting the adjustment interval in the Y coordinate direction, the noise point interval in the Y coordinate direction is also selected.

[0094] Step S500: determining an image noise region from the gradient region according to the noise region in the X-coordinate direction and the noise region in the Y-coordinate direction.

[0095] Step S600: filtering the image noise area.

[0096] In an embodiment of the present invention, there may be image noise areas in the gradient area, that is, these areas may have gradients, but cannot meet the gradient requirements, so they can be called noise points. The image noise areas are filtered and the noise information is filtered to further improve the image signal-to-noise ratio.

[0097] In some embodiments of the present invention, the step of selecting the adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction includes: selecting a candidate interval in the X-coordinate direction from the gradient region according to the derivative in the X-coordinate direction according to the X-coordinate direction. Determining whether the candidate interval in the X-coordinate direction is the adjustment interval in the X-coordinate direction according to the derivative in the X-coordinate direction within the candidate interval in the X-coordinate direction.

[0098] In this embodiment, a candidate interval is first selected, and the candidate interval may be an adjustment interval or may not be an adjustment interval. Then, whether the candidate interval is an adjustment interval is determined according to the derivative situation in the candidate interval.

[0099] In the embodiment of the present invention, according to the X-coordinate direction, the two ends of each candidate interval in the X-coordinate direction are the first pixel and the second pixel, respectively. The derivative in the X-coordinate direction of each pixel between the first pixel and the second pixel is a non-zero value. The derivative in the X-coordinate direction of the first pixel is a non-zero value. The derivative in the X-coordinate direction of the second pixel is a zero value, or the derivative in the X-coordinate direction of the second pixel is a non-zero value and the second pixel is at the edge of the gradient region.

[0100] In some embodiments of the present invention, Figure 3 and Figure 7 As shown, the above step of selecting the candidate interval in the X-coordinate direction from the gradient area according to the derivative in the X-coordinate direction includes:

[0101] Step S311, judging whether the first derivative in the X-coordinate direction having a non-zero value appears according to the X-coordinate direction.

[0102] If yes, the process proceeds to step S312, where the pixel corresponding to the derivative in the X-coordinate direction is used as the first pixel. If no, the process continues to determine whether the first non-zero derivative in the X-coordinate direction appears.

[0103] Step S313, continue to determine whether the first derivative in the X-coordinate direction with a value of 0 appears according to the X-coordinate direction;

[0104] Step S314, when it is determined that the first derivative in the X-coordinate direction with a value of 0 appears, the pixel point corresponding to the derivative in the X-coordinate direction is used as the second pixel point.

[0105] Step S315, continue to determine whether the first non-zero derivative in the X-coordinate direction appears according to the X-coordinate direction, and then return to step S312 until the last pixel point is determined.

[0106] Step S316: when the derivative in the X-coordinate direction without the first zero value continues to be determined in the X-coordinate direction, the last pixel point is used as the second pixel point to obtain at least one candidate interval.

[0107] When the derivative in the X-coordinate direction is 0, it means that there is no gradient along the X-coordinate direction, so there is no need to adjust the brightness of these pixels. Therefore, the pixels corresponding to the derivative in the X-coordinate direction with a non-zero value are selected.

[0108] In some embodiments of the present invention, Figure 3 and Figure 7 As shown, the above step of determining whether the candidate interval in the X-coordinate direction is the adjustment interval in the X-coordinate direction according to the derivative in the X-coordinate direction within the candidate interval in the X-coordinate direction includes:

[0109] Step S317, obtaining the number of non-zero derivatives in the X-coordinate direction including the derivative in the X-coordinate direction of the first pixel and the derivative in the X-coordinate direction of the second pixel within the candidate interval in the X-coordinate direction.

[0110] Step S318: When the number is greater than or equal to the number threshold, determine the candidate interval in the X-coordinate direction as the adjustment interval in the X-coordinate direction.

[0111] In the embodiment of the present invention, if the number of derivatives in the X-coordinate direction with non-zero values ​​does not meet the number threshold, it means that the candidate interval may not be a gradient area, but some noise intervals. Therefore, only the candidate intervals when the number of derivatives in the X-coordinate direction with non-zero values ​​meets the number threshold are used as adjustment intervals. Moreover, in the embodiment of the present invention, it is necessary to first determine the gradient area from the image to be processed, the purpose is to reduce the number of candidate intervals that are not adjustment intervals, greatly improve the calculation efficiency, and thus improve the efficiency of image brightness adjustment.

[0112] In some embodiments of the present invention, Figure 3 and Figure 7 As shown, the image brightness adjustment method also includes:

[0113] Step S319, when the number is less than the number threshold, determine the candidate interval in the X-coordinate direction as the noise point interval in the X-coordinate direction, so that when executing the step of selecting the adjustment interval of the gradient area in the X-coordinate direction according to the derivative in the X-coordinate direction, select the noise point interval of the gradient area in the X-coordinate direction.

[0114] In some embodiments of the present invention, the adjustment interval and the noise interval in the Y coordinate direction may be acquired based on the step of acquiring the adjustment interval and the noise interval in the X coordinate direction.

[0115] Specifically, in some embodiments of the present invention, the step of selecting the adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction includes:

[0116] According to the Y coordinate direction, select the candidate interval in the Y coordinate direction from the gradient area according to the derivative in the Y coordinate direction;

[0117] Whether the candidate interval in the Y-coordinate direction is the adjustment interval in the Y-coordinate direction is determined according to the derivative in the Y-coordinate direction within the candidate interval in the Y-coordinate direction.

[0118] According to the Y-coordinate direction, the two ends of each alternative interval in the Y-coordinate direction are the third pixel point and the fourth pixel point respectively; the derivative in the Y-coordinate direction of each pixel point between the third pixel point and the fourth pixel point is a non-zero value; the derivative in the Y-coordinate direction of the third pixel point is a non-zero value; the derivative in the Y-coordinate direction of the fourth pixel point is 0, or the derivative in the Y-coordinate direction of the fourth pixel point is a non-zero value and the fourth pixel point is at the edge of the gradient area.

[0119] In some embodiments of the present invention, Figure 4 As shown, the above step of selecting the candidate interval in the Y coordinate direction from the gradient area according to the derivative in the Y coordinate direction includes:

[0120] Step S321, according to the Y coordinate direction, determine whether the first non-zero derivative in the Y coordinate direction appears.

[0121] If so, proceed to step S322, and the pixel point corresponding to the derivative in the Y coordinate direction is used as the third pixel point.

[0122] Step S323, continue to determine whether the first derivative in the Y coordinate direction of 0 appears according to the Y coordinate direction;

[0123] Step S324, when it is determined that the first derivative in the Y coordinate direction with a value of 0 appears, the pixel point corresponding to the derivative in the Y coordinate direction is used as the fourth pixel point.

[0124] Step S325, continue to determine whether the first non-zero derivative in the Y-coordinate direction appears according to the Y-coordinate direction, and then return to step S322 until the last pixel point is determined.

[0125] Step S326, when the derivative in the Y coordinate direction without the first zero value continues to be determined in the Y coordinate direction, the last pixel point is used as the fourth pixel point to obtain at least one candidate interval.

[0126] When the derivative in the Y coordinate direction is 0, it means that there is no gradient along the Y coordinate direction, so there is no need to adjust the brightness of these pixels. Therefore, the pixels corresponding to the derivative in the Y coordinate direction with a non-zero value are selected.

[0127] In some embodiments of the present invention, Figure 4 As shown, the above step of determining whether the candidate interval in the Y coordinate direction is the adjustment interval in the Y coordinate direction according to the derivative in the Y coordinate direction within the candidate interval in the Y coordinate direction includes:

[0128] Step S327, obtaining the number of non-zero derivatives in the Y-coordinate direction within the candidate interval in the Y-coordinate direction, including the derivative in the Y-coordinate direction of the third pixel point and the derivative in the Y-coordinate direction of the fourth pixel point.

[0129] Step S328: When the number is greater than or equal to the number threshold, determine the candidate interval in the Y coordinate direction as the adjustment interval in the Y coordinate direction.

[0130] In the embodiment of the present invention, if the number of derivatives in the Y coordinate direction of non-zero values ​​does not meet the number threshold, it means that the candidate interval may not be a gradient area, but some noise intervals. Therefore, only the candidate intervals when the number of derivatives in the Y coordinate direction of non-zero values ​​meets the number threshold are used as adjustment intervals. Moreover, in the embodiment of the present invention, it is necessary to first determine the gradient area from the image to be processed, the purpose is to reduce the number of candidate intervals that are not adjustment intervals, which greatly improves the calculation efficiency, and thus improves the efficiency of image brightness adjustment.

[0131] In some embodiments of the present invention, Figure 4 As shown, the image brightness adjustment method also includes:

[0132] Step S329, when the number is less than the number threshold, determine the alternative interval in the Y-coordinate direction as the noise point interval in the Y-coordinate direction, so that when executing the step of selecting the adjustment interval of the gradient area in the Y-coordinate direction according to the derivative in the Y-coordinate direction, select the noise point interval of the gradient area in the Y-coordinate direction.

[0133] In some embodiments of the present invention, the step of determining the gradient region from the image to be processed specifically includes: performing edge detection on the image using an edge detection algorithm to obtain the gradient region.

[0134] Specifically, Figure 6 As shown, the image to be processed is a grayscale image. Step S100, the step of determining a gradient region from the image to be processed comprises:

[0135] Step S110, calculating the gradient of each pixel in the image;

[0136] Step S120, comparing each gradient magnitude with a preset dual threshold;

[0137] Step S130, determining a gradient region according to the comparison result.

[0138] Specifically, the Sobel operator can be used to detect the edge of the image and obtain the gradient area. The Sobel operator, also known as the Sobel-Federman operator or the Sobel filter, is a classic edge detection operator. When performing edge detection, the partial derivatives GX and GY of each pixel in the X and Y coordinate directions (also known as the scanning direction) are first calculated, and then the gradient size GXY of the pixel is calculated. Then GXY is compared with the set double thresholds (T1, T2, T2>T1). If it is greater than the T2 threshold, it means that the pixel is an image edge. If it is less than the T2 threshold and greater than the T1 threshold, it means that the pixel may have an image gradient, that is, it is a pixel on the gradient area. Otherwise, the pixel is not an edge, but may be a noise point, and filtering processing is required. The gradient area can be determined based on the pixel points on the gradient area. When determining the gradient area and the brightness adjustment area, the partial derivatives GX and GY of each pixel in the X and Y coordinate directions are used, which can make a more complete and accurate judgment on each pixel value of the image.

[0139] In some embodiments of the present invention, step S400 of stretching the grayscale values ​​of the pixels in the brightness adjustment area specifically involves stretching the grayscale values ​​of the pixels in the brightness adjustment area by using a nonlinear transformation.

[0140] In some embodiments of the present invention, the above-mentioned image brightness adjustment method may be used to process the image during image acquisition.

[0141] Specifically, the detector collects image information, and the analog signal is converted into a 16-bit digital signal through an analog-to-digital converter (ADC) and transmitted to the FPGA (Field-Programmable Gate Array). FPGA can efficiently implement image processing algorithms such as image filtering, edge detection, and image compression.

[0142] The image information is filtered by a FIR (Finite Impulse Response) filter on the FPGA side, and the processed image data is transmitted to the DDR3 on the DSP (Digital Signal Processing) side through a Buffer for storage.

[0143] Furthermore, the DSP can be used to implement the steps of processing the image by the above-mentioned image brightness adjustment method. First, the DSP normalizes the image to be processed, converts the 16-bit data signal to 8-bit (0-255 grayscale), and implements image filtering.

[0144] Then use the Sobel operator to detect the edge of the image and get the gradient area. The Sobel operator, also known as the Sobel-Federman operator or Sobel filter, is a classic edge detection operator. When performing edge detection, first calculate the partial derivatives GX and GY of each pixel in the X and Y coordinate directions (also known as the scanning direction), then calculate the gradient size GXY of the pixel, and then compare GXY with the set double thresholds (T1, T2, T2>T1). If it is greater than the T2 threshold, it means that the pixel is an image edge. If it is less than the T2 threshold and greater than the T1 threshold, it means that the pixel may have an image gradient, that is, it is a pixel on the gradient area. Otherwise, the pixel is not an edge.

[0145] The pixel points between the two thresholds T1 and T2 use their first-order derivatives GX and GY to detect the gradient range of the image, that is, to obtain the brightness adjustment area in the gradient area. The specific method is described by taking the X coordinate direction as an example. When detecting the first-order derivative in the X coordinate direction, when the first value is not 0, the point is defined as a grayscale transition point, that is, the first pixel point, and the grayscale value of the point is recorded. Then, when the first value is 0, or when the edge of the gradient area appears, the point is defined as another grayscale transition point, that is, the second pixel point. A counter is used to count the non-zero first-order derivatives between the two grayscale transition points. If the count value is greater than a certain gradient range, it means that the image is a gradient image within the range, and the image grayscale value within the range needs to be stretched. If it is less than the range, it can be considered that the pixel points in the interval may be a noise image or an abnormal scan image introduced by other factors, and the pixel points in the small range are secondary filtered by smoothing and other methods. By stretching the grayscale value of the brightness adjustment area and filtering the image noise area, the image in the gradient area can display more details and further filter out the noise.

[0146] Furthermore, if Figure 7As shown in the figure, the image is divided into a high brightness area (grayscale value is 0-127) and a low brightness area (grayscale value is 128-255) with the horizontal dotted line as the boundary. The pixels in the A and D areas remain unchanged, and the first-order differential value of the area (that is, the derivative value in the X-coordinate direction) is 0. When the pixel point suddenly changes to the B area, the pixel point is a grayscale transition point; in the C and F areas, the gradient value is greater than T2, then C is the edge from low brightness to high brightness, and F is the edge from high brightness to low brightness; at the pixel point where the gradient threshold is between T1 and T2, the first-order differential value is used to identify the gradient area, and the non-zero differential value in the area is counted by the counter. If the count value is greater than the gradient range threshold, such as the B and E areas in the figure, the area is identified as the adjustment interval that needs to be adjusted, and the pixel values ​​corresponding to the maximum and minimum values ​​are recorded, and the pixel values ​​of the area are stretched; if the count value is less than the threshold, such as Figure 2 If the G area in the image is identified as image noise, a secondary filtering process is performed on the area.

[0147] like Figure 8 As shown, the grayscale value of the pixel points in the brightness adjustment area is stretched by nonlinear transformation, such as gamma transformation, which is a nonlinear operation technology widely used in image processing, mainly used to adjust the grayscale of the image, thereby controlling the brightness of the image and enhancing the visual effect. The degree of scaling of the image transformation can be controlled by controlling the gamma factor in the gamma transformation. The corresponding nonlinear transformation curve is shown in the figure. At this time, the image brightness adjustment method of the embodiment of the present invention is completed.

[0148] Furthermore, the image processed by the algorithm (the image after image brightness adjustment) is transmitted back to the DDR4 on the FPGA side. The host computer software reads the image data of the DDR4 on the FPGA side through the interface and performs imaging display.

[0149] Compared with the existing image enhancement algorithm, the image brightness adjustment method of the embodiment of the present invention can further display more image details, and the algorithm can further implement image filtering to obtain a scanning electron microscope image with more information.

[0150] Specifically, the image brightness adjustment method of the embodiment of the present invention introduces the stretching of the local gradient image to display more details of the scanned image, avoiding the stretching or compression of the pixel values ​​in the high brightness interval and the compression or stretching of the pixel values ​​in the low brightness interval of the image as a whole when the nonlinear transformation is directly performed. This method identifies the gradient area of ​​the image through the first-order derivative of the pixel point, and stretches the image based on the gray value interval of the image gradient, so that the image details within the range are clearer. In other words, whether it is a high brightness area or a low brightness area, it will be effectively stretched, so that more image details can be displayed, and the image detail information of both high brightness gradient areas and low brightness gradient areas in the image can be identified.

[0151] That is to say, the image brightness adjustment method of the embodiment of the present invention uses dual threshold judgment to identify high brightness areas and low brightness image areas, and performs further nonlinear transformation processing to achieve a gradient image in the high brightness area and a gradient image in the low brightness area, thereby better identifying the convex area and the concave area in the tested sample; the method can also further filter out the noise information appearing in the image and improve the image signal-to-noise ratio. The grayscale image realized by the method of the embodiment of the present invention can largely meet the recognition of more detailed information of the image.

[0152] In addition, this method can use DSP to implement the algorithm function, effectively reducing the memory used by the host computer and effectively reducing the running time of the host computer. Of course, this method can also be used directly in the host computer. When the host computer performs image processing algorithms, the working efficiency of the main program will also be reduced. When using DSP for image processing, data acquisition and processing are all implemented by the underlying logic, which can reduce the working time of the main program.

[0153] The flow chart provided by the present embodiment is not intended to indicate that the operation of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical thinking provided by the present embodiment method, additional changes may be made to the above method.

[0154] It should be understood that in some embodiments, each part can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0155] This embodiment also provides a computer program product 10 , a computer readable storage medium 20 , and a computer device 30 . Fig. 9 is a schematic diagram of a computer program product 10 according to an embodiment of the present invention, Fig.10 is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention, Fig.11is a schematic diagram of a computer device 30 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which implements the steps of any of the above-mentioned image brightness adjustment methods when executed by a processor 32. The computer-readable storage medium 20 stores the above-mentioned computer program 11, which implements the steps of any of the above-mentioned image brightness adjustment methods when executed by the processor 32. The computer device 30 may include a memory 31, a processor 32, and the computer program 11 stored in the memory 31 and running on the processor 32.

[0156] The computer program 11 for performing the operation of the present invention may be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting data, a configuration data of an integrated circuit, or a source code or an object code written in any combination of one or more programming languages ​​and process programming languages. The computer program 11 may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer. In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0157] In the description of this embodiment, the computer program product 10 is a related product including the computer program 11 .

[0158] For the purpose of the description of the present embodiment, the computer readable storage medium 20 is a tangible device capable of retaining and storing the computer program 11, which can be any device that can contain, store, communicate, propagate or transmit the computer program 11 for use with an instruction execution system, device or apparatus or in conjunction with these instruction execution systems, devices or apparatuses. More specific examples (a non-exhaustive list) of the computer readable storage medium 20 include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination of the above.

[0159] The computer device 30 may be, for example, a server, a desktop computer, a notebook computer, a tablet computer, or a smart phone. In some examples, the computer device 30 may be a cloud computing node. The computer device 30 may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, a program module may include routines, programs, target programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 30 may be implemented in a distributed cloud computing environment where remote processing devices linked via a communication network perform tasks. In a distributed cloud computing environment, program modules may be located on a local or remote computing system storage medium including a storage device.

[0160] The computer device 30 may include a processor 32 adapted to execute stored instructions, and a memory 31 providing temporary storage space for the operation of instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 31 may include a random access memory (RAM), a read-only memory, a flash memory, or any other suitable storage system.

[0161] The computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows data to be input and output with external devices that may be connected to the computer device. The network adapter / interface may provide communication between the computer device and a network, which is typically shown as a communication network.

[0162] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A method for adjusting image brightness, characterized in that: include: Determining a gradient region from the image to be processed; Calculate the derivative of the pixel point in the gradient area, recorded as the pixel point derivative; Determine a brightness adjustment area from the gradient area according to the pixel point derivative; Grayscale values ​​of pixels within the brightness adjustment area are stretched.

2. The image brightness adjustment method according to claim 1, characterized in that: The pixel point derivatives include the derivatives of the pixel point in the X coordinate direction and the Y coordinate direction respectively; The step of determining the brightness adjustment area from the gradient area according to the pixel point derivative comprises: Selecting an adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction, and selecting an adjustment interval of the gradient region in the Y-coordinate direction according to the derivative in the Y-coordinate direction; The brightness adjustment area is determined according to the adjustment interval of the gradient area in the X-coordinate direction and the adjustment interval in the Y-coordinate direction.

3. The image brightness adjustment method according to claim 2, characterized in that: Also includes: When executing the step of selecting the adjustment interval of the gradient region in the X-coordinate direction according to the derivative in the X-coordinate direction, selecting the noise point interval of the gradient region in the X-coordinate direction; When executing the step of selecting the adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction, selecting the noise point interval of the gradient region in the Y coordinate direction; Determining an image noise area from the gradient area according to the noise area in the X-coordinate direction and the noise area in the Y-coordinate direction of the gradient area; Perform filtering processing on the image noise area.

4. The image brightness adjustment method according to claim 3, characterized in that: The step of selecting the adjustment interval of the gradual change area in the X-coordinate direction according to the derivative in the X-coordinate direction comprises: According to the X-coordinate direction, selecting a candidate interval in the X-coordinate direction from the gradient area according to the derivative in the X-coordinate direction; determining whether the candidate interval in the X-coordinate direction is an adjustment interval in the X-coordinate direction according to the derivative in the X-coordinate direction within the candidate interval in the X-coordinate direction; The step of selecting the adjustment interval of the gradient region in the Y coordinate direction according to the derivative in the Y coordinate direction comprises: According to the Y coordinate direction, selecting a candidate interval in the Y coordinate direction from the gradient area according to the derivative in the Y coordinate direction; Determine whether the candidate interval in the Y-coordinate direction is the adjustment interval in the Y-coordinate direction according to the derivative in the Y-coordinate direction within the candidate interval in the Y-coordinate direction.

5. The image brightness adjustment method according to claim 4, characterized in that: According to the X-coordinate direction, the two ends of each candidate interval in the X-coordinate direction are a first pixel point and a second pixel point respectively; the derivative in the X-coordinate direction of each pixel point between the first pixel point and the second pixel point is a non-zero value; the derivative in the X-coordinate direction of the first pixel point is a non-zero value; the derivative in the X-coordinate direction of the second pixel point is a zero value, or the derivative in the X-coordinate direction of the second pixel point is a non-zero value and the second pixel point is at the edge of the gradient area; According to the Y-coordinate direction, the two ends of each alternative interval in the Y-coordinate direction are the third pixel point and the fourth pixel point respectively; the derivative of the Y-coordinate direction of each pixel point between the third pixel point and the fourth pixel point is a non-zero value; the derivative of the Y-coordinate direction of the third pixel point is a non-zero value; the derivative of the Y-coordinate direction of the fourth pixel point is 0, or the derivative of the Y-coordinate direction of the fourth pixel point is a non-zero value and the fourth pixel point is at the edge of the gradient area.

6. The image brightness adjustment method according to claim 5, characterized in that: The step of selecting a candidate interval in the X-coordinate direction from the gradient area according to the X-coordinate direction and the derivative in the X-coordinate direction comprises: According to the X-coordinate direction, determine whether the first non-zero derivative in the X-coordinate direction appears; If yes, the pixel point corresponding to the derivative in the X-coordinate direction is taken as the first pixel point; and whether the first derivative in the X-coordinate direction with a value of 0 appears is further determined according to the X-coordinate direction; When the first derivative in the X-coordinate direction with a value of 0 is determined, the pixel point corresponding to the derivative in the X-coordinate direction is used as the second pixel point, and the first derivative in the X-coordinate direction with a value other than 0 is determined according to the X-coordinate direction until the last pixel point is determined; When the derivative in the X-coordinate direction in which the first 0 value does not appear continues to be determined according to the X-coordinate direction, the last pixel point is used as the second pixel point to obtain at least one candidate interval; The step of selecting a candidate interval in the Y coordinate direction from the gradient area according to the Y coordinate direction and the derivative in the Y coordinate direction comprises: According to the Y coordinate direction, determine whether the first non-zero derivative in the Y coordinate direction appears; If yes, the pixel point corresponding to the derivative in the Y coordinate direction is used as the third pixel point; and whether the first derivative in the Y coordinate direction with a value of 0 appears is further determined according to the Y coordinate direction; When the first derivative in the Y coordinate direction of 0 is determined, the pixel point corresponding to the derivative in the Y coordinate direction is taken as the fourth pixel point, and the first derivative in the Y coordinate direction of a non-zero value is determined according to the Y coordinate direction until the last pixel point is determined; When the derivative in the Y coordinate direction without the first 0 value continuing to be determined in the Y coordinate direction, the last pixel point is taken as the fourth pixel point to obtain at least one candidate interval.

7. The image brightness adjustment method according to claim 5, characterized in that: The step of determining whether the candidate interval of the X-coordinate direction is the adjustment interval of the X-coordinate direction according to the derivative of the X-coordinate direction within the candidate interval of the X-coordinate direction comprises: Obtaining the number of non-zero derivatives in the X-coordinate direction within the candidate interval in the X-coordinate direction, including the derivative in the X-coordinate direction of the first pixel point and the derivative in the X-coordinate direction of the second pixel point; When the number is greater than or equal to the number threshold, determining the candidate interval in the X-coordinate direction as the adjustment interval in the X-coordinate direction; The step of determining whether the candidate interval in the Y coordinate direction is the adjustment interval in the Y coordinate direction according to the derivative in the Y coordinate direction in the candidate interval in the Y coordinate direction comprises: Obtaining the number of non-zero derivatives in the Y-coordinate direction within the candidate interval in the Y-coordinate direction, including the derivative in the Y-coordinate direction of the third pixel point and the derivative in the Y-coordinate direction of the fourth pixel point; When the number is greater than or equal to the number threshold, the candidate interval in the Y coordinate direction is determined as the adjustment interval in the Y coordinate direction.

8. The image brightness adjustment method according to claim 7, characterized in that: Also includes: When the number is less than the number threshold, determining the candidate interval in the X-coordinate direction as the noise point interval in the X-coordinate direction, so that when performing the step of selecting the adjustment interval of the gradient area in the X-coordinate direction according to the derivative in the X-coordinate direction, the noise point interval of the gradient area in the X-coordinate direction is selected; When the number is less than the number threshold, the alternative interval in the Y-coordinate direction is determined to be the noise point interval in the Y-coordinate direction, so that when executing the step of selecting the adjustment interval of the gradient area in the Y-coordinate direction according to the derivative in the Y-coordinate direction, the noise point interval of the gradient area in the Y-coordinate direction is selected.

9. The image brightness adjustment method according to claim 1, characterized in that: The graphics to be processed are grayscale images; the step of determining the gradient area from the image to be processed comprises: Calculate the gradient of each pixel in the image; Comparing each of the gradient magnitudes with a preset dual threshold; The gradient region is determined according to the comparison result.

10. The image brightness adjustment method according to claim 1, characterized in that: The grayscale values ​​of the pixels in the brightness adjustment area are stretched by nonlinear transformation.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image brightness adjustment method according to any one of claims 1 to 10 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the image brightness adjustment method according to any one of claims 1 to 10 are implemented.

13. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the image brightness adjustment method according to any one of claims 1 to 10.

Images