An uneven illumination correction method for pipette images based on an improved Sobel operator

Abstract

An uneven illumination correction method for pipette images based on an improved Sobel operator. The purpose is to remove the inaccurate threshold segmentation caused by uneven illumination for subsequent image segmentation. This method includes the following steps: Step 1, perform grayscale processing on the original image using the weighted average method to generate a grayscale image and reduce the computational amount of the original image; Step 2, perform bilateral filtering on the grayscale image obtained in Step 1 to remove Gaussian noise in the image; Step 3, use the Otsu method to obtain the threshold for threshold segmentation of the image obtained in Step 2; Step 4, use the improved top-hat algorithm to remove the reflection on the pipette image; Step 5, use the improved Sobel operator to perform edge detection in four directions on the image. After roughly locating the foreground object, ignore the edges and use the brightening operator designed based on the Gaussian formula to perform brightening processing on the image. This algorithm can effectively correct the influence of uneven illumination of the pipette image on threshold segmentation.

Description

Technical Field:

[0001] The present invention relates to the technical field of micro-nano vision target extraction, and particularly to a method for correcting uneven illumination of a pipette image based on an improved Sobel operator.

[0002] Background Art:

[0003] With the progress and maturity of macroscopic industrial operations, people have increasingly shifted their focus to micro-operation technologies. In the micro-environment, automatic operation technologies cannot be separated from the combination with vision. However, at the micro-nano scale, the research object is too small, and even a slight change in environmental factors will have a great impact on the operation. For example, the imaging problems caused by uneven illumination that exist in macroscopic operations also exist and are exacerbated in micro-vision. Therefore, it is very difficult to correct uneven illumination at the physical level in the micro-environment, and it is necessary to study an image algorithm for correcting uneven illumination.

[0004] Uneven illumination will cause shadows to appear at the corners of the image, which has a great impact on image threshold segmentation, especially for micro-nano level images, and further affects subsequent visual positioning. Therefore, eliminating the shadows caused by uneven illumination has very important research significance for image visual positioning. Currently, this problem can be mainly solved in two directions. One is to use deep learning technology to utilize the current powerful hardware support to learn the illumination distribution of the image through a large number of data sets to identify and correct shadows; the other is to design image algorithms using traditional image processing technologies to improve the contrast between the foreground and the background, or to filter out the shadows generated by uneven illumination through filtering, thereby correcting or weakening the impact of uneven illumination. Compared with traditional image algorithms, deep learning is more targeted, easier and more stable to implement, but its recognition accuracy depends on a large number of data sets and the quality of the data sets. The traditional method is calculated in real time, has a wider application range but requires a certain amount of time, so the lower the complexity of the algorithm, the better. Currently, deep learning has a wider application range and is widely used in the field of industrial vision, while traditional image algorithms for correcting uneven illumination are mainly applied to the threshold segmentation of text images with less noise, and are less applied and difficult to implement in other fields. Summary of the Invention:

[0005] The purpose of the present invention is to solve the influence of uneven illumination on the threshold segmentation of a pipette image, and to propose a method for correcting uneven illumination of a pipette image based on an improved Sobel operator.

[0006] A method for correcting uneven illumination of a pipette image based on an improved Sobel operator has the following specific process:

[0007] Step 1. Perform gray-scale processing on the original image by the weighted average method to generate a gray-scale image, reducing the computational amount of the original image;

[0008] Step 2. Perform bilateral filtering on the grayscale image obtained in Step 1 to remove Gaussian noise from the picture;

[0009] Step 3. Use Otsu's method to obtain the threshold for threshold segmentation of the image obtained in Step 2;

[0010] Step 4. Use the improved top-hat algorithm to remove the reflection on the pipette image;

[0011] Step 5. Use the improved Sobel operator to perform edge detection in four directions on the image. After roughly locating the foreground object, ignore the edges and use the brightening operator designed based on the Gaussian formula to brighten the image;

[0012] Furthermore, the bilateral filtering of the grayscale image in Step 2 includes the following steps:

[0013] 2.1 Construct the spatial distance domain weighting coefficient G based on the Gaussian formula s (i,j) formula, grayscale domain weighting coefficient G r The (i,j) formula is as follows:

[0014]

[0015]

[0016] 2.2 Then the total weighting coefficient of the bilateral filtering is:

[0017] G(i,j) = G s (i,j)G r (i,j) (3)

[0018] 2.3 Let f BF (x,y) be the image after bilateral filtering, and N(i,j) be the neighborhood pixel space of the central pixel I(x,y). The formula for bilateral filtering is as follows:

[0019]

[0020] 2.4 Use this formula to generate a 3×3 convolution kernel to perform convolution processing on the entire image, complete bilateral filtering, remove Gaussian noise, and retain the edge information of the original image.

[0021] Furthermore, the use of Otsu's method to obtain the threshold for threshold segmentation of the image in Step 3 includes the following steps:

[0022] 3.1 Calculate the total number of pixels in the image;

[0023] 3.2 Calculate the pixel ratio of each grayscale value;

[0024] 3.3 Calculate the between-class variance when each grayscale value is used as a threshold;

[0025] 3.4 Find the between-class variance with the largest grayscale value as the threshold for threshold segmentation.

[0026] Furthermore, removing the reflection on the pipette image using the improved top-hat algorithm in step 4 includes the following steps:

[0027] 4.1 Perform morphological opening on the image to erase the reflective part;

[0028] 4.2 Then subtract the opened image from the original image, only retaining the reflective part and removing the expanded part of the foreground object caused by the opening operation;

[0029] 4.3 Then subtract the top-hat image from the original image to remove the shadow. At the same time, add a weight value k according to the actual situation to make the grayscale values of the reflective area and the non-reflective area the same after subtraction to obtain the best effect. Let the image after removing the reflection be f eq (i, j), and the formula is as follows:

[0030] f eq (i, j) = k[[f Θ S](i, j) Δ S](i, j) + (1 - k)f(i, j) (5)

[0031] Furthermore, in step 5, using the improved Sobel operator to perform edge detection in four directions on the image, after roughly locating the foreground object and ignoring the edges, using the brightening operator designed based on the Gaussian formula to perform brightening processing on the image includes the following steps:

[0032] 5.1 Design edge detection operators in four diagonal directions as Figure 2 shown. After performing edge detection on the image in four directions using this edge detection operator, obtain I1, I2, I3, I4, and add the four images to get I5:

[0033] I5(i, j) = I1(i, j) + I2(i, j) + I3(i, j) + I4(i, j) (6)

[0034] 5.2 Perform threshold segmentation on the obtained image I5 to remove the noise generated during the gradient edge detection process, and then perform a morphological closing operation on the image to obtain image I close , expanding the edge range.

[0035] 5.3 Combine the edge information on image I close and ignore these edge positions. Use the Figure 3 brightening operator based on the Gaussian formula to perform brightening filtering on the image from the center point to the four corners.

[0036] 5. Combine the segmentation threshold calculated in Step 2 to perform threshold segmentation on the brightened image.

[0037] The present invention has the following advantages compared with the prior art: The method for correcting uneven illumination of a pipette image by improved Sobel edge detection first uses bilateral filtering to filter Gaussian noise, enhance the details of the image, and retain the edge information of the image. Then, it first uses the Otsu method to calculate the foreground-background segmentation threshold of the image, and then uses the improved top-hat algorithm to remove the reflection on the surface of the pipette to avoid the problem of large hollow areas after threshold segmentation. Finally, it first combines the improved Sobel edge detection operator to perform edge detection on the image in four directions to roughly locate the foreground object, and then uses this algorithm to only increase the brightness of the background, enhance the contrast between the foreground and the background, greatly correct the influence of the shadow caused by uneven illumination, and effectively improve the threshold segmentation quality of the pipette image. Description of the Drawings:

[0038] Figure 1 is the algorithm flow chart;

[0039] Figure 2 is the improved Sobel edge detection operator;

[0040] Figure 3 is the brightening convolution kernel based on the Gaussian formula; Specific Embodiments:

[0041] In order to make the objectives, technical solutions and advantages of the present invention clearer, the following further describes the present invention in detail with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the present invention.

[0042] The method for correcting uneven illumination of a pipette image based on the improved Sobel operator in this embodiment is carried out according to the following steps:

[0043] Step 1: Perform grayscale processing on the original image by the weighted average method to generate a grayscale image and reduce the computational amount of the original image.

[0044] Use the formula I Gray (x,y) = 0.3×I R (x,y) + 0.59×I G (x,y) + 0.11×I B (x,y) to perform grayscale processing on the image, where I Gray represents the grayscale image, and I R , I G , I B represent the values of the red, green, and blue primary colors in the pixels of the original image. The generated image is as shown in Figure 1 shown.

[0045] Step 2: Perform bilateral filtering on the grayscale image obtained in Step 1 to remove Gaussian noise from the picture;

[0046] The core of bilateral filtering is the Gaussian formula, and the formula is as follows:

[0047]

[0048] It can be found from the formula that Gaussian filtering combines the pixels around the pixel with itself for filtering, so the image will become blurrier, and thus the edge information of the image will be lost. Bilateral filtering has a certain ability to preserve the edges of the image by introducing the consideration of the spatial distance information and grayscale information of the pixels. Let the center point coordinates be I(x, y), and the introduced spatial distance domain weighting coefficient G s (i, j) formula, and the grayscale domain weighting coefficient G r (i, j) formula are as follows:

[0049]

[0050]

[0051] Among them, σ s is the standard deviation of the spatial domain, and σ r is the standard deviation of the grayscale domain.

[0052] Then the total weighting coefficient of bilateral filtering is:

[0053] G(i, j) = G s (i, j)G r (i, j) (10)

[0054] Let f BF (x, y) be the image after bilateral filtering, and N(i, j) be the neighborhood pixel space of the central pixel I(x, y). The formula for bilateral filtering is as follows:

[0055]

[0056] Using this formula, generate a 3×3 convolution kernel to perform convolution processing on the entire image, complete bilateral filtering, remove Gaussian noise, and retain the edge information of the original image.

[0057] Step 3: Use Otsu's method to obtain the threshold for threshold segmentation of the image obtained in Step 2;

[0058] The Otsu method, also known as the Otsu algorithm, is an adaptive threshold segmentation method that can be based on the maximum between-class variance of the grayscale values of an image. This method assumes that when the between-class variance of the image grayscale values is at its maximum, it represents the optimal threshold for foreground and background segmentation. However, when the illumination is uneven, shadows are likely to appear at the corners of the image. When the grayscale values of the shadows at the corners of the image are similar to those of the foreground objects, the shadows will be misidentified as the foreground. Therefore, this method still has limitations.

[0059] In the Otsu algorithm, assume that L(x, y) is a grayscale image with pixel values ranging from 0 to 255, and N is the total number of pixels in the image. Let n i represent the number of pixels with a grayscale value of i. Then the total number of pixels can be expressed as:

[0060]

[0061] Then assume that P i represents the proportion of pixels with a grayscale value of i, which can be defined as:

[0062]

[0063] Let the binary threshold of the image be T. Then the pixels of the image can be divided into two parts, with their grayscale value intervals being [0, T] and [T + 1, 255] respectively. Let μ0 and μ1 be the grayscale value expectations of the foreground and background, ω0 and ω1 be the sums of the grayscale value probabilities of the background and foreground, and μ be the average grayscale value of all pixels in the image. Then the definitions of each symbol are as follows:

[0064]

[0065]

[0066]

[0067] Based on the above formulas, let the between-class variance be σ. The between-class variance function can be expressed as:

[0068] σ = ω0(μ0 - μ) 2 + ω1(μ1 - μ) 2 (17)

[0069] The Otsu method is to take the maximum between-class variance as the threshold T for threshold segmentation.

[0070] Step 4: Use the improved top-hat algorithm to remove the reflection on the pipette image;

[0071] This algorithm is modified based on the core idea of the morphological top-hat algorithm. The morphological top-hat algorithm mainly performs erosion operation on the original image first and then dilation operation, and then subtracts the image after this morphological operation from the original image. Assuming erosion is Θ, dilation is Δ, f is the original image, and S is the image of the convolution kernel, then the opening operation can be expressed as:

[0072] f hat (i,j) = f(i,j) - [[f Θ S](i,j) Δ S](i,j) (18)

[0073] After the image undergoes the top-hat operation, the white dots or cracks in the large-area black regions of the image are left. And our specular reflection also has similar characteristics, with the general feature being that there are several white lines or several oval-shaped white dots on the large-area black objects. Therefore, the top-hat algorithm can locate the specular reflection region. And to eliminate the shadow, we only need to subtract the top-hat from the original image to remove the shadow. At the same time, we can add a weight value k according to the actual situation to make the gray values of the specular reflection region and the non-specular reflection region the same after subtraction to obtain the best effect. Let the image after removing the specular reflection be f eq (i,j), and the formula is as follows:

[0074] f eq (i,j) = k[[f Θ S](i,j) Δ S](i,j) + (1 - k)f(i,j) (19)

[0075] Step 5: Use the improved Sobel operator to perform edge detection on the image in four directions. After roughly locating the foreground object, ignore the edges and use the brightening operator designed based on the Gaussian formula to perform brightening processing on the image;

[0076] The usual Sobel operator is a 3×3 convolution kernel. And there are some shadows on the edges of our pipette image, and the gray values of these shadows are not much different from the gray values of the pipette, which will affect the effect of gradient edge detection. Therefore, a convolution kernel operator with a larger dimension is used. According to the characteristics of the pipette image, four diagonal edge detection operators are designed as Figure 1 shown. Assume that after performing edge detection on the image in four directions, I1, I2, I3, and I4 are obtained, and the four pictures are added to get I5:

[0077] I5(i,j) = I1(i,j) + I2(i,j) + I3(i,j) + I4(i,j) (20)

[0078] Perform threshold segmentation on the obtained image I5 to remove the noise generated during the gradient edge detection process, and then perform a morphological closing operation on the image to obtain the image I close , expanding the edge range.

[0079] After that, we use a Gaussian convolution kernel such as Figure 2 to perform brightening filtering on the image from the center point to the four corners. This convolution kernel can increase the gray value of a pixel according to the gray values of surrounding pixels. Therefore, when the illumination is uneven for the background, the uneven illumination can be spread by filtering with this operator. However, for the pipette object, the edge of the pipette is connected to the background, so the gray value of the background will be spread to the inside of the pipette. Therefore, it is necessary to combine the edge information on the image I close to ignore these edge positions and perform brightening filtering on the image from the center point to the four corners. Finally, by combining the threshold calculated in step two, threshold segmentation is performed on the brightened image to achieve an ideal effect.

Claims

1. An uneven illumination correction method for pipette images based on an improved Sobel operator, characterized in that The method comprises the following steps: Step 1: Grayscale the original image by weighted average method to generate a grayscale image, reducing the computational amount of the original image; In the first step: Using the formula I Gray (x,y) = 0.3×I R (x,y) + 0.59×I G (x,y) + 0.11×I B (x,y) to perform grayscale processing on the image; Step 2: Perform bilateral filtering on the grayscale image obtained in Step 1 to remove Gaussian noise from the image; In Step 2: Use the bilateral filtering formula to generate a 3×3 convolution kernel to perform convolution on the entire image, removing Gaussian noise and retaining edge information; Step 3: Use Otsu's method to obtain the threshold for threshold segmentation of the image obtained in Step 2; In Step 3: By calculating the between-class variance of the pixel grayscale values of the foreground and background of the image when each grayscale value is used as the threshold, find the grayscale value with the largest between-class variance as the threshold for threshold segmentation; Step 4: Use the improved top-hat algorithm to remove the reflection on the pipette image; In Step 4: The characteristics of the reflection are that there are several white lines or several elliptical white dots on a large area of the object. The top-hat algorithm can locate the reflection area. Subtract the top-hat image from the original image, and then add a weight value k according to the actual situation to make the grayscale values of the reflection area and the non-reflection area the same after subtraction to obtain the best effect. The formula for removing the reflection designed is as follows: f eq (i,j) = k[[fΘS](i,j)ΔS](i,j) + (1 - k)f(i,j) Among them, f is the original image, f eq (i, j) is the image after removing specular reflection, Θ is the erosion operation, Δ is the dilation operation, and S is the convolution kernel; Step 5: Use the improved Sobel operator to perform edge detection in four directions on the image. After roughly locating the foreground object, ignore the edges and use the brightening operator designed based on the Gaussian formula to perform brightening processing on the image; In the fifth step: According to the characteristics of the pipette image, four edge detection operators in the diagonal directions are designed based on the characteristics of the Sobel operator. Assume that four images are obtained after performing edge detection on the image in four directions, and then the four images are added together. The obtained image is subjected to threshold segmentation to remove the noise generated during the gradient edge detection process. Then, a morphological closing operation is performed on the image to expand the edge range to obtain image I. close The brightening convolution kernel designed according to the Gaussian formula can increase the gray value of a pixel point based on the gray values of the surrounding pixels. Therefore, when the illumination is uneven for the background, the uneven illumination can be spread by filtering with this operator. However, the edge of the foreground is connected to the background, so the gray value of the background will be spread into the interior of the foreground. Therefore, it is necessary to combine the edge information on image I close to ignore the edge positions at these edges and perform brightening processing on the image from the center point to the four corners. Finally, in combination with the threshold calculated in the second step, threshold segmentation is performed on the brightened image.

Images