Three-dimensional image splicing method for white light interferometer

Abstract

The invention discloses a three-dimensional image splicing method for a white light interferometer, and the method comprises the steps: placing a to-be-detected sample on an objective table, and enabling a motor to move and collect in a snakelike path according to a set direction and step pitch; according to a rule of platform movement and data acquisition during large-range measurement of the white light interferometer, a data alignment mode can be automatically judged, an optimal feature matching path can be identified, and image matching, splicing and cutting can be rapidly realized; and by utilizing the characteristic that the motor only translates, comparing the normalized standard deviation of the overlapped area, automatically searching the optimal matching area, identifying the actual overlapped area and finishing image splicing, the method has the characteristics of simple algorithm, small calculation amount, high speed and high efficiency, the requirement on the motor precision is low, the system cost is low, the characteristic area does not need to be manually judged and selected, and the method is suitable for large-scale popularization and application. The method can automatically identify overlapped areas and complete image splicing, and is high in automation and intelligence degree, low in sample requirement, suitable for large-scale measurement of most samples, and high in universality.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a three-dimensional image stitching method for a white light interferometer. Background Art

[0002] White light interferometry is one of the most accurate technologies for measuring the surface morphology of materials. By determining the position of the coherent envelope peak and combining the phase information of the fringes, accurate measurement at the sub-nanometer level can be achieved.

[0003] Although white light interferometer has extremely high measurement accuracy and resolution, it is limited by the field of view and has a very small horizontal measurement range, which cannot meet the measurement of large-scale horizontal surface morphology. There are two existing methods for white light interferometer to achieve large-scale measurement. One is image stitching based on hardware devices, that is, high-precision stitching is achieved through external hardware devices such as motors and grating rulers, but this method is costly and has poor system flexibility; the other is matching stitching based on the surface geometric structure characteristics of the sample to be measured, but the existing image stitching method has problems such as complex algorithms, large amount of calculations, slow speed, poor versatility, and low stitching accuracy, which cannot meet the needs of large-scale measurement of white light interferometer.

[0004] Patent 1 (CN105509635A) proposes a method for converting noise into usable information to improve stitching accuracy. In the feature matching stage, the noise is converted into usable features to improve the result of feature matching; in the image fusion stage, the interference of the noise area on the data fusion is reduced to obtain a three-dimensional stitching result with higher accuracy. In view of the unique functional and structural characteristics of MEMS, the maximum information entropy principle is used to solve the event area boundary of the MEMS three-dimensional image to achieve the feature segmentation of the MEMS three-dimensional image. Then, based on the result of feature segmentation, other features other than noise are used for optimized matching and high fusion, reducing the impact of noise on three-dimensional image stitching and improving stitching accuracy. Finally, the data after excluding noise points is image fused to reduce the interference of the noise area on data fusion and obtain a three-dimensional stitching result with higher accuracy.

[0005] The method proposed in Patent 1 converts noise into usable features to improve feature matching results, but there is uncertainty in the generation of noise, which may have a negative impact on feature matching. The accuracy of noise recognition has a greater impact on the stitching results and may introduce additional errors, resulting in reduced stitching accuracy. Patent 1 also proposes a method for image stitching by performing feature segmentation and matching on the edge information of the sample surface, but this method is mainly aimed at large-range measurements of MEMS. For other samples, matching errors are prone to occur, the matching accuracy is low, and the versatility of this method is poor.

[0006] Patent 2 (CN112184551A) proposes a method of combining the height quality matrix with the three-dimensional image stitching technology, which reduces the impact of image noise on three-dimensional stitching. The method first observes the data of the roughly overlapping area of ​​the two images, determines a small sub-area in the roughly overlapping area as the feature area, and uses the gradient algorithm and the Harris corner point algorithm to obtain the gradient matrix and corner point matrix of the reference three-dimensional image; calculates the gradient matrix and corner point matrix of each candidate feature area in the roughly overlapping part of the three-dimensional image to be stitched, and obtains the height quality matrix of the area through the height quality evaluation method, and then calculates the feature evaluation coefficient of each candidate feature area and the feature area, and reconstructs the height of each area according to the height quality matrix and the feature evaluation coefficient, calculates the correlation between the reconstructed height matrix of the feature area and the reconstructed height matrix of the matching area, and determines the area with the largest correlation coefficient as the optimal matching area, and moves the image to be stitched so that the optimal matching area coincides with the boundary of the feature area of ​​the reference image. The height of the measured points in the overall area after stitching is benchmarked and fused, and combined with the height quality evaluation matrix, an accurate height fusion result without noise influence is obtained to complete the three-dimensional image stitching.

[0007] There is no clear standard for the threshold setting of the height quality evaluation method proposed in Patent 2. The threshold setting is greatly affected by factors such as samples and lighting, and is prone to recognition errors, resulting in reduced stitching accuracy. In addition, the method proposed in this patent requires manual judgment and selection of feature areas, has poor flexibility, and low levels of automation and intelligence.

[0008] Based on the above problems, the present invention proposes a three-dimensional image stitching method for white light interferometer that can solve the above problems. Summary of the invention

[0009] The purpose of the present invention is to provide a three-dimensional image stitching method for white light interferometer to solve the problems raised in the above background technology.

[0010] To achieve the above object, the present invention provides the following technical solution: a three-dimensional image stitching method for white light interferometer, comprising the following steps:

[0011] S1. Place the sample to be tested on the stage, set the splicing direction, quantity and motor step length on the software program, and the motor moves and collects in a serpentine path according to the set direction and step length;

[0012] S2. After completing data collection and three-dimensional surface morphology reconstruction in all regions, all three-dimensional image data are arranged according to the actual sample image distribution as image data to be spliced;

[0013] S3, moving the right image to be spliced ​​to the left column by column, so that the two images gradually overlap, normalizing the data of the overlapping area respectively, and then subtracting and calculating the standard deviation, until the last column of the right image overlaps with the left image;

[0014] S4, taking the overlapped area with the smallest standard deviation as the actual overlapped area of ​​the image data collected twice adjacently;

[0015] S5, taking the left image as a reference, subtracting the height data of the overlapping area of ​​the left image from the height data of the overlapping area of ​​the right image, taking the average height difference as the height difference of the two images, adding all the data of the right image to the obtained height difference, and then taking the average of the overlapping area of ​​the left image and the right image by pixel as the value of the pixel point, after completing the image fusion, perform the next column splicing until all the columns of images have completed the column splicing;

[0016] S6. After completing the column stitching of all columns of image data, start row stitching, move the lower image upward row by row, normalize the data of the overlapping areas respectively, and then subtract and calculate the standard deviation, until the last row of the lower image overlaps with the upper image;

[0017] S7, taking the overlapped area with the smallest standard deviation as the actual overlapped area of ​​the image data collected twice adjacently;

[0018] S8, taking the upper image as a reference, subtracting the height data of the overlapping area of ​​the lower image from the height data of the overlapping area of ​​the upper image, taking the average height difference as the height difference of the two images, adding all the data of the lower image to the obtained height difference, and then taking the average of the overlapping area of ​​the upper image and the lower image by pixel as the value of the pixel point, after completing the image fusion, perform the next row stitching until all rows of images are stitched together;

[0019] S9. After completing row stitching, the columns without valid data at the edge of the image are trimmed to complete image stitching.

[0020] As a preferred technical solution, in step S6, the number of times the rows are spliced ​​is determined, and if it is an odd number, it is right-aligned, and if it is an even number, it is left-aligned.

[0021] As a preferred technical solution, in step S6, every time the lower image moves one row, the data of the current overlapping area of ​​the lower image and the upper image are normalized, and then the standard deviation is calculated by subtracting them, until the last row of data of the upper image and the lower image overlaps and the normalization and standard deviation calculation are completed.

[0022] As a preferred technical solution, in step S3, each time the right image moves one column, the data of the current overlapping area of ​​the left image and the right image are normalized, and then the standard deviation is calculated by subtracting them until the last column of data of the left image and the right image overlap and the normalization and standard deviation calculation are completed.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] According to the rules of platform movement and data collection during large-scale measurement by white light interferometer, the data alignment method can be automatically determined, the best feature matching path can be identified, and image matching, stitching and cropping can be quickly realized. And by utilizing the characteristic that the motor only moves in translation, the normalized standard deviation of the overlapping area can be compared, the best matching area can be automatically searched, the actual overlapping area can be identified, and image stitching can be completed.

[0025] It has the characteristics of simple algorithm, small calculation amount, fast speed and high efficiency, low requirements for motor accuracy, low system cost, no need for manual judgment and selection of feature areas, can automatically identify overlapping areas and complete image stitching, high degree of automation and intelligence, low requirements for samples, suitable for large-scale measurement of most samples, and strong versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a schematic diagram of the serpentine path image acquisition of the present invention;

[0027] Figure 2 This is a schematic diagram of the splicing of columns of the present invention;

[0028] Figure 3 This is a schematic diagram of the splicing of the present invention;

[0029] Figure 4 This is a schematic diagram of image cropping after splicing according to the present invention;

[0030] Figure 5 This is the image stitching flow chart of the present invention; DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1-5 The present invention provides a three-dimensional image stitching method for white light interferometer:

[0033] Place the sample to be tested on the stage, set the direction, quantity and motor step of the splicing on the software program, and the motor moves and collects in a serpentine path according to the set direction and step, and completes the three-dimensional surface morphology reconstruction and other post-processing. Figure 1 This is a schematic diagram of the serpentine path. The data collection path is shown by the arrow in the figure.

[0034] After completing data acquisition and three-dimensional surface morphology reconstruction in all areas, all three-dimensional image data are arranged in order according to the actual sample image distribution as image data to be spliced.

[0035] Because the motor only performs translation in the front, back, left, and right directions, the coordinates of the image feature information of the overlapping area in two consecutively acquired images only have a single-direction translation, and there is no offset or rotation in multiple directions. By using this feature, the feature information of the two consecutively acquired images can be compared row by row or column by column in the direction of translation, and the data to be spliced ​​can be spliced ​​in the order of columns first and rows later. The specific operations are as follows:

[0036] Move the right image to be spliced ​​to the left column by column to make the two images gradually overlap. Every time the right image moves one column, the current overlapping area data of the left and right images are normalized, and then the standard deviation is calculated by subtracting them until the last column of data of the left and right images overlaps and the normalization and standard deviation calculation are completed. The overlapping area with the smallest standard deviation in this process is regarded as the actual overlapping area of ​​the two images.

[0037] Then, taking the left image as the reference, subtract the height data of the overlapping area of ​​the left image from the height data of the overlapping area of ​​the right image, take the average height difference as the height difference of the two images, add all the data of the right image to the obtained height difference, and then take the average of the overlapping area of ​​the left and right images as the value of the pixel point, complete one column stitching, and proceed to the next column stitching. Repeat the above operation until all columns of images have completed column stitching. Figure 2 Schematic diagram of column splicing. The black solid line area is the overlapping area of ​​the two images, and the arrow represents the movement direction.

[0038] After completing the column stitching, start stitching the rows. Because the motor may have low positioning accuracy or lost steps, the length of each row of image data may not be consistent, but the data of the last image acquired in the current row and the first image acquired in the next row are aligned, and the motor moves in a serpentine path. Therefore, when stitching the rows, odd-numbered stitching is right-aligned, and even-numbered stitching is left-aligned. Move the lower image to be stitched upward row by row to gradually overlap the two images. For each row the lower image moves, normalize the data of the current overlapping area of ​​the lower and upper images, and then subtract them to calculate the standard deviation until the last row of data of the upper and lower images overlap and complete the normalization and standard deviation calculation. The overlapping area with the smallest standard deviation in this process is taken as the actual overlapping area of ​​the two images.

[0039] Then, taking the above picture as the reference, subtract the height data of the overlapping area of ​​the lower picture from the height data of the overlapping area of ​​the upper picture, take the average height difference as the height difference of the two pictures, add all the data of the lower picture to the obtained height difference, and then take the average of the overlapping area of ​​the upper and lower pictures as the value of the pixel point, complete one row stitching, and proceed to the next row stitching, repeat the above operation according to the stitching rule of odd and even numbers until all images have completed row stitching. Figure 3 Schematic diagram of row stitching. The black solid line area is the overlapping area of ​​the two images, and the arrow represents the moving direction.

[0040] After completing the row stitching, the columns without valid data in the stitched image are cropped to complete the image stitching and realize large-scale surface morphology measurement, such as Figure 4 As shown, the dotted line area is the overlapping area of ​​the two images, and the black solid line area is the area retained after cropping.

[0041] According to the rules of platform movement and data collection during large-scale measurement by white light interferometer, the data alignment method can be automatically determined, the best feature matching path can be identified, and image matching, stitching and cropping can be quickly realized. And by utilizing the characteristic that the motor only moves in translation, the normalized standard deviation of the overlapping area can be compared, the best matching area can be automatically searched, the actual overlapping area can be identified, and image stitching can be completed.

[0042] It has the characteristics of simple algorithm, small calculation amount, fast speed and high efficiency, low requirements for motor accuracy, low system cost, no need for manual judgment and selection of feature areas, can automatically identify overlapping areas and complete image stitching, high degree of automation and intelligence, low requirements for samples, suitable for large-scale measurement of most samples, and strong versatility.

[0043] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional image stitching method for white light interferometer, characterized in that: The following steps are involved: S1. Place the sample to be tested on the stage, set the splicing direction, quantity and motor step length on the software program, and the motor moves and collects in a serpentine path according to the set direction and step length; S2. After completing data collection and three-dimensional surface morphology reconstruction in all regions, all three-dimensional image data are arranged according to the actual sample image distribution as image data to be spliced; S3, moving the right image to be spliced ​​to the left column by column, so that the two images gradually overlap, normalizing the data of the overlapping area respectively, and then subtracting and calculating the standard deviation, until the last column of the right image overlaps with the left image; S4, taking the overlapped area with the smallest standard deviation as the actual overlapped area of ​​the image data collected twice adjacently; S5, taking the left image as a reference, subtracting the height data of the overlapping area of ​​the left image from the height data of the overlapping area of ​​the right image, taking the average height difference as the height difference of the two images, adding all the data of the right image to the obtained height difference, and then taking the average of the overlapping area of ​​the left image and the right image by pixel as the value of the pixel point, after completing the image fusion, perform the next column splicing until all the columns of images have completed the column splicing; S6. After completing the column stitching of all columns of image data, start row stitching, move the lower image upward row by row, normalize the data of the overlapping areas respectively, and then subtract and calculate the standard deviation, until the last row of the lower image overlaps with the upper image; S7, taking the overlapped area with the smallest standard deviation as the actual overlapped area of ​​the image data collected twice adjacently; S8, taking the upper image as a reference, subtracting the height data of the overlapping area of ​​the lower image from the height data of the overlapping area of ​​the upper image, taking the average height difference as the height difference of the two images, adding all the data of the lower image to the obtained height difference, and then taking the average of the overlapping area of ​​the upper image and the lower image by pixel as the value of the pixel point, after completing the image fusion, perform the next row stitching until all rows of images are stitched together; S9. After completing row stitching, the columns without valid data at the edge of the image are trimmed to complete image stitching.

2. A three-dimensional image stitching method for white light interferometer according to claim 1, characterized in that: In step S6, the number of times the rows are spliced ​​is determined, and if it is an odd number, it is right-aligned, and if it is an even number, it is left-aligned.

3. A three-dimensional image stitching method for white light interferometer according to claim 2, characterized in that: In step S6, each time the lower image moves one row, the data of the current overlapping area of ​​the lower image and the upper image are normalized, and then the standard deviation is calculated by subtracting them, until the last row of data of the upper image and the lower image overlaps and the normalization and standard deviation calculation are completed.

4. The three-dimensional image stitching method for white light interferometer according to claim 1, characterized in that: In step S3, each time the right image moves one column, the data of the current overlapping area of ​​the left image and the right image are normalized, and then the standard deviation is calculated by subtracting them, until the last column of data of the left image and the right image overlap and the normalization and standard deviation calculation are completed.

Images