A method, system, device and medium for calculating the defocus amount based on a line laser

Abstract

The application discloses a defocus amount calculation method, system, device and medium based on a line laser, and relates to the field of automatic focusing. The method comprises the following steps: acquiring a laser spot image; dividing the laser spot image into k sections along the transverse direction; calculating the defocus amount of each section of the image to obtain a defocus amount set; dividing the defocus amounts with a difference within a preset error range into the same sub-set; removing the sub-set with an element number less than a preset threshold E; judging whether the total number of the remaining sub-sets is lower than a; if not, selecting the first a sub-sets with the largest element number, calculating the average value of the defocus amounts in the first a sub-sets; if yes, reacquiring the laser spot image; sorting the average values according to the numerical value, and corresponding to the defocus amount of each height step surface according to the height value sequence of the height step surfaces. Thus, the average values of the a sub-sets are sorted according to the size, and the height step surfaces under a kind of height are one-to-one corresponding, so that the defocus amounts of different height step surfaces are accurately obtained.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of automatic focusing, and particularly relates to a defocus amount calculation method, system, device and medium based on a line laser. BACKGROUND

[0002] The line laser automatic focusing technology usually determines the size and direction of the defocus amount of the system by calculating the centroid position of the surface laser spot.

[0003] However, the aforementioned defocus amount calculation result is easily affected by the high-low structure of the step sample surface. At present, there is a segmented focusing algorithm for double-layer step samples, which can distinguish the defocus amount of the high-low two step surfaces of the step sample, but is not applicable to the defocus amount identification of any step surface with more than two height types. Among them, the step sample of the application refers to a step sample with a ladder-shaped surface, such as a step standard block and an integrated circuit wafer. The step surface is the upper surface of the ladder-shaped step sample, and there is no overlapping area between any two step surfaces of the step sample.

[0004] Therefore, there is an urgent need for a defocus amount calculation method to solve the above problems. SUMMARY

[0005] The application provides a defocus amount calculation method, system, device and medium based on a line laser, which realizes accurate focusing of any height step surface of a step sample with more than two height step surfaces by using a laser spot image.

[0006] To achieve the above object, the application provides the following technical solutions:

[0007] In the first aspect of the application, a defocus amount calculation method based on a line laser is provided for the automatic focusing process of a step sample, comprising:

[0008] Obtaining a laser spot image; wherein the line laser covers all height types of step surfaces in the step sample; and any step surface is perpendicular to the central axis of the microscopic objective lens;

[0009] Dividing the laser spot image into k segments along the horizontal direction;

[0010] wherein, ; L represents the horizontal width of the line laser during focusing; floor represents the down rounding operation; min represents the minimum value operation; h a represents the width of the a-th height step surface in the horizontal direction; a represents the total amount of height types of step surfaces; and the horizontal direction represents the direction parallel to the line laser;

[0011] Calculating the defocus amount of each segment image by the spot centroid to obtain a defocus amount set;

[0012] The defocus amounts whose difference values are within the preset error range are divided into the same sub-set;

[0013] The sub-set whose element number is less than the preset threshold E is removed;

[0014] Wherein, E = min b i , b i = (m i - 2) * q i And i ∈ (1, 2,..., a); ; q i Indicates the number of linear lasers completely covering the i-th height step surface; the lateral width of the same height step surface is consistent;

[0015] It is judged whether the total amount of the remaining sub-set is less than a; if not, the first a sub-sets with the largest element number are selected, and the average value of the defocus amount in the sub-set is calculated; if yes, the laser spot image is re-acquired;

[0016] The average values are sorted according to the numerical size, and the defocus amount of each height type step surface is set according to the focusing direction corresponding to the positive and negative sign of the defocus amount and the height order of the step surface.

[0017] Optionally, after the defocus amount of each image is calculated by the spot centroid and the defocus amount set is obtained, it further includes:

[0018] The defocus amount set is clustered and analyzed based on the preset cluster density to form a plurality of clusters; wherein, the total amount of the clusters is not less than a; the cluster density is determined based on the minimum height difference of each step surface;

[0019] According to the order from large to small of the cluster size, the average value of the defocus amount in the first a clusters is calculated;

[0020] The average values are sorted according to the numerical size, and the defocus amount of each height type step surface is set according to the focusing direction corresponding to the positive and negative sign of the defocus amount and the height order of the step surface.

[0021] Optionally, the clustering analysis of the defocus amount set based on the preset cluster density to form a plurality of clusters includes:

[0022] The defocus amount set is clustered and analyzed based on a hierarchical clustering algorithm to form a plurality of clusters; wherein, the preset cluster density is determined by the similarity between clusters of the hierarchical clustering algorithm.

[0023] Optionally, the clustering analysis of the defocus amount set based on the preset cluster density to form a plurality of clusters includes:

[0024] The defocus amount set is clustered and analyzed based on a DBSCAN clustering algorithm to form a plurality of clusters; wherein, the preset cluster density is determined by the pre-set neighborhood radius and the minimum sample number.

[0025] Optionally, the cluster density is related to the minimum height difference between each height step surface; the smaller the minimum height difference is, the higher the cluster density is.

[0026] Optionally, the average values are sorted according to the numerical values, and the defocus amounts of each height category step surface are set according to the focusing direction corresponding to the positive or negative sign of the defocus amount and the height sequence of the step surface, comprising:

[0027] The a average values are sorted according to the numerical values; wherein the defocus amount is positive or negative, used to indicate the up or down moving direction of the microscope objective during focusing, and the absolute value of the defocus amount is used to indicate the moving distance of the microscope objective;

[0028] If the defocus amount is positive, indicating that the microscope objective moves downward, then the a average values sorted from small to large correspond to the defocus amounts of the a height category step surfaces from high to low; if the defocus amount is negative, indicating that the microscope objective moves downward, then the a average values sorted from small to large correspond to the defocus amounts of the a height category step surfaces from low to high.

[0029] Optionally, after the laser spot image is evenly divided into k segments along the transverse direction, further comprising:

[0030] The centroid of each segment image along the longitudinal direction is calculated to obtain a centroid set;

[0031] The centroid set is subjected to cluster analysis based on a preset cluster density to form a plurality of clusters; wherein the total number of clusters is not less than a;

[0032] The average values of the centroids in the first a clusters are calculated according to the order from large to small cluster size;

[0033] The average values are sorted according to the numerical values, and the centroids of each height category step surface are set according to the focusing direction corresponding to the positive or negative sign of the defocus amount and the height sequence of the step surface;

[0034] The difference between the centroid of each height step surface and the standard centroid is calculated, the difference is converted into a defocus amount, and the defocus amount of each height step surface is obtained.

[0035] In the second aspect of the present application, a line laser-based defocus amount calculation system is provided for the automatic focusing process of a step sample, comprising:

[0036] An image acquisition module is configured to acquire a laser spot image; wherein the line laser covers all height category step surfaces in the step sample; any step surface is perpendicular to the central axis of the microscope objective;

[0037] An image segmentation module is configured to evenly divide the laser spot image into k segments along the transverse direction;

[0038] in, L represents the lateral width of the linear laser during focusing; floor represents the floor function; min represents the minimum value operation; h a This represents the width of the horizontal direction of the step surface of height type a; 'a' represents the total number of different height types of the step surface; 'horizontal' indicates the direction parallel to the line laser beam.

[0039] The defocus calculation module is used to calculate the defocus amount of each image segment based on the centroid of the light spot, and obtain a set of defocus amounts.

[0040] The defocus amount division module is used to divide the defocus amounts whose differences are within a preset error range into the same subset;

[0041] The subset removal module is used to remove subsets whose number of elements is less than a preset threshold E;

[0042] Where, E=min b i b i =(m i -2)×q i And i∈(1,2,...,a); ;q i This indicates the number of steps at the i-th height that are completely covered by the line laser; the horizontal width of steps at the same height is consistent.

[0043] The subset judgment module is used to determine whether the total number of remaining subsets is less than a; if not, the first a subsets with the largest number of elements are selected and the average defocus amount in the first a subsets is calculated; if yes, the laser spot image is reacquired.

[0044] The defocus amount module is used to sort the average value according to its magnitude, and set the defocus amount for each type of step surface based on the focusing direction and the height order of the step surface corresponding to the positive and negative signs of the defocus amount.

[0045] In a third aspect of this application, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0046] Memory, used to store computer programs;

[0047] A processor, when executing a program stored in memory, implements the defocus calculation method described in any of the first aspects.

[0048] In a fourth aspect of this application, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when executed by a processor, the computer program implements the defocus calculation method described in any of the first aspects.

[0049] The beneficial effects of the present application are as follows:

[0050] The present application provides a defocus amount calculation method based on linear laser automatic focusing, which is used in the automatic focusing process of step samples, comprising:

[0051] Obtaining a laser spot image; wherein the linear laser covers all height types of step surfaces in the step sample; any step surface is perpendicular to the central axis of the microscope objective;

[0052] Dividing the laser spot image into k segments along the horizontal direction;

[0053] Wherein, L represents the horizontal width of the linear laser when focusing; floor represents the down rounding operation; min represents the minimum value operation; h a represents the horizontal width of the step surface of the a height type; a represents the total amount of height types of step surfaces; horizontal represents the direction parallel to the linear laser;

[0054] Calculating the defocus amount of each segment image by spot centroid to obtain a defocus amount set; dividing the defocus amounts with a difference within a preset error range into the same sub-set; removing the sub-set with an element number less than a preset threshold E;

[0055] Wherein, E = min b i , b i = (m i -2) × q i and i ∈ (1, 2,..., a); q i represents the number of linear lasers completely covering the step surface of the i height type; the horizontal width of the step surface of the same height type is consistent;

[0056] Judging whether the total amount of the remaining sub-sets is lower than a; if not, selecting the first a sub-sets with the largest element number, calculating the average value of the defocus amounts in the sub-sets; if yes, re-obtaining the laser spot image; sorting the average values according to the numerical size, and corresponding setting the defocus amount of each height type of step surface based on the focusing direction corresponding to the sign of the defocus amount and the height order of the step surface.

[0057] Based on the above processing, in the process of dividing the laser spot image, by setting the minimum value k of the segment number, for any step surface, it is guaranteed that there are at least 2 segment images completely falling on the step surface, that is, for any height type of step surface, the number of segment spot images completely falling above is higher than the number of segment spot images falling at the junction of two height types of step surfaces.

[0058] Therefore, in the calculated set of defocus amounts, the number of accurate defocus amounts corresponding to any height step surface will exceed the number of error defocus amounts corresponding to the junction. At the same time, by the division manner of the sub-sets, the segmented defocus amounts belonging to the same height step surface are divided into the same sub-set.

[0059] In combination with the setting of the threshold value E, it can be known that the number of accurate defocus amounts corresponding to the same height step surface is higher than the threshold value E, and the sub-set with the number of elements lower than the threshold value E corresponds to the sub-set composed of error defocus amounts. Further, since the number of accurate defocus amounts corresponding to any height step surface will exceed the number of error defocus amounts corresponding to the junction of two step surfaces, the first a sub-sets with the largest number of elements in the remaining sub-sets are selected respectively to correspond to the step surfaces under a kind of height. And by using the judgment means whether the total amount of the remaining sub-sets is lower than a, the division result of the sub-sets can be checked to avoid the error of the division result of the set of accurate defocus amounts.

[0060] After the average values of the defocus amounts in each sub-set are sorted by size, they can be one-to-one corresponding to the step surfaces under a kind of height, accurately distinguishing and obtaining the defocus amounts of different height step surfaces in the step sample. In the subsequent focusing process based on the defocus amount corresponding to the step surface to be focused, the interference of other height step surfaces on the spot information of the step surface to be focused is avoided, and clear imaging of each height step surface on the surface of the step sample is realized. BRIEF DESCRIPTION OF DRAWINGS

[0061] The drawings described herein are used to provide further understanding of the present application, constitute a part of the present application, the illustrative embodiments of the present application and the description thereof are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0062] Figure 1 is a schematic diagram of laser spot under different height step surfaces provided by the present application;

[0063] Figure 2 is a flowchart of a defocus amount calculation method provided by the present application;

[0064] Figure 3 is a schematic diagram of an optical path structure of a line laser automatic focusing device provided by the present application;

[0065] Figure 4 is a schematic diagram of an optical path of a line laser automatic focusing device provided by the present application;

[0066] Figure 5 is a schematic diagram of laser spot region selection provided by the present application;

[0067] Figure 6 is a schematic diagram of segmentation of a laser spot image provided by the present application;

[0068] Figure 7 is a flowchart of another defocus amount calculation method provided by the present application;

[0069] Figure 8 is a flowchart of another defocus amount calculation method provided by the present application;

[0070] Figure 9 is a flowchart of another defocus amount calculation method provided by the present application;

[0071] Figure 10 is a structural diagram of a defocus amount calculation system provided by the present application;

[0072] Figure 11 is a structural diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application.

[0074] The depth of field of a high-power microscope objective is generally only a few microns. In order to accurately focus the microscope objective on the sample surface, the operator often needs to spend a lot of time manually adjusting the distance between the objective and the sample to achieve focusing. The automatic focusing technology based on linear laser increases a linear laser signal on the basis of a general microscope, and measures the defocus amount of the microscope objective by using the change in the spot morphology or centroid position of the reflected linear laser from the surface of the measured object, as shown in Figure 1 which has a larger focusing range and faster focusing speed, and is widely used in industrial detection. Wherein, b represents the object distance of the microscope objective, f represents the focal length of the microscope objective, and l represents the transverse width of the long axis of the semi-elliptical laser spot.

[0075] For example, in the linear laser microscope automatic focusing scheme provided in Chinese patents CN118584643A or CN114994896A, the defocus amount is obtained by calculating the offset between the real-time centroid of the laser spot and the reference centroid, thereby achieving the purpose of automatic focusing of the objective. The defocus amount calculation scheme provided by the present application is applicable to the automatic focusing technology based on linear laser. The automatic focusing device and the generation of laser spot image, the adjustment of the objective based on the defocus amount, and other processing processes in the scheme can be referred to as shown in patent CN114994896A.

[0076] In some detection scenarios, due to the presence of different height step surfaces (for example, in the field of wafer detection, there are different height components or scribe lanes on the wafer surface), the laser spot image generated by the existing line laser automatic focusing system contains signals reflected by different height step surfaces, and the accuracy of the defocus amount calculated directly is low.

[0077] A microscope automatic focusing method and device for micro-nano step samples have been proposed in Chinese patent CN114994896A. By segmenting the step sample surface spot, the centroid positions of each segment are calculated, and the minimum value is selected as the top surface centroid position and the maximum value is selected as the bottom surface centroid position, thereby realizing the differentiation of the upper and lower surfaces of the step sample and solving the problem of focusing failure of samples with concave and convex structures exceeding the imaging objective lens depth of field.

[0078] However, the aforementioned prior art has the problem of being unable to distinguish the defocus amount of more than two height step surfaces when focusing on step structures with multiple layers of concave and convex surfaces, and there is still a risk of defocusing.

[0079] To solve the above problems and accurately distinguish the defocus amount of more than two height step surfaces, the present application provides a line laser-based defocus amount calculation method for the automatic focusing process of step samples, as shown in Figure 2 The method comprises the following steps:

[0080] S1, acquiring a laser spot image. Wherein, the line laser covers all height types of step surfaces in the step sample; any step surface is perpendicular to the central axis of the microscope objective.

[0081] S2, dividing the laser spot image into k segments along the horizontal direction.

[0082] Wherein, ; L represents the horizontal width of the line laser when focusing, i.e. the horizontal width of the line laser when converging into a line in the laser spot image; floor represents the down rounding operation; min represents the minimum value operation; h1 represents the width of the first height step surface along the horizontal direction; h2 represents the width of the second height step surface along the horizontal direction; h a n represents the width of the nth height step surface along the horizontal direction; a represents the total amount of height types of step surfaces. The aforementioned width can correspond to the number of pixel points along the horizontal direction in the laser spot image, or can correspond to the actual distance of the line laser or step surface along the horizontal direction.

[0083] S3, calculating the defocus amount of each segment image by spot centroid to obtain a defocus amount set.

[0084] S4, in the defocus amount set, the defocus amounts with a difference within a preset error range are divided into the same sub-set.

[0085] S5, removing a subset whose element number is less than a preset threshold E;

[0086] wherein, E = min b i , b i = (m i - 2) x q i and i ∈ (1, 2,..., a); ; q i represents the number of linear lasers completely covering the i-th height step surface; the lateral width of the same height step surface is consistent.

[0087] S6, judging whether the total number of the remaining subsets is less than a; if not, selecting the first a subsets with the largest number of elements, and calculating the average value of the defocus amount in the first a subsets; if yes, reacquiring the laser spot image.

[0088] S7, sorting the average values according to the numerical value, and corresponding setting the defocus amount of each height step surface based on the focusing direction corresponding to the sign of the defocus amount and the height order of the step surface.

[0089] Based on the above processing, in the process of dividing the laser spot image, by setting the minimum value k of the segmentation number, it is guaranteed that at least two segmented images completely fall on the step surface for any step surface, that is, for any height step surface, the number of segmented spots completely falling on the upper step surface is higher than the number of segmented spots falling on the junction of two height step surfaces.

[0090] Therefore, in the calculated defocus amount set, the number of accurate defocus amounts corresponding to any height step surface will be more than the number of error defocus amounts corresponding to the junction. At the same time, by the division mode of the subset, the segmented defocus amounts belonging to the same height step surface are divided into the same subset.

[0091] Moreover, in combination with the setting of the threshold E, it can be known that the number of accurate defocus amounts corresponding to the same height step surface is higher than the threshold E, and the subset whose element number is lower than the threshold E corresponds to the subset composed of error defocus amounts. Further, since the number of accurate defocus amounts corresponding to any height step surface will be more than the number of error defocus amounts corresponding to the junction of two step surfaces, the first a subsets with the largest number of elements selected from the remaining subsets correspond to the step surfaces under a kind of height respectively.

[0092] After sorting the average values of the defocus amounts in each subset by size, it can be one-to-one corresponding to the step surfaces under a kind of height, accurately distinguishing and obtaining the defocus amounts of different height step surfaces in the step sample. In the subsequent focusing process based on the defocus amount corresponding to the step surface to be focused, the interference of other height step surfaces on the spot information of the step surface to be focused is avoided, and clear imaging of each height step surface on the surface of the step sample is realized.

[0093] In some embodiments, step S1 can include the following contents:

[0094] S101, adjust the line laser-based autofocus device so that the laser spot emitted by the autofocus device covers all height types of step surfaces. Wherein, the autofocus device represents a device based on laser spot morphology to complete autofocus.

[0095] S102, collect the laser spot image based on the autofocus device. Or, for the original image collected by the autofocus device, select the image of the central region in the original image as the laser spot image. Wherein, the image of the central region in the original image covers all a height types of step surfaces.

[0096] In this application, a line laser-based autofocus device is used, and the spot of the line laser projected onto the step sample completely covers all types of height step surfaces. In special scenarios, if a single line laser cannot cover all height step surfaces, the spot images generated by the focusing sensor at two or more adjacent exposure times can be spliced, and the spliced spot image is used as the laser spot image obtained in step S1 to ensure that the line laser can cover the height step surface under each type.

[0097] For the optical path of the autofocus device in step S101, as shown in Figure 3 The structure is consistent with the autofocus device disclosed in Chinese patent CN114994896A, including a laser, a cylindrical lens, a baffle, a first beam splitter, a second beam splitter, a microscope objective, a motor, a focusing lens, an image sensor (or COMS camera), and a measured object (i.e., the step sample of this application). The circular parallel light emitted by the line laser enters the cylindrical lens, which is modulated by the cylindrical lens to become an asymmetric light beam that is collimated in the direction parallel to the paper and diverges in the direction perpendicular to the paper; After the asymmetric light beam passes through the baffle, it will lose half of the energy and only propagate on one side of the optical axis, and after being reflected by the first beam splitter, it reaches the second beam splitter, and after being reflected by the second beam splitter, it enters one side of the microscope objective and is projected on the surface of the measured object by the other side of the microscope objective. Due to the reflection of the surface of the measured object, the laser is collected again by the other side of the microscope objective, and after being reflected by the second beam splitter and refracted by the first beam splitter, it reaches the focusing lens, and after being converged by the focusing lens, it is received by the CMOS camera. According to the state of the laser spot received by the CMOS camera, the defocusing amount of the microscope objective is calculated, and the defocusing amount is converted into a control signal of the motor to drive the motor to move the microscope objective, thereby realizing automatic focusing.

[0098] It is noted that, in the automatic focusing scheme provided in the present application, for the laser spot image, the direction along the line segment of the laser spot, i.e. the long axis direction of the semi-elliptical spot, is the transverse direction, which corresponds to the parallel direction of the line laser when focusing, also known as the horizontal direction. Correspondingly, the direction perpendicular to the transverse direction is the longitudinal direction, which corresponds to the short axis direction of the semi-elliptical spot image.

[0099] For step S102, in the automatic focusing scheme provided in the present application, the original spot image collected by the aforementioned automatic focusing device can be directly used as the laser spot image in the subsequent steps. Wherein, the automatic focusing light path for the step sample is as shown in Figure 4 .

[0100] After the original spot image is collected by the automatic focusing device, a part of the image in the central region of the original spot image can be selected as the laser spot image in the subsequent steps, so as to avoid the adverse effects of the edge segmentation of the laser spot on the calculation of the defocus amount or the centroid position. As shown in Figure 5 , the image in the 3 / 4 region of the center of the upper original spot image is selected as the laser spot image below. Similarly, the selected part of the laser spot image also needs to completely cover the step surface of a height.

[0101] For step S2, the laser spot image is equally divided into k segments by using the longitudinal parallel lines, as shown in the laser spot image segmentation schematic diagram Figure 6 . Wherein, n represents the total amount of transverse pixel columns of the laser spot image, and m represents the total amount of pixel rows of the laser spot image along the longitudinal direction.

[0102] Based on the aforementioned calculation method of k, it can be ensured that on any height step surface, there are at least 2 segmented images completely falling on the step surface, and the number thereof is higher than the number of segmented images falling on the joint of two height step surfaces, so that not only the accurate defocus amount of the step surface can be obtained, but also the total number of accurate defocus amounts is higher than the number of “wrong defocus amounts” at the joint of two step surfaces, so as to ensure the accuracy of the subsequent subset division or clustering analysis result.

[0103] For step S3, the centroid method can be used to calculate the defocus amount of each segmented image. Wherein, the centroid calculation formula is as shown in the following formula:

[0104]

[0105] Wherein, COG yt represents the centroid of the segmented image in the y direction (longitudinal direction); f t (i, j) is the gray value of the i-th row and j-th column pixel in the t-th segmented laser image; t = 1, 2, …, k; i = 1, 2, …, m; j = 1, 2, …, n.

[0106] Calculate the current laser centroid (COG) for each image segment. now and focus reference centroid COG ref The difference is calculated and converted into the defocusing amount Δz of the microscope objective. The defocusing amount conversion formula is:

[0107]

[0108] Where s is the conversion coefficient for converting pixel coordinates to position coordinates, also commonly referred to as the sensitivity coefficient. The aforementioned defocus calculation process can be found in Chinese patents CN114994896A and CN118595592A.

[0109] Regarding the defocus amount of segmented images, if two segmented images are both formed by reflection from a step surface of the same height, then the defocus amounts corresponding to the two segments are basically the same. This application considers that in actual operation, the detected defocus amount under the same step surface still has some errors. Therefore, in step S4, defocus amounts with differences within a preset error range are selected and grouped into the same subset as the corresponding defocus amounts under the same step surface.

[0110] The error range can be understood as the allowable detection error of defocusing amount under the same height step surface during the actual detection process. The specific value of the error range can be obtained by pre-calibrating the line laser-based autofocus device. For example, before acquiring the laser spot image, the autofocus device is used to acquire laser spot images under the same height step surface of the stepped sample. Then, the laser spot image is segmented, and the defocusing amount corresponding to each segment is calculated. Finally, the range of differences between the aforementioned multiple defocusing amounts is used as the error range of this application.

[0111] For step S4, based on the calculation order of the defocus amount in each segmented image, the differences between adjacent defocus amounts are compared sequentially. If the difference is within a preset error range, the subsequent defocus amount is assigned to the same subset as the previous defocus amount. If the difference is not within the preset error range, the defocus amount is continuously compared with the preceding defocus amounts. When the difference after comparison is within the preset error range, it is assigned to a subset containing the compared defocus amount. If none of the differences are within the preset error range, the defocus amount is assigned to a new subset.

[0112] For step S5, the element number represents the number of defocus amounts in a single subset. The preset threshold E corresponds to the minimum number of complete segmented images formed by the reflection of the same height step surface in the laser spot image. Specifically, (L / K) represents the width of the segmented image along the horizontal direction of the laser spot image; according to the calculation formula of mi, it represents the number of segmented images that can be split by a single height step surface of the i-th type. Considering that the two ends of the height step surface of the i-th type and the adjacent height step surfaces together form a segmented image, the value corresponding to mi-2 can be determined as the segmented image formed by the reflection of a single height step surface of the i-th type.

[0113] Further, since the laser spot image can contain multiple height step surfaces of the i-th type, the number q of complete height step surfaces of the i-th type contained in the laser spot image is set i The total number of segmented images formed by the reflection of the height step surface of the i-th type can be calculated. Wherein, q i And the specific value of h i can be obtained by pre-scanning the step sample.

[0114] Combined with the division method of the defocus amount in the same subset in step S4, it can be known that for K defocus amounts, the defocus amounts under the same height step surface belong to the same subset, and combined with the setting condition that the preset threshold E is the minimum value in the set bi, i∈(1, 2,..., a). It can be known that the total amount of accurate defocus amounts corresponding to each height step surface is certainly not lower than the threshold E, and the element number in the corresponding subset is not less than the threshold E.

[0115] Therefore, the subset composed of error defocus amounts can be removed by step S5. The error defocus amount of the present application represents the defocus amount corresponding to the segmented image formed by the reflection of the same height step surface, i.e. the defocus amount that cannot accurately represent any height step surface.

[0116] Based on the screening method of the subset in step S5, the subset with an element number lower than the threshold E is removed, and combined with the setting of K value in step S2, the number of accurate defocus amounts of any height step surface is higher than the number of error defocus amounts at both ends. In actual work, the total number of the remaining subsets is usually a, which respectively corresponds to the accurate defocus amounts of each height step surface.

[0117] Considering that the threshold E is usually small, in some embodiments, step S6 includes the following contents:

[0118] S601, judge whether the total number of the remaining subsets is lower than a, if not, go to step S602; if yes, go to step S1.

[0119] S602, select the first a subsets with the largest element number, and calculate the average value of the defocus amounts in the first a subsets.

[0120] When the number of the remaining subsets is not less than a, the average values of the first a subsets with the largest number of elements are selected, corresponding to a height step surface. When the number of the remaining subsets is less than a, it indicates that the current calculation result is incorrect, and the laser spot image needs to be obtained again in step S1 to re-execute the defocus amount calculation scheme.

[0121] In some embodiments, step S7 includes the following contents:

[0122] S701. Sort the a average values according to the numerical size. Wherein, the defocus amount is divided into positive and negative values, which are used to represent the up and down movement directions of the microscope objective during the focusing process, and the absolute value of the defocus amount is used to represent the movement distance of the microscope objective.

[0123] For step S701, the average values can be sorted from small to large or from large to small according to the numerical size. It should be noted that during the sorting process, the defocus amount containing positive and negative values is sorted according to the size, rather than relying on the absolute value for sorting.

[0124] S702. If the defocus amount is a positive value indicating that the microscope objective moves downward, then the a average values sorted from small to large correspond to the defocus amounts of a height step surface from high to low; if the defocus amount is a negative value indicating that the microscope objective moves downward, then the a average values sorted from small to large correspond to the defocus amounts of a height step surface from low to high.

[0125] For step S702, if the defocus amount is a positive value indicating that the microscope objective moves downward, then the larger the defocus amount value of a step surface, the farther the distance between the step surface and the microscope objective along the downward direction, that is, the lower the step surface. Therefore, the a average values sorted from small to large correspond to the defocus amounts of a height step surface from high to low.

[0126] Similarly, if the defocus amount is a negative value indicating that the microscope objective moves downward, then the larger the defocus amount value of a step surface, the farther the distance between the step surface and the microscope objective along the upward direction, that is, the higher the step surface. Therefore, the a average values sorted from small to large correspond to the defocus amounts of a height step surface from low to high.

[0127] In some embodiments, after step S7, the obtained defocus amount of the step surface to be focused can be used in the automatic focusing process, that is, according to the defocus amount corresponding to the step surface to be focused, the distance between the microscope objective and the step surface to be focused is adjusted, so that the step surface to be focused is within the depth of field of the microscope objective.

[0128] In actual work, the defocus amount is usually converted into a motor driving signal, and the motor in the autofocus device is driven to move the microscopic objective lens, so as to adjust the distance between the microscopic objective lens and the step surface to be focused, thereby achieving the purpose that the step surface to be focused is within the depth of field of the microscopic objective lens. The microscopic objective focusing process in this part can be referred to in Chinese Patent CN114994896A.

[0129] In view of the foregoing defocus amount autofocusing scheme, the widths of the step surfaces of the same height need to be consistent. In order to further expand the application range of the defocus amount calculation scheme, steps S4-S6 can be replaced by steps S8-S9 as shown in the following table: Figure 7

[0130] S8, cluster analysis is performed on the defocus amount set based on a preset cluster density to form a plurality of clusters. The total number of clusters is not less than a. The cluster density represents the closeness of the sample points (i.e., the defocus amount) in the cluster. The preset cluster density is determined based on the minimum height difference of each height step surface. If the numerical value of the minimum height difference is larger, the cluster density can be appropriately reduced. If the numerical value of the minimum height difference is smaller, i.e., the heights of the different height step surfaces are closer, the cluster density needs to be correspondingly increased.

[0131] S9, the average values of the defocus amounts in the first a clusters are calculated in the order from the largest to the smallest cluster size.

[0132] S7, the average values are sorted according to the numerical values, and the defocus amounts of each height step surface are correspondingly set based on the focusing direction corresponding to the positive and negative signs of the defocus amounts and the height order of the step surfaces.

[0133] Based on the above processing, in the laser spot image equal division process, by setting the minimum number of segments K, for any step surface, it is ensured that at least two complete segmented images fall on its surface, i.e., for any height step surface, the number of complete segmented spots falling on the height step surface is higher than the number of segmented spots falling on the junction of two height step surfaces.

[0134] Therefore, in the calculated defocus amount set, the number of accurate defocus amounts corresponding to any height step surface exceeds the number of error defocus amounts corresponding to the junction of two step surfaces, so that after the subsequent cluster analysis, the a clusters correspond to the step surfaces of the a heights. After sorting the average values of the defocus amounts in each cluster by size, the step surfaces of the a heights can be one-to-one corresponded, and the defocus amounts of the different height step surfaces in the step sample can be accurately distinguished. In the subsequent focusing process based on the defocus amount corresponding to the step surface to be focused, the interference of the other height step surfaces on the spot information of the step surface to be focused is avoided, and clear imaging of each height step surface on the surface of the step sample is achieved. ​

[0135] In some embodiments, step S8 comprises the following:

[0136] S801, performing clustering analysis on the set of defocus values based on a hierarchical clustering algorithm to form a plurality of clusters. The preset cluster density is determined by the inter-cluster similarity of the hierarchical clustering algorithm.

[0137] The hierarchical clustering algorithm analyzes data at different levels based on the similarity between clusters to form a tree-shaped clustering structure, and generally includes two division strategies: an agglomerative strategy from bottom to top and a divisive strategy from top to bottom. For the clustering analysis of the set of defocus values, any of the aforementioned division strategies can be used.

[0138] In the present application, the inter-cluster similarity can be determined by the minimum distance, maximum distance, average distance, center distance, and minimum variance method. In the present application, the minimum distance or minimum variance method is generally used. It can be understood that the inter-cluster similarity is used to control the cluster density of step S8, and its size is still determined according to the minimum height difference of different height step surfaces. The smaller the minimum height difference, the smaller the inter-cluster similarity. It should be noted that the total number of clusters finally generated in step S801 cannot be less than a. If it is less than a, the inter-cluster similarity is reduced and the clustering analysis is performed again to obtain more clusters.

[0139] In some embodiments, step S8 can further comprise the following:

[0140] S802, performing clustering analysis on the set of defocus values based on a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm to form a plurality of clusters. The preset cluster density is determined by the preset neighborhood radius epsilon and the minimum sample number minPts.

[0141] Specifically, the neighborhood radius is related to the minimum height difference of different height step surfaces, and the smaller the minimum height difference, the smaller the inter-cluster similarity. The minimum sample number needs to be limited to not more than the aforementioned threshold E, and to reduce the total number of clusters in the clustering analysis, the minimum sample number can be directly set to the threshold E in actual work. In addition, the specific values of the neighborhood radius and the minimum sample number can be determined by pre-calibration of the step sample. Alternatively, when the total number of clusters finally generated in step S802 is less than a, the neighborhood radius and the minimum sample number need to be adjusted and the clustering analysis is performed again.

[0142] In some embodiments, after obtaining the defocus value in step S3 and before step S8, the defocus value calculation scheme provided by the present application further comprises the following steps:

[0143] S10, performing outlier filtering, missing value filling and data standardization on the elements of the defocus amount set.

[0144] Specifically, the outlier represents a defocus amount that is significantly beyond the focus range. For example, if the calculated defocus amount is 100 um, which is significantly beyond the range of 20-40 um of the object distance b between the microscopic objective lens and the stepped sample surface, the defocus amount is an outlier.

[0145] Step S10 corresponds to the data preprocessing step in the cluster analysis, including outlier filtering, missing value processing and data standardization, etc., to improve the accuracy and stability of the clustering results. Among them, interpolation, mean filling, median and other methods can be used to process missing values. Z-score standardization, min-max standardization and other methods are used to scale the data to the same scale to complete the data standardization processing. In addition, outliers can be detected and removed by box plot, IQR and other methods.

[0146] Considering that in the autofocus scheme provided in the present application, the y-direction centroid of each segmented image is first calculated, and then the defocus amount is calculated by comparing with the standard centroid. Therefore, in the autofocus method provided in the present application, the centroids of the segmented images can also be clustered in advance, and then the defocus amount corresponding to each height step surface is calculated according to the clustering analysis result of the centroids. As shown in FIG. 2, after step S2, the defocus amount calculation method provided in the present application can also include the following steps S11-S12, which are used to replace steps S3-S4. Figure 8

[0147] S11, calculating the centroid of each segmented image along the longitudinal direction to obtain a centroid set;

[0148] S12, in the centroid set, the centroids with a difference value within a preset difference value range are divided into the same subset. The centroid with a difference value within a preset difference value range corresponds to the centroid of the segmented image formed by the reflection of the step surface of the same height.

[0149] Correspondingly, step S7 also needs to be replaced by the following steps S13-S14.

[0150] S13, sorting the average values according to the numerical size, and setting the centroid of each height category step surface according to the focusing direction corresponding to the positive and negative signs of the defocus amount and the height sequence of the step surface.

[0151] S14, calculating the difference between the centroid of each height step surface and the standard centroid, converting the difference into a defocus amount, and obtaining the defocus amount of each height step surface.

[0152] ​Simultaneously, referring to steps S8 to S9, cluster analysis of the centroid can be used to implement the defocusing calculation scheme of this application. Specifically, as... Figure 9 As shown, after step S11, the defocusing amount calculation method provided in this application further includes the following steps S15-S16, which replace steps S12, S5 and S6, as follows:

[0153] S15. Perform cluster analysis on the centroid set based on the preset cluster density to form multiple clusters. The total number of clusters is not less than a.

[0154] S16. Calculate the average value of the centroids within the first a clusters, in order of cluster size from largest to smallest.

[0155] For the subset partitioning process in step S12, you can directly refer to the content of step S4. The centroid calculation formula in step S11 and the defocus calculation formula in step S14 can be directly referred to the content of step S3 mentioned above. Similarly, for the cluster analysis process in step S15, you can refer to the content of step S8. No specific explanation will be given here.

[0156] To facilitate understanding of the defocus calculation scheme provided in this application, the following explanation uses the centroid in the y-direction as an example to illustrate the defocus calculation and autofocus process, including the following:

[0157] Step 1: Open as follows Figure 3 The laser and image sensor (i.e., CMOS camera) within the autofocus device shown project a line laser beam that propagates only on one side of the optical axis onto the surface of the stepped sample, such as... Figure 5 As shown, the line laser covers all types of stepped surfaces.

[0158] Step 2: Adjust the exposure time of the image sensor to obtain the laser spot image returned from the surface of the stepped sample.

[0159] Step 3, as follows Figure 6 As shown, a laser spot image is selected and divided into k segments. The laser image in each segment is calculated. y Center of mass (COG) in direction yt .

[0160] Step 4: Identify the step surface that needs to be in focus, which belongs to the top, bottom, or middle layer of all height step surfaces (i.e., the top, bottom, and middle surfaces).

[0161] Step 5: Since a smaller centroid of the laser spot image indicates a higher corresponding step surface, to achieve focusing on the top layer of the stepped sample surface, the minimum value among the centroids of segment k is selected as the current laser centroid COG. nowIf the bottom layer of the stepped sample surface is to be focused, the maximum value in the k centroid is selected as the current laser centroid COG now That is,

[0162]

[0163] If the middle layer of the stepped sample surface is to be focused, the k centroid data needs to be sub-set filtered, and the sub-set of the first a elements sorted from large to small is selected. The mean value of each group of data is taken, and the mean values are sorted in size, and the value corresponding to the target layer is selected as the current laser centroid COG now .

[0164]

[0165] Alternatively, if the middle layer of the stepped sample surface is to be focused in step five, clustering analysis is also performed, the same height surface image centroid data is filtered into a group, the data set of the target group number is obtained, the mean value of each group of data is taken, and the mean values are sorted in size, and the value corresponding to the target layer is selected as the current laser centroid COG now .

[0166] Step six, the current laser spot image centroid COG now is compared with the focusing reference centroid COG ref When COG now falls within the reference range, that is,

[0167] , it represents that the focusing is completed, and if COG now is not within the reference range, step seven is performed. Wherein, s is the focusing sensitivity of the objective lens, and its unit is pixel / depth of field, and s and COG ref are calibrated by the pre-focusing calibration method.

[0168] Step seven, the difference between the current laser centroid COG now and the focusing reference centroid COG ref is converted into the defocus amount Δz of the objective lens, and the conversion formula is:

[0169]

[0170] Step eight, the defocus amount is converted into a motor driving signal, the motor drives the microscopic objective lens to move to adjust the distance between the microscopic objective lens and the stepped surface to be focused, and steps two to eight are repeated until the focusing is completed.

[0171] Based on the same inventive concept, the application also provides a defocus amount calculation system based on linear laser automatic focusing, which is used in the automatic focusing process of a stepped sample. As shown in Figure 10 , the system comprises:

[0172] Image acquisition module 1001 is used to acquire laser spot images;

[0173] Image segmentation module 1002 is used to divide the laser spot image horizontally into k segments;

[0174] in, L represents the lateral width of the linear laser during focusing; floor represents the floor function; min represents the minimum value operation; h a This represents the width of the horizontal direction of the step surface of height type a; 'a' represents the total number of different height types of the step surface; 'horizontal' indicates the direction parallel to the line laser beam.

[0175] The defocus calculation module 1003 is used to calculate the defocus amount of each segment of the image through the centroid of the light spot, and obtain the defocus amount set.

[0176] The defocus amount division module 1004 is used to divide the defocus amounts whose differences are within a preset error range into the same subset;

[0177] Subset removal module 1005 is used to remove subsets whose number of elements is less than a preset threshold E;

[0178] Where, E=min b i b i =(m i -2)×q i And i∈(1,2,...,a); ;q i This indicates the number of steps at the i-th height that are completely covered by the line laser; the horizontal width of steps at the same height is consistent.

[0179] The subset judgment module 1006 is used to determine whether the total number of remaining subsets is less than a; if not, the first a subsets with the largest number of elements are selected and the average defocus amount in the first a subsets is calculated; if yes, the laser spot image is reacquired.

[0180] The defocus amount corresponding module 1007 is used to sort the average value according to the numerical value, and set the defocus amount for each type of step surface according to the focusing direction and the height order of the step surface based on the positive and negative signs of the defocus amount.

[0181] This application also provides an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104.

[0182] The memory 1103 is configured to store a computer program.

[0183] The processor 1101 is configured to execute the program stored in the memory 1103, and implement any of the above defocus amount calculation methods.

[0184] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0185] The communication interface is configured to communicate between the above electronic device and other devices.

[0186] The memory can include a Random Access Memory (RAM), and can also include a Non-Volatile Memory (NVM), for example, at least one disk memory. Optionally, the memory can also be at least one storage device located away from the above processor.

[0187] The above processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0188] In another embodiment provided in the present application, a computer readable storage medium is also provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement any of the above defocus amount calculation method steps.

[0189] In another embodiment provided in the present application, a computer program product containing instructions is also provided, and when the computer program product is executed on a computer, the computer is caused to execute any of the defocus amount calculation method steps in the above embodiments.

[0190] The above examples are only used to illustrate the technical solutions of the present application, but not to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those ordinarily skilled in the art should understand: the technical solutions recorded in the foregoing examples can still be modified, or some technical features can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for calculating a defocus amount based on a line laser, for an autofocus process of a step sample, characterized in that, The method comprises the following steps: acquiring a laser spot image; wherein the line laser covers all height types of step surfaces in the step sample; and any step surface is perpendicular to the central axis of the microscope objective; dividing the laser spot image into k segments in the lateral direction; wherein, ; L represents a lateral width of the line laser at the time of focusing; floor represents a down-rounding operation; min represents a minimum operation; h a represents a width in a lateral direction of the a-th height step face; a represents a total amount of height types of the step faces; lateral represents a direction parallel to the line laser; calculating the defocus amount of each segment image by the spot centroid to obtain a defocus amount set; dividing the defocus amounts with a difference within a preset error range into the same sub-set; removing the sub-set with an element number less than a preset threshold E; wherein E = min b i , b i = (m i - 2) x q i and i ∈ (1, 2,..., a); ; q i represents the number of linear lasers completely covering the i-th height step surface; the lateral width of the same height step surface is consistent; judging whether the total number of the remaining sub-sets is less than a; if not, selecting the first a sub-sets with the largest element number, and calculating the average value of the defocus amounts in the sub-sets; if yes, re-acquiring the laser spot image; sorting the average values according to the numerical size, and setting the defocus amount of each height type step surface according to the defocus sign corresponding to the focusing direction and the height sequence of the step surface.

2. The method of calculating the amount of defocus according to claim 1, wherein, After calculating the defocus amount of each segment image by the spot centroid to obtain a defocus amount set, the method further comprises the following steps: performing clustering analysis on the defocus amount set based on a preset cluster density to form a plurality of clusters; wherein the total number of the clusters is not less than a; and the cluster density is determined based on the minimum height difference of each step surface; calculating the average value of the defocus amounts in the first a clusters according to the order from large to small cluster size; sorting the average values according to the numerical size, and setting the defocus amount of each height type step surface according to the defocus sign corresponding to the focusing direction and the height sequence of the step surface.

3. The method of calculating the amount of defocus according to claim 2, wherein, The method of performing clustering analysis on the defocus amount set based on a preset cluster density to form a plurality of clusters comprises the following steps: performing clustering analysis on the defocus amount set based on a hierarchical clustering algorithm to form a plurality of clusters; wherein the preset cluster density is determined by the inter-cluster similarity of the hierarchical clustering algorithm.

4. The method of calculating the amount of defocus according to claim 2, wherein, The method of performing clustering analysis on the defocus amount set based on a preset cluster density to form a plurality of clusters comprises the following steps: performing clustering analysis on the defocus amount set based on a DBSCAN clustering algorithm to form a plurality of clusters; wherein the preset cluster density is determined by the preset neighborhood radius and the minimum sample number.

5. The method of calculating the amount of defocus according to claim 2, wherein, The cluster density is related to the minimum height difference between the height step surfaces; the smaller the minimum height difference is, the higher the cluster density is.

6. The method of calculating the amount of defocus according to any one of claims 1 or 2, wherein, The method of sorting the average values according to the numerical size, and setting the defocus amount of each height type step surface according to the defocus sign corresponding to the focusing direction and the height sequence of the step surface comprises the following steps: sorting the a average values according to the numerical size; wherein the defocus amount is divided into positive and negative values, which are used to represent the up and down movement directions of the microscope objective during focusing, and the absolute value of the defocus amount is used to represent the movement distance of the microscope objective; if the defocus amount is positive, indicating that the microscope objective moves downward, then the a average values sorted from small to large correspond to the defocus amounts of the a height types of step surfaces from high to low; if the defocus amount is negative, indicating that the microscope objective moves upward, then the a average values sorted from small to large correspond to the defocus amounts of the a height types of step surfaces from low to high.

7. The method of calculating the amount of defocus according to claim 1, wherein, After dividing the laser spot image into k segments in the lateral direction, the method further comprises the following steps: calculating the centroid of each segment image in the longitudinal direction to obtain a centroid set; performing clustering analysis on the centroid set based on a preset cluster density to form a plurality of clusters; wherein the total number of the clusters is not less than a; According to the order of the cluster size from large to small, calculate the average of the first a cluster centroids; According to the numerical value size, sort the average, and based on the focusing direction corresponding to the sign of the defocus amount and the height order of the step surface, set the centroid of each height step surface; Calculate the difference between the centroid of each height step surface and the standard centroid, convert the difference into the defocus amount, and obtain the defocus amount of each height step surface.

8. A line laser-based defocus amount calculation system for an autofocus process of a step sample, characterized by, The method comprises the following steps: An image acquisition module is configured to acquire a laser spot image; wherein the line laser covers all height step surfaces in the step sample; and any step surface is perpendicular to the central axis of the microscope objective; An image segmentation module is configured to divide the laser spot image into k segments along the horizontal direction; wherein, ; L represents a lateral width of the line laser at the time of focusing; floor represents a down-rounding operation; min represents a minimum operation; h a represents a width in a lateral direction of the a-th height step face; a represents a total amount of height types of the step faces; lateral represents a direction parallel to the line laser; A defocus amount calculation module is configured to calculate the defocus amount of each segment image by the spot centroid, and obtain a defocus amount set; A defocus amount division module is configured to divide the defocus amount within the preset error range into the same sub-set; A sub-set removal module is configured to remove the sub-set with the number of elements less than the preset threshold E; wherein E = min b i , b i = (m i - 2) x q i and i ∈ (1, 2,..., a); ; q i represents the number of linear lasers completely covering the i-th height step surface; the lateral width of the same height step surface is consistent. A sub-set judgment module is configured to judge whether the total number of the remaining sub-sets is less than a; if not, select the first a sub-sets with the largest number of elements, calculate the average of the defocus amount in the first a sub-sets; if yes, re-acquire the laser spot image; A defocus amount corresponding module is configured to sort the average according to the numerical value size, and based on the focusing direction corresponding to the sign of the defocus amount and the height order of the step surface, set the defocus amount of each height step surface.

9. An electronic device, comprising: The device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory is configured to store a computer program; The processor is configured to execute the program stored on the memory, and realize the defocus amount calculation method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and the computer program is executed by the processor to realize the defocus amount calculation method in any one of claims 1-7.

Images