A multi-point focus fitting micro-imaging plane leveling method, system and device

Abstract

The application discloses a multi-point focusing fitting microscopic imaging plane leveling method, system and device, which is used for high-precision automatic leveling of an imaging plane. The method comprises the following steps: establishing a two-dimensional sampling grid based on a measured plane, and determining grid node coordinates; establishing a space coordinate system, and calling calibration data to obtain a calibration image plane; continuously collecting an image sequence covering the two-dimensional sampling grid; dividing the image sequence according to the grid node coordinates to obtain a grid image sequence; calculating the definition of the grid image sequence by using a no-reference definition evaluation method to obtain a definition sequence; establishing an index relationship between the grid node coordinates and the maximum definition in the corresponding definition sequence to obtain a definition point set; performing plane fitting based on the definition point set to obtain a global focal plane model; calculating parallel parameters of the global focal plane relative to the calibration image plane according to the global focal plane model; and adjusting the posture of a placement platform of the measured plane through the parallel parameters.

Description

Technical Field

[0001] This embodiment relates to the field of image processing, and in particular to a method, system and apparatus for multi-point focusing fitting of microscopic imaging plane leveling. Background Technology

[0002] With the development of high-precision manufacturing of display panels, semiconductor wafers, and microstructure devices, microscopic imaging inspection systems have gradually become a key component.

[0003] In existing technologies, the most common focusing and leveling methods are single-point autofocus, which involves scanning the sample center point or a local area in the Z direction and using a sharpness function to determine the optimal focus point to achieve autofocus. However, this only ensures that the local area is sharp and cannot cover the entire field of view. Furthermore, when using single-point autofocus, if the sample has flatness errors or is tilted, the edge areas often become out of focus, leading to a decrease in detection accuracy. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides a method, system, and apparatus for multi-point focusing and fitting of microscopic imaging plane leveling, used for high-precision automatic leveling of the imaging plane.

[0005] The technical solution provided in this application is described below:

[0006] The first aspect of this application provides a multi-point focusing fitting microscopic imaging plane leveling method, including:

[0007] A two-dimensional sampling grid is established based on the plane being measured, and the coordinates of the grid nodes are determined.

[0008] Establish a spatial coordinate system and call up the calibration data to obtain the calibration image plane;

[0009] The camera module is controlled to perform axial scanning perpendicular to the plane under test, and continuously acquires image sequences covering the two-dimensional sampling grid.

[0010] The image sequence is divided according to the grid node coordinates to obtain a grid image sequence;

[0011] The sharpness of the grid image sequence is calculated using a no-reference sharpness evaluation method to obtain a sharpness sequence;

[0012] Establish the index relationship between the coordinates of the grid nodes and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set;

[0013] Based on the set of sharpness points, a plane fitting is performed to obtain a global focal plane model;

[0014] The parallelism parameters of the global focal plane relative to the calibration image plane are calculated based on the global focal plane model.

[0015] The attitude of the placement platform of the measured plane is adjusted by using the parallel parameters.

[0016] Optionally, after adjusting the attitude of the placement platform of the measured plane using the parallel parameters, the method includes:

[0017] Determine whether the parallel parameter is less than a preset parameter;

[0018] If not, the parallel parameters of the measured plane after attitude adjustment are recalculated to iterate the parallel parameters until the parallel parameters are less than the preset parameters.

[0019] Optionally, after establishing the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set, the method further includes:

[0020] Calculate the difference between the maximum and minimum focal lengths of the sharpness point set;

[0021] Calculate the mean square error of all focus values ​​in all sharpness point sets;

[0022] The data reliability of the sharpness point set is verified by the difference and the mean square error, and the verification result is obtained.

[0023] When the verification result is unsuccessful, outliers in the sharpness point set are removed by a random sampling consensus algorithm, and the sharpness point set is updated.

[0024] Optionally, the step of calculating the sharpness of the grid image sequence using a no-reference sharpness evaluation method to obtain a sharpness sequence includes:

[0025] Calculate the sharpness of each grid image in the grid image sequence;

[0026] Using grid images as an index, establish a correspondence between the sharpness of each grid image and its shooting height;

[0027] A sharpness sequence is generated based on the correspondence.

[0028] Optionally, calculating the sharpness of each grid image in the grid image sequence includes:

[0029] All images in the grid image sequence are divided into several sub-images according to the grid node coordinates;

[0030] Calculate the sharpness of each sub-image;

[0031] The average sharpness of all images is calculated based on the sharpness of the sub-images, using images as the unit.

[0032] The sharpness of all images in the grid image sequence is determined by the average sharpness.

[0033] Optionally, after establishing the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set, the method further includes:

[0034] Verify whether all sharpness points in the sharpness point set are within the height range of the axial scan;

[0035] When a sharpness point exists outside the height range, the initial height of the axial scan is adjusted based on the absolute value of the difference between the sharpness point coordinates and the height range, and the image sequence is reacquired based on the adjusted initial height.

[0036] Optionally, adjusting the attitude of the placement platform of the measured plane using the parallel parameters includes:

[0037] Obtain the pitch and roll angles from the parallel parameters;

[0038] The pitch angle and the roll angle are respectively mapped to the placement platform of the measured plane, and the angle adjustment amount is calculated based on the mapping results;

[0039] When the angle adjustment exceeds the preset range, the attitude of the placement platform is adjusted according to the pitch angle and / or roll angle.

[0040] A second aspect of this application provides a multi-point focusing fitting microscopic imaging plane leveling system, the system comprising:

[0041] The first establishment unit is used to establish a two-dimensional sampling grid based on the measured plane and determine the coordinates of the grid nodes;

[0042] The second establishment unit is used to establish a spatial coordinate system and call up calibration data to obtain the calibration image plane;

[0043] The acquisition unit is used to control the imaging module to perform axial scanning perpendicular to the plane under test, and continuously acquire image sequences covering the two-dimensional sampling grid.

[0044] A partitioning unit is used to partition the image sequence according to the grid node coordinates to obtain a grid image sequence;

[0045] The first calculation unit is used to calculate the sharpness of the grid image sequence using a no-reference sharpness evaluation method to obtain a sharpness sequence;

[0046] The third establishment unit is used to establish the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence, so as to obtain the sharpness point set;

[0047] A fitting unit is used to perform plane fitting based on the sharpness point set to obtain a global focal plane model;

[0048] The second calculation unit is used to calculate the parallelism parameters of the global focal plane relative to the calibration image plane based on the global focal plane model.

[0049] The first adjustment unit is used to adjust the attitude of the placement platform of the measured plane using the parallel parameters.

[0050] Optionally, the system includes:

[0051] The judgment unit is used to determine whether the parallel parameter is less than a preset parameter;

[0052] An iterative unit is used to recalculate the parallel parameters of the measured plane after attitude adjustment when the judgment result of the judgment unit is negative, so as to iterate the parallel parameters until the parallel parameters are less than the preset parameters.

[0053] Optionally, the system further includes:

[0054] The third calculation unit is used to calculate the difference between the maximum and minimum focal values ​​of the sharpness point set;

[0055] The fourth calculation unit is used to calculate the mean square error of all focal values ​​in all sharpness point sets;

[0056] The first verification unit is used to verify the data reliability of the sharpness point set through the difference and the mean square error, and obtain the verification result.

[0057] The elimination unit is used to eliminate outliers in the sharpness point set and update the sharpness point set when the verification result is unsuccessful.

[0058] Optionally, the first computing unit is specifically used for:

[0059] Calculate the sharpness of each grid image in the grid image sequence;

[0060] Using grid images as an index, establish a correspondence between the sharpness of each grid image and its shooting height;

[0061] A sharpness sequence is generated based on the correspondence.

[0062] Optionally, the first computing unit is specifically used for:

[0063] All images in the grid image sequence are divided into several sub-images according to the grid node coordinates;

[0064] Calculate the sharpness of each sub-image;

[0065] The average sharpness of all images is calculated based on the sharpness of the sub-images, using images as the unit.

[0066] The sharpness of all images in the grid image sequence is determined by the average sharpness.

[0067] Optionally, the system further includes:

[0068] The second verification unit is used to verify whether all sharpness points in the sharpness point set are within the height range of the axial scan.

[0069] The second adjustment unit is used to adjust the initial height of the axial scan based on the absolute value of the difference between the coordinates of the sharpness point and the height range when there is a sharpness point outside the height range, and to reacquire the image sequence based on the adjusted initial height.

[0070] Optionally, the first adjustment unit is specifically used for:

[0071] Obtain the pitch and roll angles from the parallel parameters;

[0072] The pitch angle and the roll angle are respectively mapped to the placement platform of the measured plane, and the angle adjustment amount is calculated based on the mapping results;

[0073] When the angle adjustment exceeds the preset range, the attitude of the placement platform is adjusted according to the pitch angle and / or roll angle.

[0074] A third aspect of this application provides a microscopic imaging plane leveling device for multi-point focusing fitting, the device comprising:

[0075] Processor, memory, input / output units, and bus;

[0076] The processor is connected to the memory, the input / output unit, and the bus;

[0077] The memory stores a program, which the processor invokes to execute the first aspect and any one of the optional methods in the first aspect.

[0078] A fourth aspect of this application provides a computer-readable storage medium on which a program is stored, which, when executed on a computer, performs the methods of the first aspect and any one of the first aspects.

[0079] As can be seen from the above technical solutions, this application has the following advantages:

[0080] By establishing a two-dimensional sampling grid and performing axial scanning, multi-point image data can be acquired simultaneously across the entire field of view. Combined with a referenceless sharpness evaluation algorithm, the focal position of each sampling point is automatically calculated and located, constructing a global focal plane model. This allows for quantitative characterization of the spatial attitude of the measured plane and the calibration image plane with micrometer-level precision. The calculated parallel parameters drive the automatic adjustment of the placement platform, achieving rapid leveling of the sample surface. The entire process requires no manual intervention and automatically completes data acquisition, sharpness calculation, plane fitting, and attitude correction, significantly improving the focusing efficiency and leveling accuracy of the microscopic imaging system. Attached Figure Description

[0081] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0082] Figure 1 This is a schematic flowchart of an embodiment of the microscopic imaging plane leveling method for multi-point focusing fitting in this application;

[0083] Figure 2a This is a schematic flowchart of an embodiment of the first stage of the multi-point focusing fitting microscopic imaging plane leveling method in this application;

[0084] Figure 2b This is a schematic flowchart of an embodiment of the second stage of the multi-point focusing fitting microscopic imaging plane leveling method in this application;

[0085] Figure 3 This is a schematic diagram of an embodiment of the microscopic imaging plane leveling system for multi-point focusing fitting in this application;

[0086] Figure 4 This is a schematic diagram of another embodiment of the microscopic imaging plane leveling system for multi-point focusing fitting in this application;

[0087] Figure 5 This is a schematic diagram of an embodiment of the microscopic imaging plane leveling device for multi-point focusing fitting in this application. Detailed Implementation

[0088] It should be noted that the multi-point focusing fitting microscopic imaging plane leveling method provided in this application can be applied to terminals, systems, and servers. For example, the terminal can be a smartphone, computer, tablet, smart TV, smartwatch, portable computer terminal, or a desktop computer or other fixed terminal. For ease of explanation, this application uses the terminal as the execution subject for illustration.

[0089] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0090] Please see Figure 1 This application first provides an embodiment of a microscopic imaging plane leveling method based on multi-point focusing fitting, which includes:

[0091] S101. Establish a two-dimensional sampling grid based on the plane under test and determine the coordinates of the grid nodes;

[0092] A two-dimensional sampling grid refers to a matrix of measurement points arranged according to row and column rules on the plane being measured, used to cover the entire field of view of the plane being measured during photography. The grid node coordinates represent the spatial position of each measurement point in the matrix on the plane being measured, denoted as (x...). i y i ).

[0093] The terminal constructs a two-dimensional sampling grid on the test plane of the product under inspection (such as a display panel, wafer, or microstructure sample) and determines the coordinates of each grid node. In actual production, several rows and columns of measurement points can be evenly distributed within the field of view, covering the center and edge areas to ensure the representativeness of subsequent fitting. The advantage of this approach is that, compared to single-point focusing, it can simultaneously acquire information from the entire field, providing sufficient constraints for subsequent global modeling.

[0094] Specifically, in this embodiment, the two-dimensional sampling grid should typically be arranged with at least a 3×3 grid (to cover the center and edge areas of the field of view). A 5×5 or higher density distribution is recommended. The specific number can be flexibly determined according to actual needs, and no special limitation is made here.

[0095] S102. Establish a spatial coordinate system and call up the calibration data to obtain the calibration image plane;

[0096] The calibration image plane is a reference imaging plane obtained through calibration in the system coordinate system, and is used as a benchmark for attitude evaluation.

[0097] Specifically, after the imaging device of the detection equipment is manufactured or maintained, an imaging geometry calibration is performed, which generates the intrinsic and extrinsic parameters of the imaging device, as well as the position and normal information of the image plane in the current spatial coordinate system. The image plane, as an ideal plane captured by the camera of the detection equipment, is used to quantify the attitude deviation of the current measured plane relative to the camera imaging plane, ensuring that subsequent parallelism evaluation has a unified reference.

[0098] S103. Control the shooting module to perform axial scanning perpendicular to the plane under test, and continuously acquire image sequences covering the two-dimensional sampling grid;

[0099] Axial scanning refers to continuously acquiring a sequence of images at different heights along the Z direction, which is perpendicular to the plane being measured.

[0100] The terminal-controlled imaging module performs an axial scan from top to bottom along the Z-axis of the spatial coordinate system, continuously acquiring image sequences covering the entire grid area without changing the X and Y positions. In engineering, the scan stroke, movement speed, trigger interval, and exposure time can be set to achieve high-speed and stable equidistant sampling. Taking the panel detection line as an example, setting an appropriate stroke ensures that the sharpness curve of each grid area experiences a transition from "out of focus to focus and back to out of focus," thereby guaranteeing the reliability of subsequent peak extraction and avoiding the time loss caused by repeatedly scanning each grid point.

[0101] Specifically, for example, setting the camera exposure time t and the starting position Z on the z-axis. s The scanning stroke L, the physical interval distance space for triggering the camera to take a picture, and the z-axis movement speed v (e.g., t=1ms, shooting stroke L=4mm, interval space=0.1mm, v=4mm / s, the specific values ​​can be determined according to actual needs, and no special limitation is made here) control the z-axis to drive the imaging module from the starting position Z s It begins to move downwards at high speed continuously. At each interval of space distance, the motion controller outputs a position comparison pulse signal to trigger the camera to take pictures until the Z-axis movement reaches the set stroke L. During the Z-axis movement, the camera will take a large number of images. For example, according to the above parameter settings, a total of 41 images will be collected, denoted as {In | n=0, 1, 2, 3, ... , 39,40}.

[0102] It should be noted that the camera captures the entire plane being measured, thus enabling multi-point / region focusing. This avoids repeating the Z-axis scanning acquisition process for each point / region. Therefore, in this embodiment, the allowable range for the scanning travel is more flexible, ensuring that each point / region exists in the image sequence in a state from blurry to clear to blurry again, in order to minimize the time spent on repeated scanning acquisition.

[0103] S104. Divide the image sequence according to the grid node coordinates to obtain a grid image sequence;

[0104] Reference-free sharpness evaluation quantifies the imaging sharpness of a single image or its sub-regions (such as the coordinates of each grid node in a two-dimensional sampling grid) without relying on a standard sharpness reference image.

[0105] The terminal spatially divides the acquired image sequence based on the grid node coordinates, obtaining a grid image sequence for each grid region. In practice, the terminal can crop or index each image according to the grid window based on the image sequence, so that each grid node corresponds to an image sequence that varies with height, laying the data foundation for subsequent node-by-node sharpness analysis.

[0106] S105. The sharpness of the grid image sequence is calculated using a no-reference sharpness evaluation method to obtain a sharpness sequence;

[0107] A sharpness sequence represents the sharpness curve of the same spatial location as the shooting height.

[0108] When calculating the sharpness of a grid image sequence, the terminal quantifies the imaging quality of each grid image using a no-reference image sharpness evaluation method. This "no-reference image sharpness evaluation method" refers to an algorithm that determines the richness of image detail and sharpness by analyzing the image's own grayscale variation characteristics, frequency distribution characteristics, or structural characteristics without relying on an ideal reference image.

[0109] In practical implementation, the terminal employs various methods for calculating image sharpness. For example, a gradient function-based evaluation method determines edge sharpness by analyzing the spatial variation of image pixel grayscale. Alternatively, an image transform domain-based evaluation method can be used, converting the image to the frequency domain and determining sharpness based on the energy distribution of high-frequency information. An entropy function method can also be employed, calculating the entropy of image grayscale information to reflect the richness of image detail. Furthermore, a secondary blur algorithm can be used, blurring the image again and comparing its difference from the original image to infer whether the original image is sharp. When processing images with obvious local structural features, a structural similarity measurement method can be used to estimate image sharpness through the feature values ​​of brightness, contrast, and structural information.

[0110] S106. Establish the index relationship between the coordinates of the grid nodes and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set;

[0111] The sharpness point set is the set that associates the height corresponding to the sharpness peak of each grid node with its coordinates, denoted as . The sharpness point parameter is denoted as .

[0112] For each grid node, the terminal determines the position of the sharpness peak in the sharpness sequence and establishes an index relationship between the shooting height corresponding to that position and the coordinates of that node, forming a sharpness point set. To avoid the peak falling at the edge of the sequence due to improper setting of the scanning start height, when a peak is detected to exceed the preset range, the scanning start point is adjusted and the image sequence is re-acquired to ensure that the peak is completely covered.

[0113] S107. Perform plane fitting based on the sharpness point set to obtain a global focal plane model;

[0114] The global focal plane model is an approximate plane obtained by fitting a set of sharpness points, used to characterize the spatial attitude of the best imaging plane across the entire field.

[0115] A global focal plane model is obtained by fitting a plane based on the sharpness point set. Least squares fitting is preferred, combined with Random Sample Consensus (RANSAC) for outlier removal to suppress interference from local scratches, blemishes, or texture anomalies. This step transforms the discrete "optimal focal length height-spatial position" data into a continuous approximate plane, giving the overall focus state a computable geometric expression, significantly reducing regional errors compared to relying solely on single-point focusing.

[0116] S108. Calculate the parallelism parameters of the global focal plane relative to the calibration image plane based on the global focal plane model;

[0117] Parallel parameters are indices that quantify the relative attitude between the global focal plane and the calibration image plane, and include at least tilt components around the x-axis and y-axis, serving as control variables for platform leveling.

[0118] Parallel parameters are calculated based on the spatial relationship between the global focal plane model and the calibration image plane. The parallel parameters include at least attitude angular components around the x-axis and y-axis. These parameters represent the relative tilt of the measured plane and the calibration image plane, and provide a quantitative description of the measured plane.

[0119] S109. Adjust the attitude of the placement platform of the measured plane using the parallel parameters.

[0120] The parallel parameters are mapped to the placement platform on the measured plane, and the terminal adjusts the attitude of the platform based on the parallel parameters. In engineering implementation, the degrees of freedom of the placement platform include the horizontal axis angle and the vertical axis angle. That is, during compensation, the terminal calculates the corresponding adjustment amount based on the parallel parameters and executes the adjustment control logic.

[0121] This embodiment establishes a two-dimensional sampling grid and performs axial scanning, enabling simultaneous acquisition of multi-point image data across the entire field of view. Combined with a referenceless sharpness evaluation algorithm, the focal position of each sampling point is automatically calculated and located, constructing a global focal plane model. This allows for quantitative characterization of the spatial attitude of the measured plane and the calibration image plane with micrometer-level precision. The calculated parallel parameters drive the automatic adjustment of the placement platform, achieving rapid leveling of the sample surface. The entire process requires no manual intervention, automatically completing data acquisition, sharpness calculation, plane fitting, and attitude correction, significantly improving the focusing efficiency and leveling accuracy of the microscopic imaging system.

[0122] Please see Figure 2a and Figure 2b This application provides another embodiment of a microscopic imaging plane leveling method based on multi-point focusing fitting, which includes:

[0123] S201. Establish a two-dimensional sampling grid based on the plane being measured, and determine the coordinates of the grid nodes;

[0124] S202. Establish a spatial coordinate system and call up the calibration data to obtain the calibration image plane;

[0125] S203. Control the shooting module to perform axial scanning perpendicular to the plane under test, and continuously acquire image sequences covering the two-dimensional sampling grid;

[0126] S204. Divide the image sequence according to the grid node coordinates to obtain a grid image sequence;

[0127] Steps S201 to S204 in this embodiment are similar to steps S101 to S104 in the previous embodiment, and will not be described in detail here.

[0128] S205. Divide all images in the grid image sequence into several sub-images according to the grid node coordinates;

[0129] After the terminal completes the establishment of a two-dimensional sampling grid on the measured plane and acquires the image sequence covering the grid, it has divided the entire image into multiple corresponding grid images by the grid node coordinates. Each grid image reflects the imaging situation of a sampling area on the measured plane at different shooting heights.

[0130] The terminal further segments each grid image in the grid image sequence according to a pre-defined row and column division method, splitting the entire image into several sub-images of the same size. For example, when inspecting a display panel, the terminal divides a grid image into three equal parts horizontally and vertically, resulting in nine sub-images, each corresponding to a local image at a different location within the area. This method refines the spatial information within each grid region without requiring additional data acquisition, ensuring that subsequent sharpness calculations no longer rely solely on a single location but cover the entire grid region, thus improving the reliability of focus determination.

[0131] S206. Calculate the sharpness of each sub-image;

[0132] After obtaining multiple sub-images corresponding to each grid image, the terminal performs sharpness calculations on each sub-image individually. Sub-image sharpness refers to the result of quantifying the image sharpness based on the pixel features of the sub-image itself using a no-reference image sharpness evaluation method.

[0133] The terminal selects specific algorithms based on the actual application scenario. For example, by statistically analyzing the gradient intensity of edge pixels in the sub-image, the more dramatic the gradient change, the clearer the edge. Or by analyzing the high-frequency components in the frequency domain of the sub-image, a higher proportion of high-frequency components indicates that the sub-image has richer details and more focused imaging.

[0134] In scenarios involving the inspection of semiconductor wafers or display panels, each sub-image often contains several circuit lines or pixel units. The sharpness value calculated using the method described above can effectively reflect whether the local area is in focus. The terminal records the sharpness value corresponding to each sub-image, laying the foundation for subsequent comprehensive evaluation at the image level.

[0135] S207. Calculate the average sharpness of all images based on the sharpness of the sub-images, using the image as the unit.

[0136] After completing the sharpness calculation for all sub-images, the terminal summarizes these sharpness results on a whole grid image basis.

[0137] Specifically, the terminal statistically averages the sharpness values ​​of all sub-images within the same grid image to obtain the average sharpness value corresponding to that grid image. The significance of this process is that individual sub-images may be affected by localized stains, reflections, or texture anomalies, making it easy to misjudge if a single point's result is used directly. However, by averaging multiple sub-images within the same grid image, the disturbances caused by local anomalies can be offset to some extent, making the overall sharpness evaluation of the grid image more stable. For example, when detecting a glass substrate with localized bright spots, the sharpness of individual sub-images may be abnormally high or low, but the average result can still accurately reflect the overall focus of that grid area.

[0138] S208. Determine the sharpness of all images in the grid image sequence using the average sharpness.

[0139] After obtaining the average sharpness value of each grid image, the terminal uses this average sharpness value as the sharpness evaluation result for that grid image, completing the sharpness determination process for all images in the entire grid image sequence. At this point, for the same grid node, the terminal has obtained multiple grid images corresponding to that node at different shooting heights, along with the sharpness values ​​of these grid images. Taking the detection of a display panel as an example, the multiple grid images corresponding to a certain grid node represent the imaging state of that area at different heights. After this step, the terminal can clearly know whether the area is sharp or slightly blurry at each height. This result will be used subsequently to construct a sharpness curve as a function of height and further used to determine the focus position.

[0140] S209. Using grid images as an index, establish the correspondence between the sharpness of each grid image and the shooting height;

[0141] After determining the sharpness of the entire grid image sequence, the terminal further establishes a one-to-one correspondence between the sharpness of each grid image and its corresponding shooting height. Shooting height refers to the height position of the shooting module relative to the measured plane during axial scanning. The terminal records the corresponding height value for each grid image acquired. In practical applications, scanning can be performed at fixed height intervals, for example, acquiring one frame every 100 micrometers. Through this step, the terminal establishes an ordered index relationship between the grid image identifier, its sharpness, and the shooting height. This mapping directly reflects the imaging changes of the same grid area at different heights, providing a data foundation for subsequently generating a sequence where sharpness varies with height.

[0142] S210. Generate a sharpness sequence based on the correspondence.

[0143] After establishing the correspondence between grid image sharpness and shooting height, the terminal generates a sharpness sequence based on this correspondence. The sharpness sequence is a set of sharpness values ​​arranged in order of shooting height, showing the trajectory of sharpness change for the same grid node during scanning from low to high or from high to low. The terminal typically concatenates the sharpness values ​​corresponding to the corresponding heights according to the actual axial scanning order, thus forming a curve reflecting the change in sharpness with height. In detection applications, this sharpness sequence often exhibits a trend of "first becoming clearer, then becoming blurrier," and the height corresponding to the maximum value in the sequence is the focal height of that grid node.

[0144] At this point, the terminal has prepared the data foundation for focus localization for each grid node. Subsequently, the focus position can be extracted based on these sharpness sequences, and further plane fitting and attitude evaluation can be performed.

[0145] S211. Establish the index relationship between the coordinates of the grid nodes and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set;

[0146] Step S211 in this embodiment is similar to step S106 in the previous embodiment, and will not be described in detail here.

[0147] S212. Calculate the difference between the maximum and minimum focal values ​​of the sharpness point set;

[0148] After establishing the sharpness point set, the terminal further analyzes the focal height corresponding to each grid node in the sharpness point set. Specifically, the terminal assigns a focal position to each sharpness point, determined by the maximum sharpness value. To determine the overall data dispersion and identify the rationality of the focal distribution, the terminal first finds the maximum and minimum focal positions among all sharpness points and calculates the difference between them. In practical applications, such as when inspecting a display panel, if a certain area of ​​the surface has warping or local depressions, the focal position obtained in that area will deviate from other areas. This difference can reflect whether the overall height variation range of the measured plane is abnormal, thus providing a basis for subsequent data reliability verification.

[0149] S213. Calculate the mean square error of all focus values ​​in all sharpness point sets;

[0150] After obtaining the difference between the maximum and minimum values, the terminal continues to analyze the focus position distribution of all sharpness points. Specifically, the terminal calculates the mean square error (MSE) of all focus heights to reflect the overall dispersion of the point set. In actual system operation, if some sharpness points have high noise or inaccurate focus determination due to local reflections, blemishes, or other reasons, these points will deviate significantly from the overall trend, increasing the MSE. By analyzing the MSE, the system can quantify the overall stability of the focus data, providing a basis for subsequent data screening.

[0151] S214. Verify the data reliability of the sharpness point set using the difference and the mean square error to obtain the verification result;

[0152] Based on the difference and mean square error, the terminal verifies the reliability of the sharpness point set. The verification process identifies data points that do not conform to the overall variation pattern by judging the distribution trend of the focal height. For example, when the focal position of a node is far from the reasonable distribution range of other nodes, the system will determine that the data may contain errors. When inspecting high-precision microstructure devices, changes in illumination, surface defects, or local defocusing can cause some points to be misidentified as focal points. This verification can prevent abnormal points from directly participating in subsequent plane fitting, improving the accuracy of the fitting model.

[0153] S215. When the verification result is unsuccessful, outliers in the sharpness point set are removed by random sampling consistency algorithm, and the sharpness point set is updated.

[0154] When the verification results show unreliable data points, the terminal performs outlier removal on the sharpness point set. In the specific implementation, the terminal uses a random sampling consistency method to calculate the point set. By randomly sampling multiple times and comparing the overall consistency, data points deviating from the overall trend are identified and removed from the point set. After removal, the system updates the sharpness point set to contain only reliable focus data. In microscopic imaging measurement scenarios, this step can significantly improve the accuracy of plane fitting and avoid errors caused by outliers to the focal plane model.

[0155] S216. Verify whether all sharpness points in the sharpness point set are within the height range of the axial scan;

[0156] After data reliability processing is completed, to ensure that the focus data is completely within the effective range of the axial scan, the terminal checks the focus height of all sharpness points. In practice, the terminal compares the focus position of each sharpness point with the start and end heights of the current axial scan to determine if any points exceed the scan range. If the focus height of a point exceeds the scan range, for example, appearing above or below the scan area, it indicates that the previous scan range did not cover the true focus position of that point. In this case, the scan parameters need to be corrected to prepare for subsequent data acquisition.

[0157] S217. When there are sharpness points outside the height range, adjust the initial height of the axial scan according to the absolute value of the difference between the sharpness point coordinates and the height range, and reacquire the image sequence based on the adjusted initial height.

[0158] If a focal point exists outside the scanning interval, the terminal adjusts the starting height of the axial scan based on the height difference between that point and the scanning interval boundary. The adjustment direction is determined by the sign of the difference; for example, when the focal point is above the upper scan boundary, the starting height needs to be increased; when the focal point is below the lower scan boundary, the starting height needs to be decreased. After adjustment, the terminal re-executes the axial scan and acquires a new image sequence to ensure that the new scanning interval completely covers the focal areas of all grid nodes. In actual microscopic measurement scenarios, this step ensures that the focal point determination process is not missed due to improper scanning interval settings, thereby improving the overall system's focal point positioning accuracy and stability.

[0159] S218. Perform plane fitting based on the sharpness point set to obtain a global focal plane model;

[0160] S219. Calculate the parallelism parameters of the global focal plane relative to the calibration image plane based on the global focal plane model;

[0161] Steps S218 to S219 in this embodiment are similar to steps S107 to S108 in the previous embodiment, and will not be described in detail here.

[0162] S220. Obtain the pitch angle and roll angle from the parallel parameters;

[0163] After obtaining the global focal plane model, the terminal extracts angular information reflecting the attitude of the measured plane from the model. The focal plane model characterizes the height distribution of each grid node in the axial direction. By analyzing the spatial tilt trend of this plane, the terminal can determine the angular quantities around the two principal axes, including the pitch angle in the horizontal direction and the roll angle in the vertical direction. These two angles indicate the direction and degree of rotation of the measured plane relative to the ideal imaging plane, respectively. In practical microscopic inspection scenarios, such as imaging display panels or wafers, if the focal plane shows an increasing or decreasing trend in a certain direction, it can be determined that the area is tilted and requires correction in the corresponding direction.

[0164] S221. Map the pitch angle and the roll angle to the placement platform of the measured plane, respectively, and calculate the angle adjustment amount based on the mapping results;

[0165] After acquiring the pitch and roll angles, the terminal converts these two angular quantities into attitude adjustment parameters to be performed by the placement platform. The placement platform typically has the ability to fine-tune around two rotation axes. To achieve the desired image parallelism, the terminal converts the angle values ​​into corresponding rotation commands based on the mapping relationship between the angles and the platform mechanism. For example, when the focal plane is tilted forward, the pitch angle is positive, and the terminal generates a control command to fine-tune the platform backward; when the focal plane is raised to the left, the roll angle is negative, and the platform will perform a slight reverse rotation according to the compensation requirements in that direction. Through this mapping method, the angular information in the focal plane model is accurately converted into attitude correction actions that can be executed upon landing.

[0166] S222. When the angle adjustment exceeds the preset range, the attitude of the placement platform is adjusted according to the pitch angle and / or roll angle.

[0167] After calculating the angle adjustment amount, the terminal performs a threshold judgment on the magnitude of the adjustment. When the calculated adjustment amount exceeds the preset executable range, such as exceeding the platform's single fine-tuning capability or potentially leading to overcompensation, the terminal generates a new attitude correction command based on the pitch angle and / or roll angle to limit the platform's rotational movements within a reasonable range. In high-precision microscopic imaging applications, a single adjustment typically allows only a small displacement; therefore, this step avoids excessively large adjustments that could cause vibration or overshoot in the imaging system, making the adjustment process more stable and controllable.

[0168] S223. Determine whether the parallel parameter is less than a preset parameter;

[0169] After the platform completes an attitude adjustment, the terminal needs to verify the current parallel state. The terminal reconstructs the focal plane model according to the aforementioned steps, calculates the updated parallel parameters, and compares these parameters with a preset threshold. This threshold is typically determined based on the depth-of-field range of the imaging system or the required accuracy for product inspection. When the parallel parameters fall within the set threshold, it indicates that the current planar attitude meets the imaging requirements. If the parallel parameters do not fall within the set threshold, step S224 is executed.

[0170] S224. If not, then recalculate the parallel parameters of the measured plane after attitude adjustment, and iterate the parallel parameters until the parallel parameters are less than the preset parameters.

[0171] If the judgment result indicates that the parallel parameters still do not reach the preset threshold, the terminal will execute the attitude correction process again, recalculate the adjusted parallel parameters, and repeat the judgment steps until the parallel parameters meet the requirements. This repeated closed-loop process allows the system to gradually approach the parallel state while avoiding excessive single adjustments that could cause vibration or error accumulation. Through this progressive correction mechanism, the platform can ultimately ensure that the measured plane and the calibration image plane are highly consistent, thereby obtaining a clear and uniform microscopic imaging effect across the entire field of view.

[0172] This embodiment first establishes a two-dimensional sampling grid on the measured plane and determines the coordinates of each grid node, thereby achieving spatial distribution sampling across the entire field of view. Subsequently, a spatial coordinate system is established and calibration data is retrieved to obtain a calibration image plane, providing a unified reference for subsequent attitude assessment. The imaging module is controlled to perform axial scanning along the vertical direction, continuously acquiring image sequences covering the entire grid area during the scanning process, thereby obtaining multi-layered imaging data containing focus change information.

[0173] During image processing, sharpness is evaluated for each region through grid division and block calculation, resulting in a sharpness sequence reflecting focus changes. By analyzing the sharpness peak locations, a correspondence between grid nodes and focus coordinates is established, forming a sharpness point set. Based on this set, data verification and anomaly removal are performed to ensure data stability and reliability. When the focus exceeds the scanning height range, the system automatically adjusts the scanning start height and re-acquires the image, ensuring complete coverage of focus information.

[0174] Subsequently, plane fitting is performed on the sharpness point set to establish a global focal plane model. Parallel parameters are then calculated based on the model and decomposed into pitch and roll angles to guide platform attitude adjustment. Finally, closed-loop control is formed through threshold judgment and iterative correction. This method achieves automatic extraction of multi-point focusing data, focal plane modeling, and attitude self-calibration. It can complete high-precision automatic leveling of the microscopic imaging system without manual intervention, significantly improving imaging consistency and measurement accuracy.

[0175] The above provides a detailed description of the microscopic imaging plane leveling method using multi-point focusing fitting in the embodiments of this application. The following will provide a detailed description of the microscopic imaging plane leveling system and apparatus using multi-point focusing fitting.

[0176] Please see Figure 3 This application provides an embodiment of a multi-point focusing fitting microscopic imaging plane leveling system, which includes:

[0177] The first establishment unit 301 is used to establish a two-dimensional sampling grid based on the measured plane and determine the grid node coordinates;

[0178] The second establishment unit 302 is used to establish a spatial coordinate system and call up calibration data to obtain the calibration image plane;

[0179] The acquisition unit 303 is used to control the shooting module to perform axial scanning perpendicular to the plane under test, and continuously acquire image sequences covering the two-dimensional sampling grid;

[0180] The segmentation unit 304 is used to segment the image sequence according to the grid node coordinates to obtain a grid image sequence;

[0181] The first calculation unit 305 is used to calculate the sharpness of the grid image sequence using a no-reference sharpness evaluation method to obtain a sharpness sequence;

[0182] The third establishing unit 306 is used to establish the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence, so as to obtain a sharpness point set;

[0183] Fitting unit 307 is used to perform plane fitting based on the sharpness point set to obtain a global focal plane model;

[0184] The second calculation unit 308 is used to calculate the parallelism parameters of the global focal plane relative to the calibration image plane based on the global focal plane model.

[0185] The first adjustment unit 309 is used to adjust the attitude of the placement platform of the measured plane using the parallel parameters.

[0186] In this embodiment, the functions of each unit are the same as those described above. Figure 1 The steps in the illustrated embodiments are the same and will not be repeated here.

[0187] Please see Figure 4 This application provides another embodiment of a multi-point focusing fitting microscopic imaging plane leveling system, which includes:

[0188] The first establishment unit 401 is used to establish a two-dimensional sampling grid based on the measured plane and determine the grid node coordinates;

[0189] The second establishment unit 402 is used to establish a spatial coordinate system and call up calibration data to obtain the calibration image plane;

[0190] Acquisition unit 403 is used to control the shooting module to perform axial scanning perpendicular to the measured plane and continuously acquire image sequences covering the two-dimensional sampling grid;

[0191] The partitioning unit 404 is used to partition the image sequence according to the grid node coordinates to obtain a grid image sequence;

[0192] The first calculation unit 405 is used to calculate the sharpness of the grid image sequence using a no-reference sharpness evaluation method to obtain a sharpness sequence;

[0193] The third establishing unit 406 is used to establish the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence, so as to obtain a sharpness point set;

[0194] The second verification unit 407 is used to verify whether all sharpness points in the sharpness point set are within the height range of the axial scan.

[0195] The second adjustment unit 408 is used to adjust the initial height of the axial scan based on the absolute value of the difference between the coordinates of the sharpness point and the height range when there is a sharpness point outside the height range, and to reacquire the image sequence based on the adjusted initial height.

[0196] The third calculation unit 409 is used to calculate the difference between the maximum and minimum focal values ​​of the sharpness point set;

[0197] The fourth calculation unit 410 is used to calculate the mean square error of all focal values ​​in all sharpness point sets;

[0198] The first verification unit 411 is used to verify the data reliability of the sharpness point set through the difference and the mean square error, and obtain the verification result.

[0199] The elimination unit 412 is used to eliminate outliers in the sharpness point set and update the sharpness point set when the verification result is unsuccessful, by using a random sampling consistency algorithm.

[0200] Fitting unit 413 is used to perform plane fitting based on the sharpness point set to obtain a global focal plane model;

[0201] The second calculation unit 414 is used to calculate the parallelism parameters of the global focal plane relative to the calibration image plane based on the global focal plane model.

[0202] The first adjustment unit 415 is used to adjust the attitude of the placement platform of the measured plane using the parallel parameters.

[0203] Judgment unit 416 is used to determine whether the parallel parameter is less than a preset parameter;

[0204] The iteration unit 417 is used to recalculate the parallel parameters of the measured plane after attitude adjustment when the judgment result of the judgment unit 416 is negative, so as to iterate the parallel parameters until the parallel parameters are less than the preset parameters.

[0205] In this embodiment, the first computing unit 405 is specifically used for:

[0206] Calculate the sharpness of each grid image in the grid image sequence;

[0207] Using grid images as an index, establish a correspondence between the sharpness of each grid image and its shooting height;

[0208] A sharpness sequence is generated based on the correspondence.

[0209] In this embodiment, the first computing unit 405 is specifically used for:

[0210] All images in the grid image sequence are divided into several sub-images according to the grid node coordinates;

[0211] Calculate the sharpness of each sub-image;

[0212] The average sharpness of all images is calculated based on the sharpness of the sub-images, using images as the unit.

[0213] The sharpness of all images in the grid image sequence is determined by the average sharpness.

[0214] In this embodiment, the first adjustment unit 415 is specifically used for:

[0215] Obtain the pitch and roll angles from the parallel parameters;

[0216] The pitch angle and the roll angle are respectively mapped to the placement platform of the measured plane, and the angle adjustment amount is calculated based on the mapping results;

[0217] When the angle adjustment exceeds the preset range, the attitude of the placement platform is adjusted according to the pitch angle and / or roll angle.

[0218] In this embodiment, the functions of each unit are the same as those described above. Figure 2a and Figure 2b The steps in the illustrated embodiments are the same and will not be repeated here.

[0219] Please see Figure 5 This application provides an embodiment of a microscopic imaging plane leveling device for multi-point focusing fitting, comprising:

[0220] Processor 501, memory 502, input / output unit 503, bus 504;

[0221] The processor 501 is connected to the memory 502, the input / output unit 503 and the bus 504;

[0222] The processor 501 specifically executes... Figures 1 to 2b The specific operations corresponding to the steps in the method will not be elaborated here.

[0223] This application also relates to a computer-readable storage medium on which a program is stored, characterized in that, when the program is run on a computer, it causes the computer to perform any of the methods described above.

[0224] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0226] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0227] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0228] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for leveling a microscopic imaging plane using multi-point focusing fitting, characterized in that, The method includes: A two-dimensional sampling grid is established based on the plane being measured, and the coordinates of the grid nodes are determined. Establish a spatial coordinate system and call up the calibration data to obtain the calibration image plane; The camera module is controlled to perform axial scanning perpendicular to the plane under test, and continuously acquires image sequences covering the two-dimensional sampling grid. The image sequence is divided according to the grid node coordinates to obtain a grid image sequence; The sharpness of the grid image sequence is calculated using a no-reference sharpness evaluation method to obtain a sharpness sequence; Establish the index relationship between the coordinates of the grid nodes and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set; Based on the set of sharpness points, a plane fitting is performed to obtain a global focal plane model; The parallelism parameters of the global focal plane relative to the calibration image plane are calculated based on the global focal plane model. The attitude of the placement platform of the measured plane is adjusted by the parallel parameters. The step of calculating the sharpness of the grid image sequence using a no-reference sharpness evaluation method to obtain a sharpness sequence includes: Calculate the sharpness of each grid image in the grid image sequence; Using grid images as an index, establish a correspondence between the sharpness of each grid image and its shooting height; A sharpness sequence is generated based on the correspondence.

2. The method according to claim 1, characterized in that, After adjusting the attitude of the placement platform of the measured plane using the parallel parameters, the method includes: Determine whether the parallel parameter is less than a preset parameter; If not, the parallel parameters of the measured plane after attitude adjustment are recalculated to iterate the parallel parameters until the parallel parameters are less than the preset parameters.

3. The method according to claim 1, characterized in that, After establishing the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set, the method further includes: Calculate the difference between the maximum and minimum focal lengths of the sharpness point set; Calculate the mean square error of all focus values ​​in all sharpness point sets; The data reliability of the sharpness point set is verified by the difference and the mean square error, and the verification result is obtained. When the verification result is unsuccessful, outliers in the sharpness point set are removed by a random sampling consensus algorithm, and the sharpness point set is updated.

4. The method according to claim 1, characterized in that, The calculation of the sharpness of each grid image in the grid image sequence includes: All images in the grid image sequence are divided into several sub-images according to the grid node coordinates; Calculate the sharpness of each sub-image; The average sharpness of all images is calculated based on the sharpness of the sub-images, using images as the unit. The sharpness of all images in the grid image sequence is determined by the average sharpness.

5. The method according to any one of claims 1 to 4, characterized in that, After establishing the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence to obtain the sharpness point set, the method further includes: Verify whether all sharpness points in the sharpness point set are within the height range of the axial scan; When a sharpness point exists outside the height range, the initial height of the axial scan is adjusted based on the absolute value of the difference between the sharpness point coordinates and the height range, and the image sequence is reacquired based on the adjusted initial height.

6. The method according to any one of claims 1 to 4, characterized in that, The method of adjusting the attitude of the placement platform of the measured plane using the parallel parameters includes: Obtain the pitch and roll angles from the parallel parameters; The pitch angle and the roll angle are respectively mapped to the placement platform of the measured plane, and the angle adjustment amount is calculated based on the mapping results; When the angle adjustment exceeds the preset range, the attitude of the placement platform is adjusted according to the pitch angle and / or roll angle.

7. A multi-point focusing fitting microscopic imaging plane leveling system, characterized in that, The system includes: The first establishment unit is used to establish a two-dimensional sampling grid based on the measured plane and determine the coordinates of the grid nodes; The second establishment unit is used to establish a spatial coordinate system and call up calibration data to obtain the calibration image plane; The acquisition unit is used to control the imaging module to perform axial scanning perpendicular to the plane under test, and continuously acquire image sequences covering the two-dimensional sampling grid. A partitioning unit is used to partition the image sequence according to the grid node coordinates to obtain a grid image sequence; The first calculation unit is used to calculate the sharpness of the grid image sequence using a no-reference sharpness evaluation method to obtain a sharpness sequence; The third establishment unit is used to establish the index relationship between the grid node coordinates and the corresponding maximum sharpness value in the sharpness sequence, so as to obtain the sharpness point set; A fitting unit is used to perform plane fitting based on the sharpness point set to obtain a global focal plane model; The second calculation unit is used to calculate the parallelism parameters of the global focal plane relative to the calibration image plane based on the global focal plane model. The first adjustment unit is used to adjust the attitude of the placement platform of the measured plane using the parallel parameters; The first computing unit is specifically used for: Calculate the sharpness of each grid image in the grid image sequence; Using grid images as an index, establish a correspondence between the sharpness of each grid image and its shooting height; A sharpness sequence is generated based on the correspondence.

8. A microscopic imaging plane leveling device for multi-point focusing fitting, characterized in that, The device includes: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, which the processor invokes to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a program that, when executed on a computer, performs the method as described in any one of claims 1 to 6.

Images