A dynamic focusing method, system and medium based on spot images

Abstract

The application provides a dynamic focusing method and system based on a light spot image and a medium. The method comprises the following steps: focusing on any surface of a to-be-measured object to obtain a calibration image; calculating the centroid of the light spot of the focusing surface of the to-be-measured object in the calibration image as a zero point; obtaining a corresponding pixel row-pixel value curve diagram of the light spot image obtained by processing the surface of the to-be-measured object; screening a split point by analyzing the peak point and the valley point of the pixel row-pixel value curve diagram, and calculating the corresponding light spot centroid on the left and / or right side of the split point corresponding to the target layer of the to-be-measured object to obtain a defocus amount; and analyzing whether the defocus amount falls into a control range to realize dynamic focusing. The dynamic focusing method is simple to calculate and is convenient for hardware implementation. Meanwhile, the application also simplifies the calculation of the split point to improve the calculation efficiency, and confirms the split point to ensure the accuracy of dynamic focusing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a dynamic focusing method and system based on laser spot image and a medium. BACKGROUND

[0002] Optical microscope magnifies the image of the measured object surface by lens imaging. In order to obtain meaningful images, the microscope objective must be accurately focused on the sample surface, and the depth of field of the high-power microscope objective is generally only a few microns. The operator of the microscope objective often needs to spend a lot of time manually adjusting the distance between the objective and the measured object to achieve focusing. Microscopic automatic focusing technology calculates the current defocus amount of the objective through a feedback signal and converts it into a motor movement signal, and the focusing process is automated by moving the objective through the motor.

[0003] According to the type of feedback signal, the microscopic automatic focusing technology can be divided into image-based automatic focusing technology and laser-based automatic focusing technology. The image-based automatic focusing technology uses an image sharpness evaluation function to determine the current defocus amount with a quantitative sharpness value. The judgment result of this technology depends on the state of the collected image and is easily affected by the lighting environment. Moreover, the focusing speed is slow and the focusing range is small.

[0004] The laser-based automatic focusing technology adds a laser signal to the ordinary microscope. The defocus amount of the measured object is obtained by calculating the shape of the laser spot (including the centroid, radius and curvature, etc.), and the moving direction and distance of the objective are controlled according to the defocus amount, so that the distance between the microscope objective and the measured object surface is within the depth of field of the objective, thereby realizing the automatic focusing of the microscope objective. Therefore, this technology has a larger focusing range and faster focusing speed, and is widely used in industrial detection field.

[0005] If the measured object has a transparent surface and the transparent surface is the detection target, the laser image obtained by using the laser-based automatic technology will have two layers of spots, corresponding to the upper surface and the lower surface of the measured object. For a lens with fixed focal length, the change of the distance between the focal point position and the imaging plane causes the change of the spot shape. Since the moving direction and the required movement amount to focus on the upper surface or the lower surface cannot be determined, it is impossible to determine whether the focusing is accurate. In the existing focusing method, the centroid of the spot in the laser image is continuously calculated to determine whether the focusing position is accurate. However, this method increases the workload of focusing, and cannot accurately calculate the adjustment distance to focus on the upper surface or the lower surface according to the distribution of the spot in the segmentation point and the energy distribution diagram.

[0006] Chinese patent CN 113267884A discloses a multi-layer automatic focusing method and system. By adding a semicircular diaphragm in the light beam, the focused light spots on the detector plane are distributed on both sides of the detector according to the focusing position. The focusing position of the objective lens can be determined according to the light spots. An evaluation function related to the intensity and size diameter of the light spot is also proposed, which can accurately determine the moving direction, determine the moving step, and judge the focusing accuracy to achieve the focusing of the target object on multiple planes. The technical means used in this patent are different from those in this scheme.

[0007] Chinese patent CN 107076964A discloses an image-based laser automatic focusing system. This patent provides an accurate, reliable and robust laser-based automatic focusing solution for lens-based microscope applications using slides or microtiter plates. The fluorescence characteristics of biological samples and the peak values of focused image projections are comprehensively utilized to identify the number and position of multiple laser reflections, and then the relative size, intensity and position of each spot are measured to estimate the z position of the objective lens relative to the inner surface. The technical means used in this patent are different from those in this scheme.

[0008] In addition, for automatic focusing in the case of relative movement between the microscope objective and the object to be measured, Chinese patent CN 110716301B proposes a technical solution that uses a Z-direction motion device and a stylus to move the slide to the best focus point; Chinese patent CN 106019550B proposes a technical solution that controls the state change of a liquid lens to realize the coordinated movement of the focusing tracking execution module; Chinese patent CN 119758576A determines the slide focusing reference point of each device by vacuum adsorbing a calibration sheet, greatly reducing the focusing range in the automatic focusing process.

[0009] In summary, the technical means used in the prior art for dynamic focusing are different from those in this scheme. SUMMARY

[0010] The present application proposes a dynamic focusing method, system and medium based on laser spot image, which at least solves one of the above technical problems.

[0011] To achieve the above purpose, the present application proposes the following technical scheme:

[0012] A dynamic focusing method based on spot image, comprising:

[0013] Focus on any surface of the object to be measured to obtain a calibration image; calculate the centroid of the spot on the focusing surface of the object to be measured in the calibration image as the zero point of the several spot images reflected by the surface of the object to be measured, for distinguishing the upper surface spot and the lower surface spot of the object to be measured;

[0014] Obtaining a light spot image corresponding to any position on the surface of the object to be measured; wherein the upper surface of the object to be measured is a transparent surface;

[0015] Calculating a pixel row-pixel value curve corresponding to the light spot image of the current position, screening peak points and valley points of the current curve;

[0016] Calculating the absolute difference between the pixel value of each valley point in the current curve and the maximum pixel value on its two sides, and taking the valley point corresponding to the maximum sum of the absolute difference on the two sides as the segmentation point; based on the target layer and the zero point, selecting the curve on the side corresponding to the segmentation point, and calculating the light spot centroid of the target layer to calculate the defocus amount between the current position of the object to be measured and the target layer;

[0017] If the defocus amount falls within the control range, automatic focusing is realized; otherwise, based on the defocus amount, a light spot image of the next position of the object to be measured is obtained, a new defocus amount between the next position and the target layer is calculated, and the process is repeated until the new defocus amount falls within the control range.

[0018] Further, it also includes: screening the first valley point on the right side of each peak point in the current curve as the first valley point array according to the abscissa of the peak points and the valley points in the current curve; and determining the segmentation point by calculating the absolute difference on the two sides of the valley points in the first valley point array.

[0019] Further, before calculating the light spot centroid of the target layer, it also includes: calculating the contrast on the side corresponding to the segmentation point based on the pixel value of the segmentation point and the maximum pixel value on its corresponding side; and determining the curve corresponding to the light spot centroid used to calculate the light spot centroid of the target layer according to the size relationship between the contrast and the contrast threshold.

[0020] Further, it also includes:

[0021] If the contrast on either side of the segmentation point is less than the contrast threshold, it indicates that the number of light spots in the light spot image is one, and then the centroid is calculated based on the pixel row-pixel value curve corresponding to the light spot image of the current position; otherwise, it indicates that the number of light spots in the light spot image is two, and then the centroid is calculated based on the target layer and the pixel row-pixel value curve on the left or right side of the segmentation point.

[0022] Further, screening the peak points and the valley points of the current curve includes: calculating the gradient of each point in the current curve; if the product of the gradient of the current point and the gradient of the previous point is not greater than zero, and the gradient of the current point is less than zero or the gradient of the previous point is greater than zero, then the current point is a peak point; if the product of the gradient of the current point and the gradient of the previous point is not greater than zero, and the gradient of the current point is greater than zero or the gradient of the previous point is less than zero, then the current point is a valley point.

[0023] Further, the pixel row-pixel value curve includes a first curve, wherein the horizontal coordinate of the first curve is the row coordinate of the light spot image, and the vertical coordinate of the first curve is the pixel mean value or the pixel sum of each row of pixels of the light spot image.

[0024] Further, the pixel row-pixel value curve includes a second curve, wherein the horizontal coordinate of the second curve is the row coordinate of the light spot image, and the pixel mean value or the pixel sum of each row of pixels of the light spot image is calculated as the vertical coordinate of the second curve after the pixel value of the light spot image below the gray threshold is set to zero.

[0025] Based on the same inventive concept, the application further provides a dynamic focusing system based on a light spot image, comprising:

[0026] The calibration module focuses on any surface of the object to be measured to obtain a calibration image, calculates the light spot centroid of the focusing surface of the object to be measured in the calibration image as the zero point of the light spot images formed by the reflection of the surface of the object to be measured, and is used to distinguish the light spots on the upper and lower surfaces of the object to be measured.

[0027] The preprocessing module obtains the light spot image corresponding to any position on the surface of the object to be measured, wherein the upper surface of the object to be measured is a transparent surface, calculates the pixel row-pixel value curve corresponding to the light spot image at the current position, and screens the peak points and valley points of the current curve.

[0028] The segmentation point determination module calculates the absolute difference between the pixel value of each valley point in the current curve and the maximum pixel value on both sides of the valley point, takes the valley point corresponding to the maximum sum of the absolute difference on both sides as the segmentation point, and calculates the light spot centroid of the target layer based on the segmentation point.

[0029] The focusing module calculates the light spot centroid of the target layer based on the segmentation point, calculates the defocus amount of the current position of the object to be measured and the target layer according to the zero point, and realizes automatic focusing if the defocus amount falls within the control range; otherwise, the light spot image of the next position of the object to be measured is obtained based on the defocus amount, a new defocus amount of the next position and the target layer is calculated, and the process is repeated until the new defocus amount falls within the control range.

[0030] Further, the segmentation point determination module further includes: screening the first valley point on the right side of each peak point in the current curve as a first valley point array according to the horizontal coordinates of the peak points and the valley points in the current curve; and determining the segmentation point by calculating the absolute difference on both sides of the valley points in the first valley point array.

[0031] Further, the contrast calculation module calculates the contrast on the side corresponding to the segmentation point based on the pixel value of the segmentation point and the maximum pixel value on the side corresponding to the segmentation point, and determines the curve corresponding to the light spot centroid used to calculate the light spot centroid of the target layer according to the size relationship between the contrast and the contrast threshold.

[0032] In another aspect, the present application also provides a computer readable storage medium storing a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the method described above.

[0033] The present application has the following advantages:

[0034] The present application provides a dynamic focusing method based on a laser-based microscopic automatic focusing technology, wherein a calibration image is obtained by pre-focusing, and the centroid coordinates calculated from the calibration image are used as the zero points for distinguishing the upper and lower surfaces of the object to be measured; then the centroid of the light spot image reflected by the surface of the object to be measured at any position is calculated to obtain the defocus amount, so as to realize the focusing control of the surface of the object to be measured at the next position and realize the dynamic focusing process of the surface of the object to be measured. The dynamic focusing method provided by the present application is simple to operate and easy to realize.

[0035] In the present application, the pixel row-pixel value curve of the light spot image obtained by processing the surface of the object to be measured is obtained; the peak points and valley points of the pixel row-pixel value curve are analyzed, and the split points are screened; the pixel row-pixel value curve on the left and / or right of the split point corresponding to the target layer of the object to be measured is used to calculate the corresponding light spot centroid, so as to obtain the defocus amount; and the dynamic focusing is realized by analyzing whether the defocus amount falls within the control range. The method provided by the present application is simple to calculate and easy to realize by hardware.

[0036] In the present application, for the case that there are multiple peak points and valley points in the pixel row-pixel value curve, the gradient graph corresponding to the pixel row-pixel value curve is used to screen the peak points and valley points; and for the case that there are multiple continuous valley points and peak points with similar pixel values, the cross comparison of the abscissa of the screened peak points and valley points is used to simplify the peak points and valley points, so as to improve the calculation efficiency of the split points.

[0037] In addition, the present application also provides a process for confirming the split points, wherein the contrast on the left and right of the split points is calculated, the number of light spots of the light spot image is judged according to the size relationship between the contrast and the contrast threshold, so as to determine the curve corresponding to the light spot centroid used for calculating the target layer; and the split effect of the split point is determined according to the calculation result of the contrast, so as to improve the accuracy of the dynamic focusing. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is the optical principle diagram of the laser-based microscopic automatic focusing technology;

[0039] Figure 2 is the flowchart of the dynamic focusing method provided by the present application;

[0040] Figure 3 is the light spot image obtained by the present application;

[0041] Figure 4 is a curve graph of the first curve in the embodiment 1 of the present application;

[0042] Figure 5 is a flow chart of the dynamic focusing method in the embodiment 2 of the present application;

[0043] Figure 6 is a flow chart of the dynamic focusing method in the embodiment 3 of the present application;

[0044] Figure 7 is an example of the dynamic focusing system based on the light spot image proposed by the present application;

[0045] Figure 8 is another example of the dynamic focusing system based on the light spot image proposed by the present application;

[0046] In the figure: 1-laser unit; 2-cylindrical lens; 3-baffle; 4-reflector; 5-first beam splitter; 6-second beam splitter; 7-microscope objective; 8-measured object; 9-focusing lens; 10-illumination light source; 11-third beam splitter; 12-tube lens. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, but not all the embodiments.

[0048] As shown in the figure, the optical principle of the laser-based microscopic automatic focusing technology is as follows: Figure 1

[0049] The laser unit 1 emits a parallel laser beam, which is modulated by the cylindrical lens 2 into an asymmetric beam that diverges in the curvature direction of the cylindrical lens 2 and is collimated in the non-curvature direction. The asymmetric beam loses half of the energy through the baffle 3 and only propagates on one side of the optical axis. After being reflected by the reflector 4, the first beam splitter 5 and the second beam splitter 6, the laser beam enters the microscope objective 7 and converges to the surface of the measured object 8 through the microscope objective 7.

[0050] The laser beam reflected by the surface of the measured object 8 passes through the microscope objective 7, the second beam splitter 6 and the first beam splitter 5, and converges to the surface of the first sensor through the focusing lens 9, forming a laser spot; the spot image reflected by the surface of the measured object is collected by the first sensor.

[0051] The image processing unit calculates the defocus amount according to the shape of the laser spot in the spot image, and converts the defocus amount into a control signal of the driving unit, so that the driving unit drives the microscope objective 7 to move, realizing automatic focusing. ​

[0052] The illumination light source 10 forms a coaxial illumination light path with the third beam splitter 11, the second beam splitter 6, and the microscope objective 7. The tube lens 12, the second sensor, and the microscope objective 7 form an imaging light path. The second sensor is used to collect the surface image of the measured object 8 after focusing.

[0053] Preferably, the laser unit 1 can be a line laser or a point laser. If it is a point laser, a collimating lens needs to be configured to obtain a parallel light beam to ensure that the laser unit emits a parallel laser beam. If multiple point lasers are used, preferably, the positions of the light outlets of the multiple point lasers are arranged in a straight line in the horizontal direction or the vertical direction in the horizontal plane.

[0054] Preferably, the driving unit can be a motor, a motor, or the like driving device to realize precise movement control of the microscope objective 7.

[0055] Based on the optical principle of the laser-based microscopic automatic focusing technology, the embodiment proposes a dynamic focusing method based on a light spot image, which is suitable for the above-mentioned double-sensor microscopic automatic focusing system to solve the automatic focusing problem in the application scenario where the relative motion occurs between the microscope objective and the measured object. Embodiment 1

[0056] As shown in Figure 2 The embodiment proposes a dynamic focusing method based on a light spot image, which includes the following steps:

[0057] S0 focuses on any surface of the measured object to obtain a calibration image; the centroid of the light spot of the focusing surface of the measured object in the calibration image is calculated as the zero point of the light spot images formed by the reflection of the surface of the measured object, which is used to distinguish the upper surface light spot and the lower surface light spot of the measured object.

[0058] Preferably, before obtaining the light spot image in the moving scenario, the surface of the measured object needs to be manually calibrated. Any surface of the measured object is focused, and the light spot image of the surface (the focusing surface of the measured object) is collected by using the first sensor, that is, the calibration image. The centroid of the light spot in the calibration image is used as the zero point of the light spot images formed by the reflection of the surface of the measured object in the subsequent moving scenario, which is used to distinguish the upper surface light spot and the lower surface light spot of the measured object.

[0059] In this embodiment, if the upper surface of the measured object is the focusing surface, the light spot of the focusing surface of the measured object is above the light spot of the non-focusing surface in the calibration image. Preferably, the position of the pixel where the centroid is located can be defined as zero, and the direction of the light spot of the focusing surface is recorded as up, and the direction of the light spot of the non-focusing surface is recorded as down. Here, the up and down directions are only used to represent the relative position relationship.

[0060] Preferably, if the lower surface of the object to be measured is the focusing surface, the focusing surface spot of the object to be measured in the calibration image is below the non-focusing surface spot, at this time, the position of the pixel where the centroid is defined is zero, and the direction of the focusing surface spot is down and the direction of the non-focusing surface spot is up.

[0061] S1 obtains a light spot image corresponding to any position on the surface of the object to be measured; wherein the upper surface of the object to be measured is a transparent surface.

[0062] The object to be measured can have multiple transparent surfaces, and the upper surface must be a transparent surface. The object to be measured can be an object such as glass, wafer, etc. with at least a transparent upper surface.

[0063] The upper and lower surfaces of the object to be measured are planes. The upper surface of the object to be measured is located below the microscope objective, and the upper surface of the object to be measured is parallel to the microscope objective.

[0064] If the object to be measured is placed at an angle or the upper and lower surfaces of the object to be measured have a certain angle of inclination, the angles of inclination of the upper and lower surfaces can be the same or different. By adjusting the inclination of the surface of the object to be measured or the angle of the focusing spot of the laser light path design adjustment, the plane where the microscope objective is located and the adjusted upper surface of the object to be measured remain relatively parallel.

[0065] As shown in Figure 3 affected by the material, reflectivity and distance of the surface of the object to be measured, the laser image (light spot image) can capture the light spots on the upper and lower surfaces of the object to be measured, so that multiple light spots appear in the laser image (light spot image). The size and position of the light spot are affected by the focal point position, and the light spot changes with the change of the focal point position during the focusing process, and the focal point position changes with the change of the position of the microscope objective along the objective axis.

[0066] Preferably, the microscope objective and the object to be measured maintain a certain speed of relative motion in the embodiment, that is, the multiple light spot images of the surface of the object to be measured obtained by the second sensor correspond to different positions on the surface of the object to be measured.

[0067] Preferably, in order to satisfy the real-time automatic focusing of the surface of the object to be measured, the speed of the above relative motion and the collection speed of the second sensor have the following relationship: the defocus amount calculated by the light spot image corresponding to the previous position, the adjusted microscope automatic focusing system, and the light spot image of the next position.

[0068] S2 calculates the pixel row-pixel value curve corresponding to the light spot image of the current position, and screens the peak points and valley points of the current curve.

[0069] According to the light spot image obtained by the present application, the accurate light spot of the single layer surface cannot be obtained, and thus the defocus amount of the focusing target layer cannot be determined. According to the prior art, as the distance between the microscopic objective lens and the ideal focal plane changes, the shape of the light spot changes, and thus in the light spot images at different distances between the microscopic objective lens and the ideal focal plane, the centroid position of the light spot moves up and down (i.e., the row coordinates of the light spot centroid are different in the light spot images at different distances) as the position of the microscopic objective lens along the objective lens axis changes, and thus only the light spot centroid of different pixel rows in the light spot image needs to be calculated to obtain the defocus amount of the target layer.

[0070] For the pixel row-pixel value curve corresponding to the light spot image, the present embodiment provides the following two calculation methods:

[0071] The pixel row-pixel value curve includes a first curve, wherein the horizontal coordinate of the first curve is the row coordinate of the light spot image, and the vertical coordinate of the first curve is the pixel mean value or the pixel sum of each row of pixels of the light spot image.

[0072] The pixel row-pixel value curve includes a second curve, wherein the horizontal coordinate of the second curve is the row coordinate of the light spot image; the pixel value of the pixel lower than the gray threshold value in the light spot image is set to zero, and then the pixel mean value or the pixel sum of each row is calculated as the vertical coordinate of the second curve.

[0073] Preferably, in the pixel row-pixel value curve corresponding to the light spot image, the vertical coordinate can also be the pixel mean value of part of the row pixels. The screening method of the part of the row pixels is subject to actual application, and the present embodiment does not limit the specific screening method.

[0074] According to the data information of the light spot image obtained in the current scene, the first curve or the second curve is selected to calculate the light spot centroid. Preferably, the two curves can also be combined for use to calculate the light spot centroid of the light spot image.

[0075] In the present embodiment, the first curve is selected to calculate the division point, and the second curve is selected to calculate the centroid to improve the accuracy of the centroid calculation. In the present embodiment, the vertical coordinates of the first curve and the second curve are both pixel mean values.

[0076] Preferably, the mathematical expressions of the first curve f1(i) and the second curve f2(i) provided by the present embodiment are as follows:

[0077]

[0078] wherein the light spot image is I(j, i), i represents the row coordinate of any pixel in the light spot image, j represents the column coordinate of any pixel in the light spot image, w is the width of the light spot image, and h is the height of the light spot image.

[0079] In the embodiment, the second curve suppresses the pixel points in the light spot image with low gray value, and suppresses the pixel value lower than the gray threshold Thr to 0, so as to improve the accuracy of the calculation of the centroid. The mathematical expression is as follows:

[0080]

[0081] In the embodiment, the first curve corresponding to the current position light spot image is as shown in the following figure: Figure 4 In the figure, the horizontal coordinate i is the row coordinate of the light spot image, and the vertical coordinate f1(i) is the average pixel value of each row of pixels in the light spot image. In the figure, i = 1, 2,..., h.

[0082] In the pixel row-pixel value curve diagram, if the segmentation point is known, the left side of the segmentation point is the light spot pixel distribution on the upper surface of the object to be measured, and the right side is the light spot pixel distribution on the lower surface of the object to be measured. Then, the centroids of the upper and lower surfaces can be calculated according to the pixel distributions of the light spots on the upper and lower surfaces, respectively.

[0083] Generally, in the first curve diagram or the second curve diagram, the difference between the pixel values of any three consecutive points is used to determine the peak point and the valley point, that is, the gradient distribution of the current point (such as calculating the difference between the pixel values of the current point and the previous point and the next point) is used to determine whether the current point is a peak point or a valley point.

[0084] Preferably, the step value of the pixel point can also be set, that is, the peak point and the valley point are not determined by consecutive pixel points, such as the previous point and the next point being separated from the current point by a pixel point, a ≥ 1, a ∈ N + ; and the gradient distribution of the current point is calculated by selecting part of the pixel points.

[0085] In addition, in addition to the simple difference determination, the calculation results of the product or the ratio can also be used to determine whether the current point is a peak point or a valley point.

[0086] In the embodiment, the first curve diagram is taken as an example, the difference between the pixel values of any two consecutive points in the first curve diagram is used to calculate the gradient diagram corresponding to the first curve diagram, grad f1(i) = f1(i+1) - f1(i), i = 1, 2,..., h-1. The product of any two consecutive points in the gradient diagram and the size relationship between the selected two consecutive points and zero are used to determine whether the previous point in the two consecutive points is a peak point or a valley point. Each point is sequentially determined whether it is a valley point or a peak point by traversing the gradient diagram.

[0087] Specifically, the gradient of each point in the current curve diagram is calculated; if the product of the gradient of the current point and the gradient of the previous point is not greater than zero, and the gradient of the current point is less than zero or the gradient of the previous point is greater than zero, the current point is a peak point; if the product of the gradient of the current point and the gradient of the previous point is not greater than zero, and the gradient of the current point is greater than zero or the gradient of the previous point is less than zero, the current point is a valley point.

[0088] In summary, if grad f1(i-1) x grad f1(i)≤0, it indicates that i can be a peak point or a valley point;

[0089] If grad f1(i-1)>0 or grad f1(i)<0, i is a peak point;

[0090] If grad f1(i-1)<0 or grad f1(i)>0, i is a valley point.

[0091] For subsequent calculation, all the peak points selected from the first curve are recorded as a first peak point coordinate array T1, and all the valley points are recorded as a first valley point coordinate array V1 in this embodiment.

[0092] Since the light spot image obtained on the surface of the measured object has light spots on the upper and lower surfaces, and the focusing position in the double-layer sample (measured object) and the energy distribution change continuously during the focusing process, multiple peak points and valley points appear in the pixel row-pixel value curve graph.

[0093] According to the distribution characteristics of the pixel row-pixel value curve graph, the division point is one of the valley points; therefore, it is necessary to select the division point from multiple valley points, on the one hand to exclude interference data, and on the other hand, according to the selected division point, the distance from the division point to the upper surface or the lower surface of the measured object can be calculated.

[0094] S3 calculates the absolute difference between the pixel value of each valley point in the current curve graph and the maximum pixel value on both sides, and takes the valley point corresponding to the maximum value of the sum of the absolute difference on both sides as the division point.

[0095] In this embodiment, taking the first curve as an example, the pixel value of each valley point and the pixel value of each peak point on the left and right sides of the valley point are calculated, respectively, the maximum value of the absolute difference on the left and right sides of the valley point is selected, respectively, the maximum value of the absolute difference on the left side of the valley point is added to the maximum value of the absolute difference on the right side, and the sum of the absolute difference on both sides of the valley point is taken as the sum of the absolute difference on both sides.

[0096] Each valley point has a corresponding sum of the absolute difference on both sides, and the valley point corresponding to the maximum value of the sum of the absolute difference on both sides is selected as the division point.

[0097] S4 selects the curve graph on the side corresponding to the division point based on the target layer and the zero point, and calculates the light spot centroid of the target layer to calculate the defocus amount of the current position of the measured object and the target layer.

[0098] The target layer is the upper surface or the lower surface of the measured object.

[0099] According to the relative direction of the upper surface light spot and the lower surface light spot of the object to be measured defined by the zero point in the present application, based on the selected target layer for focusing, the pixel row-pixel value curve graph on the left or right side of the segmentation point is selected to calculate the light spot centroid of the target layer.

[0100] In the present embodiment, if the upper surface of the object to be measured is defined as the focusing surface in the calibration image, the focusing surface light spot direction is defined as up, and the non-focusing surface light spot is defined as down. In the pixel row-pixel value curve graph, the data on the left side of the segmentation point corresponds to the light spot above the zero point, and the data on the right side of the segmentation point corresponds to the light spot below the zero point.

[0101] The abscissa of the segmentation point is i split If the target layer is the upper surface of the object to be measured, the value range of i in the calculation formula of the pixel row c corresponding to the centroid is [1, i split ];

[0102] If the target layer is the lower surface of the object to be measured, the value range of i in the calculation formula of the pixel row c corresponding to the centroid is [i split , h].

[0103] Taking the first curve as an example, the present embodiment proposes a centroid calculation formula, and the calculation formula of the pixel row c corresponding to the centroid is as follows:

[0104]

[0105] In the formula, the value range of i is related to the selected target layer.

[0106] Or taking the second curve, the calculation formula of the pixel row c corresponding to the centroid is as follows:

[0107]

[0108] In the formula, the value range of i is related to the selected target layer.

[0109] The defocus amount between the micro objective lens and the target layer of the object to be measured is obtained by subtracting the pixel row corresponding to the centroid of the data on the left and right sides of the segmentation point from the pixel row corresponding to the centroid, and then converting the pixel row to the actual distance. Then the defocus amount between the micro objective lens and the target layer of the object to be measured is obtained, and then the driving unit is controlled to drive the micro objective lens to move to realize automatic focusing.

[0110] Based on the pixel row-pixel value curve graph of the present embodiment, the interval pixel row for focusing to the target layer of the object to be measured is obtained by the abscissa of the segmentation point and the abscissa of the centroid of the data on the left and right sides of the segmentation point. The defocus amount between the micro objective lens and the target layer of the object to be measured is obtained by the ratio between the size of the interval pixel row in the light spot image and the magnification of the light spot image, and then the driving unit is controlled to drive the micro objective lens to move to realize automatic focusing.

[0111] Image magnification is the ratio between the image size and the actual size of the object in the image. It describes the degree to which the size of the object displayed in the image is magnified relative to its true size.

[0112] If the defocus amount falls within the control range, autofocus is achieved; otherwise, based on the defocus amount, the spot image of the next position of the object under test is obtained, and the new defocus amount between the next position and the target layer is calculated until the new defocus amount falls within the control range.

[0113] This embodiment determines whether the light spot image acquired at the current position is successfully focused by judging whether the calculated defocus amount falls within the control range. If it falls within the range, it indicates that the focus is successful, and at this time, a trigger signal can be sent to the first sensor to simultaneously acquire the image information of the object under test.

[0114] If the object is not within the control range, the drive unit is adjusted according to the current defocus amount, so that the second sensor can acquire the light spot image of the object surface at the next position in the adjusted state and perform dynamic autofocus.

[0115] In this embodiment, the centroid of the spot image at the current position is calculated to obtain the defocus amount between the microscope objective and the target layer of the test object at the current position, which is used to control the drive unit to move the microscope objective. Based on the adjusted distance between the microscope objective and the surface of the test object, the second sensor acquires the spot image of the test object surface at the next position, and then calculates the defocus amount at the new position to realize the dynamic focusing process. Example 2

[0116] like Figure 5 As shown, based on Example 1, in order to reduce the computational load of the above-mentioned segmentation point screening, before the segmentation point screening (S3), multiple consecutive valley points with similar pixel values ​​can be screened, and only one of the multiple consecutive valley points with similar pixel values ​​can be retained, so as to reduce the calculation of the absolute difference between the same peak point and multiple possible valley points with similar pixel values.

[0117] The core of filtering multiple consecutive valleys with similar pixel values ​​lies in determining whether there is a valley between any two consecutive peaks and the number of valleys based on the cross-coordinates of the peaks and valleys.

[0118] The simplification process for valley points includes: based on the horizontal coordinates of the peaks and valleys in the current curve, selecting the first valley point to the right of each peak point as the first valley point array; and using the valley points in the first valley point array, calculating the absolute difference between the two sides to determine the dividing point.

[0119] Based on the above valley point screening and simplification process, the segmentation points are selected from the simplified valley point set.

[0120] Since the simplification of the valley points involves comparison of each valley point with each peak point, while screening the valley points, a plurality of continuous peak points with similar pixel values can also be screened at the same time, and by retaining only one of the plurality of continuous peak points with similar pixel values, the calculation of the absolute difference between the same valley point and the plurality of peak points with similar pixel values that can exist can be reduced.

[0121] In the embodiment, all the peak points screened by the first curve are recorded as a first peak point coordinate array T1, and all the valley points are recorded as a first valley point coordinate array V1 in S2.

[0122] Taking the first curve as an example, the specific process of simultaneously simplifying the peak points and the valley points is as follows:

[0123] Valley point simplification process:

[0124] The horizontal coordinate T1(1) of the first peak point in T1 is compared with the horizontal coordinate V1(1) of the first valley point in V1; if T1(1)>V1(1), it indicates that the first valley point in V1 is on the left side of the first peak point in T1, and the horizontal coordinates of the next valley points are compared with T1(1) until the horizontal coordinate V1(α) of the αth (α>1) valley point in V1 is greater than T1(1), which indicates that the αth valley point in V1 is on the right side of the first peak point in T1 (i.e., the αth valley point in V1 is the first valley point on the right side of the first peak point in T1), and the αth valley point in V1 is recorded in the second valley point coordinate array V2.

[0125] Peak point simplification process:

[0126] The horizontal coordinate T1(2) of the second peak point in T1 is compared with the horizontal coordinate V1(α) of the αth valley point in V1; if T1(2)>V1(α), it indicates that the second peak point in T1 is on the right side of the αth valley point in V1, and the second peak point in T1 is recorded in the second peak point coordinate array T2.

[0127] If T1(2)<V1(α), the horizontal coordinates of the next peak points are compared with V1(α) until the horizontal coordinate T1(β) of the βth (β>2) peak point in T1 is greater than V1(α), and the βth peak point in T1 is recorded in the second peak point coordinate array T2.

[0128] The above-mentioned valley point simplification process and peak point simplification process are repeatedly crossed until the simplification judgment is performed on all the peak points and valley points in the first curve, and the second valley point coordinate array V2 and the second peak point coordinate array T2 are obtained.

[0129] Based on the above-mentioned valley point screening and simplification process, the division points are screened from the simplified second valley point coordinate array V2 and the second peak point coordinate array T2. Embodiment 3

[0130] As Figure 6As shown, based on the above embodiments, this embodiment also proposes a process for determining the segmentation point before calculating the centroid, so as to facilitate the centroid calculation.

[0131] Before calculating the centroid of the target layer, the following steps are also taken: based on the pixel value of the segmentation point and the maximum pixel value of its corresponding side, the contrast of the side corresponding to the segmentation point is calculated; according to the relationship between the contrast and the contrast threshold, the number of spots in the spot image is determined so as to determine the curve corresponding to the calculation of the centroid of the target layer.

[0132] The contrast ratio includes the ratio of the difference between the maximum pixel value on the corresponding side and the pixel value at the segmentation point to the sum of the maximum pixel value on the corresponding side and the pixel value at the segmentation point, which is used as the contrast ratio on the side corresponding to the segmentation point.

[0133] In this embodiment, the left-side contrast ratio of the segmentation point is given. left The calculation formula is as follows:

[0134]

[0135] In the formula, the x-coordinate of the dividing point is i split ,f(i split f(i) represents the pixel value corresponding to the dividing point in the pixel row-pixel value curve. split If ) can be the first curve, then it is f1(i split );f(i split If ) can be the second curve, then it is f2(i split );

[0136] f(i split,l,top f(i) represents the maximum pixel value to the left of the dividing point in the pixel row-pixel value curve. split,l,top If ) can be the first curve, then it is f1(i split,l,top );f(i split,l,top If ) can be the second curve, then it is f2(i split,l,top );

[0137] f(i split ) and f(i split,l,top The selected pixel row-pixel value curve is consistent.

[0138] If we calculate the contrast ratio to the right of the segmentation point right Then f(i) in the above formula split,l,top Replace ) with f(i split,r,top ), f(i split,r,top ) represents the maximum pixel value to the right of the dividing point in the pixel row-pixel value curve; similarly, f(i split ) and f(i split,r,top The selected pixel row-pixel value curve is consistent.

[0139] The range of the contrast ratio (including left contrast and right contrast) is [0, 1].

[0140] According to the contrast calculation formula, when f(i split ) is constant, the greater the maximum pixel value on the left or right side of the segmentation point, the greater the contrast. The greater the contrast, the better the segmentation effect of the segmentation point.

[0141] If the contrast on either side of the segmentation point is less than the contrast threshold, it indicates that the number of light spots in the light spot image is one, and the centroid is calculated based on the pixel row-pixel value curve corresponding to the light spot image at the current position. Otherwise, it indicates that the number of light spots in the light spot image is two, and the centroid is calculated based on the target layer, and the pixel row-pixel value curve on the left or right side of the segmentation point is selected.

[0142] Wherein, the number of light spots is one, indicating that the light spot on the upper surface of the measured object coincides with the light spot on the lower surface of the measured object.

[0143] As shown in Figure 7 , based on the same inventive concept, the present application proposes a dynamic focusing system based on light spot image to execute the method as described above, comprising:

[0144] The calibration module focuses on any surface of the measured object to obtain a calibration image, and calculates the light spot centroid of the focusing surface of the measured object in the calibration image as the zero point of several light spot images formed by the reflection of the surface of the measured object, which is used to distinguish the light spots on the upper and lower surfaces of the measured object.

[0145] The preprocessing module obtains the light spot image corresponding to any position on the surface of the measured object, wherein the upper surface of the measured object is a transparent surface, and calculates the pixel row-pixel value curve corresponding to the light spot image at the current position, and screens the peak points and valley points of the current curve.

[0146] The segmentation point determination module calculates the absolute difference between the pixel value of each valley point in the current curve and the maximum pixel value on both sides, and takes the valley point corresponding to the maximum sum of the absolute difference on both sides as the segmentation point.

[0147] The focusing module selects the curve on the side corresponding to the segmentation point based on the target layer, calculates the light spot centroid of the target layer, and calculates the defocus amount of the current position of the measured object from the target layer based on the zero point. If the defocus amount falls within the control range, automatic focusing is achieved. Otherwise, based on the defocus amount, the light spot image at the next position of the measured object is obtained, the new defocus amount of the next position from the target layer is calculated, and the process is repeated until the new defocus amount falls within the control range.

[0148] Further, the split point determination module further comprises: screening the first valley point on the right side of each peak point as the first valley point array according to the abscissa of the peak point and the valley point in the current curve graph; and determining the split point by calculating the absolute difference value on both sides of the valley point in the first valley point array.

[0149] As shown in Figure 8 Further, the contrast calculation module is further configured to calculate the contrast on the side corresponding to the split point based on the pixel value of the split point and the maximum pixel value on the corresponding side; and determine the number of light spots of the light spot image according to the size relationship between the contrast and the contrast threshold, so as to determine the curve graph corresponding to the light spot centroid used for calculating the target layer.

[0150] Based on the same inventive concept, the present application further provides a computer readable storage medium comprising a computer program, wherein the computer program is executed by a processor to implement the method described above.

[0151] The technical features of the above embodiments can be combined in any manner. To make the description concise, all possible combinations of the technical features in the above embodiments are not described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0152] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A dynamic focusing method based on spot images, characterized in that, include: Focus on any surface of the object under test and acquire a calibration image; Calculate the centroid of the light spot on the focal surface of the test object in the calibration image, and use it as the zero point of several light spot images formed by reflection from the surface of the test object, which is used to distinguish the light spot on the upper surface of the test object from the light spot on the lower surface of the test object. Acquire a spot image corresponding to any position on the surface of the object under test; wherein the upper surface of the object under test is a transparent surface; Calculate the pixel row-pixel value curve corresponding to the spot image at the current location, and filter the peaks and valleys of the current curve. Calculate the absolute difference between the pixel value of each valley point in the current curve and the maximum pixel value on both sides. The valley point corresponding to the maximum sum of the absolute differences on both sides is used as the segmentation point. Based on the target layer and the zero point, select the curve on the side corresponding to the segmentation point and calculate the centroid of the spot in the target layer to calculate the defocus amount between the current position of the object under test and the target layer. If the defocus amount falls within the control range, autofocus is achieved; otherwise, based on the defocus amount, the spot image of the next position of the object under test is obtained, and the new defocus amount between the next position and the target layer is calculated until the new defocus amount falls within the control range.

2. The dynamic focusing method based on spot images according to claim 1, characterized in that, Also includes: Based on the x-coordinates of the peaks and valleys in the current curve, filter the first valley to the right of each peak and use it as the first valley array. Calculate the absolute difference between the two sides of each valley point in the first valley point array to determine the dividing point.

3. The dynamic focusing method based on spot images according to claim 1, characterized in that, Before calculating the centroid of the target layer, the following steps are also taken: based on the pixel value of the segmentation point and the maximum pixel value of its corresponding side, the contrast of the side corresponding to the segmentation point is calculated; according to the relationship between the contrast and the contrast threshold, the number of spots in the spot image is determined so as to determine the curve corresponding to the calculation of the centroid of the target layer.

4. The dynamic focusing method based on spot images according to claim 3, characterized in that, Also includes: If the contrast on either side of the segmentation point is less than the contrast threshold, it indicates that there is one spot in the spot image. In this case, the centroid is calculated using the pixel row-pixel value curve corresponding to the spot image at the current position. Otherwise, it indicates that there are two spots in the spot image. In this case, the centroid is calculated using the pixel row-pixel value curve to the left or right of the segmentation point, depending on the target layer.

5. The dynamic focusing method based on spot images according to claim 1, characterized in that, Filtering peaks and valleys in the current curve includes: calculating the gradient of each point in the current curve; if the product of the gradient of the current point and the gradient of the previous point is not greater than zero, and the gradient of the current point is less than zero or the gradient of the previous point is greater than zero, then the current point is a peak; if the product of the gradient of the current point and the gradient of the previous point is not greater than zero, and the gradient of the current point is greater than zero or the gradient of the previous point is less than zero, then the current point is a valley.

6. The dynamic focusing method based on spot images according to claim 1, characterized in that, The pixel row-pixel value curve includes: a first curve; wherein the horizontal axis of the first curve is the row coordinate of the spot image, and the vertical axis of the first curve is the pixel mean or pixel sum of each row of pixels in the spot image.

7. The dynamic focusing method based on spot images according to claim 1, characterized in that, The pixel row-pixel value curve includes: a second curve; wherein the horizontal axis of the second curve is the row coordinate of the spot image; the pixel values ​​in the spot image below the grayscale threshold are set to zero, and the average or total pixel value of each row is calculated as the vertical axis of the second curve.

8. A dynamic focusing system based on spot images, characterized in that, include: The calibration module focuses on any surface of the object under test and acquires a calibration image. Calculate the centroid of the light spot on the focal surface of the object under test in the calibration image, and use it as the zero point of several light spot images formed by reflection from the surface of the object under test, to distinguish the light spots on the upper and lower surfaces of the object under test; The preprocessing module acquires a spot image corresponding to any position on the surface of the object under test; wherein the upper surface of the object under test is a transparent surface; calculates the pixel row-pixel value curve corresponding to the spot image at the current position, and filters the peaks and valleys of the current curve. The segmentation point determination module calculates the absolute difference between the pixel value of each valley point in the current curve graph and the maximum pixel value on both sides, and takes the valley point corresponding to the maximum sum of the absolute differences on both sides as the segmentation point; The focusing module, based on the target layer, selects the curve corresponding to the segmentation point, calculates the centroid of the light spot in the target layer, and calculates the defocus amount between the current position of the test object and the target layer based on the zero point. If the defocus amount falls within the control range, automatic focusing is achieved; otherwise, based on the defocus amount, the light spot image of the next position of the test object is obtained, and the new defocus amount between the next position and the target layer is calculated until the new defocus amount falls within the control range.

9. The dynamic focusing system based on spot images according to claim 8, characterized in that, The segmentation point determination module also includes: based on the horizontal coordinates of the peaks and valleys in the current curve, filtering the first valley point to the right of each peak point as the first valley point array; and using the valley points in the first valley point array, calculating the absolute difference between the two sides to determine the segmentation point.

10. The dynamic focusing system based on spot images according to claim 8, characterized in that, Also includes: The contrast calculation module calculates the contrast of the side corresponding to the segmentation point based on the pixel value of the segmentation point and the maximum pixel value of the corresponding side. Based on the relationship between contrast and contrast threshold, the number of spots in the spot image is determined to identify the curve corresponding to the centroid of the spot used to calculate the target layer.

Images