Automatic focal length tracking device and method of microscopic scanning detection system

Abstract

The invention belongs to the technical field of microscopic focusing, and relates to an automatic focal length tracking device of a microscopic scanning detection system, which comprises a main camera, a horizontal camera and a spectroscope, the spectroscope is mounted in a microscope barrel, is a semi-transparent and semi-reflective optical plane mirror and is used for dividing imaging light transmitted by an objective lens into two paths, and the two paths of imaging light are respectively transmitted to the main camera and the horizontal camera; an imaging target surface of the main camera is vertical to a transmission imaging optical axis received by the main camera; an included angle alpha is formed between an imaging target surface of the horizontal camera and a reflected imaging optical axis received by the imaging target surface, and alpha is not equal to 90 degrees; the control module calculates a definition function and outputs a compensation instruction of the main camera according to imaging data of the horizontal camera.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of microscopic focusing technology, and relates to an automatic focal length tracking device and method of a microscopic scanning detection system. BACKGROUND

[0002] In a microscopic scanning detection system, a main camera is usually used to acquire a detection image of a sample under test. However, in a long-time or large-area scanning detection process, due to unevenness of the sample surface, limited platform motion accuracy, temperature drift or vibration and other environmental factor disturbances, the object distance between the main camera and the sample will be slightly offset, resulting in unstable focal length and causing the collected image to have reduced clarity or even be blurred. This problem is particularly prominent in scenarios such as semiconductor wafer detection, pathological section scanning and high-resolution material characterization, which require long-time continuous imaging, and directly affects the accuracy and efficiency of detection.

[0003] Existing automatic focusing or focal length tracking schemes mostly rely on single-camera frame-by-frame calculation of a sharpness evaluation function, or scanning at multiple focal point positions through mechanical stepping, to find the best focal point. Although this kind of method can achieve automatic focusing to some extent, it still has obvious deficiencies: first, real-time performance is poor, single-camera imaging needs to occupy a large number of frame rates for sharpness calculation, feedback is lagged, and cannot meet the needs of high-speed scanning; second, precision is insufficient, single-camera calculation is easily affected by noise interference and contrast fluctuation, resulting in unstable focusing determination and appearing of false compensation or frequent jitter; third, integration is complex, some systems try to increase laser ranging modules or multi-dimensional scanning mechanisms to improve precision, but device cost and maintenance cost are increased, which limits its popularization in industrial application. As can be seen, the existing technology cannot balance real-time performance, stability and system integration, and a new automatic focal length tracking scheme still needs to be proposed for improvement. SUMMARY

[0004] The present application aims at the deficiencies of the prior art, and provides an automatic focal length tracking device and method of a microscopic scanning detection system. By setting a beam splitter in the microscope light path and simultaneously configuring a main camera and a horizontal camera, the horizontal camera is used to acquire an out-of-focus distribution image at a high frame rate, a sharpness function determination is combined, a compensation instruction is output in real time, and automatic focal length tracking of the main camera in the scanning detection process is realized, so as to overcome the problems of poor real-time performance and insufficient precision in the prior art.

[0005] In order to achieve the above-mentioned purpose, the present application adopts the following technical scheme: An automatic focal length tracking device of a microscopic scanning detection system, comprising a main camera, a horizontal camera and a beam splitter; the beam splitter is installed in a microscope tube and is a semi-transparent and semi-reflective optical plane mirror, used for splitting the imaging light transmitted by an objective lens into two paths and transmitting to the main camera and the horizontal camera respectively; the imaging target surface of the main camera is perpendicular to the transmitted imaging light axis received thereby; an included angle α is formed between the imaging target surface of the horizontal camera and the reflected imaging light axis received thereby, wherein α≠90; Wherein, the control module calculates a definition function according to the imaging data of the horizontal camera and outputs a compensation instruction of the main camera.

[0006] Further, the included angle α makes the images collected by the horizontal camera correspond to the lower defocus area, the in-focus area and the upper defocus area in sequence from top to bottom when the main camera is in focus.

[0007] An automatic focal length tracking method of a microscopic scanning detection system, using the device described above, characterized in that it comprises the following steps: The main camera shoots detection images at a frame rate N; The horizontal camera shoots focal length tracking images at a frame rate M, and the frame rate M of the horizontal camera is greater than or equal to 3 times the frame rate N of the main camera; According to the evaluation objects extracted from the multiple frames of focal length tracking images shot by the horizontal camera, the definition function value is calculated and compared to determine the focal length state and output the compensation instruction of the main camera.

[0008] Further, it further comprises the following steps: Before the main camera is operated, the main camera is first adjusted to be in focus, and then the horizontal camera is adjusted so that the central area falls on the ideal focal plane as a reference.

[0009] Further, the selection of the evaluation objects according to the multiple frames of focal length tracking images shot by the horizontal camera specifically comprises: The detection images shot by the horizontal camera are equally divided into X strips along the vertical direction, X≥5, the strips correspond to different focal depth positions, and the layer spacing=maximum defocus amount ΔZmax / X.

[0010] Further, the calculation of the definition function value and the comparison to determine the focal length state specifically comprise the following steps: At least 3 frames of focal length tracking images shot by the horizontal camera are selected; In time sequence, the selected strips are marked in the 3 frames of focal length tracking images and numbered as A, B and C respectively, A represents the upper defocus critical area, B represents the in-focus area, and C represents the lower defocus critical area; Fclear is brought into the evaluation function, the better the focus, the larger the Fclear value, and Fclear(A), Fclear(B) and Fclear(C) are compared; If normal focusing, F clear (B) > F clear (A) ≈ F clear (C); If above focus, F clear (A) > F clear (B) > F clear (C); If below focus, F clear (C) > F clear (B) > F clear (A).

[0011] Further, the compensation instruction of the output main camera specifically comprises the steps of: When the object distance deviates from the ideal focus surface by more than 2 / 3 of the depth of focus LZ, triggering the Z-direction compensation; The Z-direction compensation is only performed once between every two frames of main camera imaging, and the compensation is performed by the object table or the objective lens.

[0012] The technical scheme of the present application can realize rapid detection and automatic compensation of the focal length state in the microscopic scanning detection process, so that the main camera can still maintain stable and clear imaging under high-speed scanning conditions, thereby significantly improving the reliability and efficiency of detection. The present application divides the optical path into two paths by a beam splitter, the main camera is used to obtain a detection image, the horizontal camera acquires a focal length tracking image at a frame rate higher than that of the main camera, and accurate focal length state determination is realized based on the defocus distribution region and the multi-frame sharpness function comparison, avoiding the disadvantages of feedback lag of the single camera method. Since only a beam splitter and a horizontal camera need to be added in the microscope, the overall structure is simple, easy to integrate, widely applicable, and can be applied in various scenes such as biological section scanning, material surface detection, semiconductor defect detection, etc., and has good industrialization prospects.

[0013] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structure particularly pointed out in the written description and the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0014] The present application will be described in detail below with reference to the accompanying drawings, so that the above advantages of the present application are more apparent.

[0015] Figure 1 is a schematic diagram of an automatic focal length tracking device of a microscopic scanning detection system of the present application; Figure 2 is a partial schematic diagram of an automatic focal length tracking device of a microscopic scanning detection system of the present application; Figure 3 is a main camera focusing imaging diagram of an automatic focal length tracking device of a microscopic scanning detection system of the present application; Figure 4It is the horizontal camera object distance overlarge imaging figure of the automatic focal length tracking device of a microscopic scanning detection system of the present application; Figure 5 It is the object distance too small imaging figure of the automatic focal length tracking device of a microscopic scanning detection system of the present application; Figure 6 It is the position deviation diagram between the image surface and the ideal image surface of the automatic focal length tracking device of a microscopic scanning detection system of the present application; Figure 7 It is the cutting diagram of the automatic focal length tracking method of a microscopic scanning detection system of the present application; Figure 8 It is the partition diagram of the automatic focal length tracking method of a microscopic scanning detection system of the present application. DETAILED DESCRIPTION

[0016] Embodiments of the present application are described below in detail, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary, and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0017] In the description of the present application, it is understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the drawings described, and are only for the purpose of facilitating the description of the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application.

[0018] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0019] In the embodiments of the present application, unless otherwise specifically defined and limited, the terms "mounting", "connection", "connection", "fixing" and the like should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0020] Reference is made to the accompanying drawings Figures 1-8 As shown in the drawings, an automatic focal length tracking device of a microscopic scanning detection system comprises a main camera 100, a horizontal camera 200 and a beam splitter 300. The beam splitter 300 is installed in a microscope tube and adopts a half-transmission half-reflection optical plane mirror structure, which is used to divide the imaging light rays transmitted by an objective lens into two paths, one path of the transmitted light is transmitted to the main camera 100 for acquiring a detection image, and the other path of the reflected light is transmitted to the horizontal camera 200 for real-time monitoring of the focal length state.

[0021] The imaging target surface of the main camera 100 is arranged perpendicularly to the transmitted imaging optical axis to ensure the resolution and imaging stability of the detection image acquired by the main camera 100. An included angle α is formed between the imaging target surface of the horizontal camera 200 and the optical axis of the received reflected light, where α≠90°. Through the non-right-angle design, the image collected by the horizontal camera 200 can form a distribution characteristic containing a lower defocus area, a focus area and an upper defocus area in the vertical direction.

[0022] In actual application, the main camera 100 is responsible for acquiring the detection image of the sample, and the horizontal camera 200 synchronously acquires the focal length state information in the light path. A control module is connected with the horizontal camera 200, receives and analyzes the imaging result of the horizontal camera 200, and outputs a corresponding compensation instruction to drive the object table or the objective lens to make a Z-direction fine adjustment when the focal length deviation is detected, so that the main camera 100 reenters the focus state.

[0023] Compared with the prior art, the embodiment realizes real-time monitoring of the imaging state of the main camera 100 by arranging the beam splitter 300 in the microscope light path and introducing the horizontal camera 200, avoids the problem of slow response of the single-camera focusing scheme, and through the non-right-angle arrangement, the image collected by the horizontal camera 200 naturally contains the focal length distribution information, which provides a basis for subsequent judgment and compensation, thereby significantly improving the stability and efficiency of the microscopic scanning detection process.

[0024] Further, the setting of the included angle α makes the image collected by the horizontal camera 200 present a specific focal length distribution rule in the vertical direction when the main camera 100 is in the focus state, i.e., the lower region of the image corresponds to the lower defocus area, the middle region corresponds to the focus area, and the upper region corresponds to the upper defocus area.

[0025] The design idea is that by setting the imaging target surface of the horizontal camera 200 and the reflected light optical axis in a non-right-angle relationship, the light rays from different focal depth positions are spatially offset on the imaging surface of the horizontal camera 200. In this way, when the main camera 100 is in focus, the image collected by the horizontal camera 200 is naturally divided into three regions of upper and lower defocus areas and a middle focus area, thereby being able to intuitively reflect the up and down offset of the objective lens focal plane.

[0026] The working principle is that when the object distance of the sample and the objective lens changes, the focal plane will move up or down. If the focal plane deviates to one side of the objective lens, the position of the in-focus area in the horizontal camera 200 imaging is moved up, and the lower out-of-focus area range is expanded. If the focal plane deviates in the opposite direction of the objective lens, the in-focus area is moved down in the imaging, and the upper out-of-focus area range is expanded. By observing the position change of the in-focus area in the image, the focal length state and the deviation direction can be determined.

[0027] With the above design, the technical advantages are that the focal length state is intuitive and visible, the image of the horizontal camera 200 naturally contains the distribution of the lower out-of-focus area-in-focus area-upper out-of-focus area, avoiding the limitation that it is difficult to distinguish the upper and lower out-of-focus areas by single-point imaging; It is convenient to determine the deviation direction. By the up and down deviation of the in-focus area in the image, it can be directly judged whether the object distance between the sample and the objective lens is too large or too small, thereby providing an accurate direction for automatic compensation; The stability and robustness are enhanced. Compared with the method relying on single-point sharpness evaluation, the reliability of focus determination is improved by the spatial distribution characteristics, and the probability of false judgment caused by noise fluctuation or insufficient local contrast is reduced.

[0028] The embodiment provides an automatic focal length tracking method of a microscopic scanning detection system, which is applied to the automatic focal length tracking device. The method comprises the following steps: First, the main camera 100 acquires a detection image at a frame rate N, which is used to record the microscopic imaging result of the measured sample. At the same time, the horizontal camera 200 acquires a focal length tracking image at a frame rate M, wherein the frame rate M is not less than three times the frame rate N of the main camera 100. By making the sampling frequency of the horizontal camera 200 much higher than that of the main camera 100, multiple focal length tracking images can be obtained within the time of one imaging cycle of the main camera 100, so that the focal length change can be captured in real time, and the compensation lag caused by insufficient sampling is avoided.

[0029] In the multiple focal length tracking images acquired by the horizontal camera 200, the control module extracts evaluation objects by an image processing method. The evaluation objects can be independent target objects in the image, such as a single die in a semiconductor detection scene or a single cell in a biological detection scene. Since the present application adopts a scanning and photographing mode in motion, the object table maintains uniform linear motion during scanning, so the same evaluation object will appear in different vertical positions in the multiple images obtained by the horizontal camera 200. Combined with the structural design that there is an angle α between the imaging target surface of the horizontal camera 200 and the optical axis, these different positions correspond to different focal depth positions such as the lower out-of-focus area, the in-focus area and the upper out-of-focus area.

[0030] The control module calculates the sharpness function of the evaluation object region, and compares the sharpness function values in multiple frames of images to determine whether the main camera 100 is currently in a focus state, or in an upper defocus or lower defocus state. When the determination result indicates that the main camera 100 is imaging deviated from the ideal focal plane, the control module immediately outputs a compensation instruction to drive the stage or the objective lens to fine-tune in the Z direction, so that the main camera 100 is restored to the focus state.

[0031] The design idea of this embodiment is to utilize the division of labor and cooperation between the main camera 100 and the horizontal camera 200: the main camera 100 undertakes normal detection imaging tasks, while the horizontal camera 200 is used for focal length tracking at a higher frame rate. By comparing and evaluating the evaluation object in multiple frames at a high sampling frequency, the system can immediately identify and compensate when the focal length deviates slightly, thereby realizing dynamic tracking and stable control of the focal length.

[0032] Compared with the prior art, this method realizes real-time detection and rapid compensation of focal length changes while ensuring the continuity of imaging of the main camera 100, and can effectively improve the imaging stability and detection efficiency of the microscopic scanning detection process.

[0033] Further, before the main camera 100 is formally operated, the system is calibrated. Specifically, first, the main camera 100 is adjusted so that its imaging is in the focus state; then the horizontal camera 200 is adjusted so that its central region corresponds to the ideal focal plane of the main camera 100, and this position is taken as the reference.

[0034] The design idea is that since the main camera 100 is the core component for collecting detection images in the microscopic scanning detection system, it is necessary to prioritize ensuring that the main camera 100 is in the focus state to ensure the accuracy and reliability of the detection images. On this basis, the position of the horizontal camera 200 is adjusted so that its central region falls on the ideal focal plane position of the main camera 100. Through this reference calibration, the images collected by the horizontal camera 200 can exhibit the standard distribution of the lower defocus region, the focus region and the upper defocus region in the focus state.

[0035] The working principle is that when the main camera 100 is aligned with the sample and completes focusing, this focal plane is defined as the ideal focal plane. At this time, the installation position or imaging parameters of the horizontal camera 200 are fine-tuned so that its central imaging region accurately corresponds to the ideal focal plane. In this way, during the subsequent detection process, once the object distance between the sample and the objective lens deviates, the focus region in the imaging of the horizontal camera 200 will correspondingly move up or down, thereby providing a reliable reference benchmark for the control module to determine the focal length state.

[0036] Further, the step of selecting the evaluation object according to the multiple frames of focus tracking images captured by the horizontal camera 200 specifically comprises: dividing the focus tracking images captured by the horizontal camera 200 into X strips along the vertical direction, where X is not less than 5. Each strip corresponds to a different focal depth position, and the interlayer spacing between the strips is set as the ratio of the maximum defocus amount ΔZmax and X, i.e., ΔZmax / X.

[0037] In actual applications, the horizontal camera 200 is arranged at a non-right angle with the optical axis of the reflected light, and the imaging result thereof presents the distribution characteristics of the lower defocus area, the in-focus area, and the upper defocus area in the vertical direction. By dividing the image into multiple strips, each strip can correspond to a different focal depth range, thereby realizing the differentiation of different focal plane positions.

[0038] In a preferred embodiment, the control module can determine the evaluation object region in the divided strip region according to specific detection requirements, for subsequent calculation and comparison of the sharpness function. In this way, the strip position and focal depth can be used to determine and estimate the focal length offset.

[0039] The design idea is that: by the strip division method, the focal length information contained in the imaging of the horizontal camera 200 is discretized into multiple independent intervals, so as to perform sharpness evaluation and trend analysis in each interval. The change of the sharpness function value of different strips can reflect the direction and degree of the up and down shift of the focal plane, thereby improving the accuracy of focal length determination.

[0040] The technical advantages brought by this step are: improving the determination accuracy, by dividing the image into multiple strips, the focal length distribution can be refined, and the system can capture smaller focal length changes; enhancing the anti-interference capability, the sharpness values of different strip regions can be compared with each other, avoiding the interference of single region noise fluctuation on the overall determination; strong adaptability, one or more strips can be selected as the evaluation object according to the sample characteristics and detection requirements, thereby improving the applicability of the method.

[0041] Further, in this embodiment, the method is applicable to the case where the main camera 100 is in the scanning shooting mode. When the object table moves between two adjacent shooting points, the horizontal camera 200 performs high-speed continuous shooting at a frame rate higher than that of the main camera 100, thereby obtaining multiple focus tracking images in one scanning motion.

[0042] In the focal length tracking images, the control module selects an evaluation block by image processing method, and performs tracking analysis on the block. Since the object table is in a continuous motion state, the evaluation block will appear in different focal depth positions such as the lower defocus area, the in-focus area and the upper defocus area in the multiple frames of images of the horizontal camera 200. By calculating the sharpness function value of the block at different positions, it can be judged which position has the clearest imaging, and the focal depth state corresponding to the clearest position can reflect the defocus amount of the objective lens.

[0043] In order to further realize quantitative analysis of the focal length offset, the embodiment divides the images collected by the horizontal camera 200 into X strips along the vertical direction, wherein the central strip corresponds to the ideal in-focus area. As the evaluation block crosses different strips in the movement process, the offset degree of the evaluation block relative to the center can reflect the change trend of the focal length. The farther the strip is from the center position, the greater the corresponding focal depth offset value. By establishing the mapping relationship between the strip position and the physical focal depth, the control module can convert the actual focal depth offset of the microscope objective relative to the ideal focal plane according to the vertical offset distance of the strip where the evaluation block is located.

[0044] The technical advantages of this embodiment are: on the one hand, by combining scanning motion with high-speed snapshot, multiple focal length tracking images can be obtained without increasing the dwell time, and real-time monitoring of the focal length state can be realized; on the other hand, by the corresponding relationship between the strip position and the focal depth offset, the focal length determination is expanded from qualitative judgment to quantitative calculation, thereby significantly improving the accuracy and controllability of compensation control.

[0045] Further, assuming that the moving speed of the object table or the objective lens in the Z direction is V, then the object surface offset ΔS and the sampling time interval T satisfy: T=ΔS / V; This formula shows that when the object distance is displaced, the sampling interval of the horizontal camera 200 can be dynamically adjusted according to the offset and the scanning speed, so as to collect enough focal length tracking images between the imaging of the two frames of main camera 100. This design enables the system to still maintain real-time focal length monitoring during high-speed scanning, effectively avoiding image blur caused by delayed compensation of the main camera 100.

[0046] Further, the step of calculating the sharpness function value and comparing to determine the focal length state comprises: the control module selects at least three frames from the multiple frames of focal length tracking images continuously captured by the horizontal camera 200 for analysis. In time sequence, the pre-selected evaluation strips are marked in the three frames of focal length tracking images respectively and numbered in sequence as A, B and C, wherein A represents the upper defocus critical area, B represents the in-focus area, and C represents the lower defocus critical area. Then, the three frames of evaluation strips are respectively brought into the sharpness evaluation function Fclear for calculation to obtain the sharpness values corresponding to the three frames. Since the sharpness value is proportional to the focusing degree, the clearer the image is, the larger the corresponding Fclear value is.

[0047] By comparing the size relationship of Fclear(A), Fclear(B) and Fclear(C), the focal length state of the main camera 100 can be determined. When in the normal in-focus state, the sharpness of the B area is the highest, i.e. Fclear(B) > Fclear(A) ≈ Fclear(C); when the object distance is up-defocused, the A area is the clearest, followed by the B area and the C area, i.e. Fclear(A) > Fclear(B) > Fclear(C); when the object distance is down-defocused, the C area is the clearest, followed by the B area and the A area, i.e. Fclear(C) > Fclear(B) > Fclear(A).

[0048] This method can accurately distinguish the in-focus state from the defocus states in different directions by establishing the comparison relationship of the sharpness function values in the three areas of up-defocus, in-focus and down-defocus. Compared with the traditional method which relies on a single frame of image, the determination logic based on multiple frame comparison in this embodiment uses the comparison trend in different areas instead of isolated values, effectively eliminates the misjudgment caused by noise or local texture difference, and improves the robustness and accuracy of focal length determination.

[0049] The technical advantages brought by this step are: strong anti-interference capability, through multiple frame comparison, avoiding the influence of single frame by noise or sudden interference, improving the stability of determination; Ability to distinguish direction, through the relative relationship of the sharpness values among the three frames, the focal length can be determined to be biased towards up-defocus or down-defocus, instead of only detecting the blur degree; Simple and realizable determination logic, the size relationship of the sharpness values can be used to complete the focal length state determination, low algorithm complexity, easy to quickly realize in hardware.

[0050] Further, the step of introducing the evaluation function Fclear specifically comprises: in each frame of focus tracking image collected by the horizontal camera 200, the control module extracts an evaluation object block containing the feature to be detected based on a preset coordinate or a dynamic tracking algorithm. The size of the block is set to be no less than 5*5 pixels and no more than 1 / 10 of the vertical height of the image, so as to ensure the representativeness and efficiency of the calculation.

[0051] After the block is obtained, the region is first subjected to a grayscale processing, and then at least one of a Sobel operator, a Laplacian operator or a Tenengrad operator is used to calculate the gradient amplitude of each pixel point in the block, thereby generating a corresponding gradient amplitude matrix. Based on the matrix, a sharpness evaluation value Fclear can be obtained through the following scalar function: wherein, is the grayscale value of the block at the coordinate (x, y), and M and N are the size of the block; The block is subjected to a two-dimensional discrete Fourier transform, and the proportion of high-frequency component energy is calculated as Fclear. The Fclear value of the current frame is compared with the evaluation value of the adjacent frame, and a defocus state judgment logic is combined to output a focal length adjustment direction and a compensation amount.

[0052] wherein, G(x, y) represents the gradient amplitude of the block at the pixel point (x, y), and M and N are the length and width of the block. The larger the function value is, the clearer the details and edges in the image are, and the better the focusing state is.

[0053] In addition to the gradient method in the spatial domain, the embodiment can also use a frequency domain analysis method: the block is subjected to a two-dimensional discrete Fourier transform (2D-DFT), and the proportion of high-frequency component energy is calculated as the value of the sharpness function Fclear. Since the image contains more high-frequency detail information under the in-focus state, the higher the proportion of high-frequency component energy is, the better the focusing effect is.

[0054] After the Fclear value is obtained, the control module compares it with the evaluation value of the adjacent frame or the adjacent strip, combines the defocus judgment logic in Embodiment 6, and outputs a focal length adjustment direction and a compensation amount.

[0055] The technical advantage of this embodiment is that multiple operators can be selected, and Sobel, Laplacian, Tenengrad or frequency energy method can be flexibly selected according to the characteristics of the sample image, thereby improving the adaptability. The evaluation result is reliable, the gradient or frequency domain analysis on the block region can effectively reflect the image definition, and the influence of overall brightness or noise fluctuation on the determination is avoided; The determination is combined with compensation, the definition function value is directly related to the focal length state, the determination logic is combined with the defocus determination logic, and real-time compensation control is realized.

[0056] Further, the compensation instruction of the output main camera 100 specifically includes the following steps: when the control module determines that the position of the object distance deviating from the ideal focal plane exceeds 2 / 3 of the preset focal depth L Z , the Z-direction compensation operation is triggered. The compensation action is limited in frequency in the continuous imaging process of the main camera 100, that is, the compensation instruction is only allowed to be executed once between every two frames of image acquisition of the main camera 100. The compensation action can be executed by the Z-direction driving mechanism of the object table or the focusing mechanism of the objective lens, so as to realize the accurate adjustment of the object distance between the main camera 100 and the sample.

[0057] The design idea is that: by setting the deviation threshold, the compensation is avoided from being frequently triggered when the object distance fluctuates slightly, so as to ensure the stability of the compensation action; and by limiting the frequency of the compensation execution, the continuous imaging process of the main camera 100 is not disturbed by frequent mechanical movements, so as to maintain the continuity and consistency of the imaging.

[0058] The working principle is that: the control module calculates and compares the definition function value according to the focal length tracking image acquired by the horizontal camera 200, and determines whether the current object distance has deviated from the ideal focal plane. When the deviation degree is less than 2 / 3L Z , the compensation is not triggered to avoid invalid action; when the deviation degree exceeds the threshold, the compensation instruction is immediately issued, and the object table or the objective lens is driven to perform fine adjustment in the Z-direction. Since the compensation is only executed once between every two frames of imaging of the main camera 100, the acquisition of the detection image is not disturbed.

[0059] The design of the embodiment can achieve the following technical advantages: reducing invalid compensation, avoiding repeated compensation for slight focal length fluctuation through threshold determination, and improving the stability of system operation; ensuring the continuity of imaging, limiting the compensation frequency, keeping the detection image of the main camera 100 stable between continuous frames, and avoiding image blur or loss caused by frequent mechanical action; flexible execution mode, the compensation can be executed by the object table or the objective lens driving mechanism, which is suitable for different structures of the microscopic scanning detection system and improves the applicability of the system.

[0060] Further, in order to quantify the relationship between the horizontal camera 200 image strip position and the actual focal length offset of the object plane, the embodiment establishes the following geometric mapping model. Assuming that the pixel size of the horizontal camera 200 is p, the row number offset of the evaluation strip center relative to the image center is r, and the angle α is the inclination angle between the imaging target surface of the horizontal camera 200 and the reflected light optical axis, then the projection distance dimg of the strip on the imaging surface of the horizontal camera 200 can be expressed as: ; According to the imaging geometric relationship, the projection distance corresponds to the equivalent displacement ΔS on the object plane. Therefore, the mapping relationship can be established as: ; Wherein, K(α) is a proportional coefficient related to the optical path angle and the system magnification, which can be obtained by experimental calibration. In this way, the vertical position change of the horizontal camera 200 strip can be directly converted into the actual focal length offset of the object plane, providing data basis for the output compensation of the control module.

[0061] Further, in order to establish the conversion relationship between the back focal distance ΔZ observed on the imaging surface of the horizontal camera 200 and the actual working distance Δz of the objective lens, the embodiment utilizes the magnification formula: ; Wherein M is the magnification of the objective lens. Thus we can get: ; That is, the back focal displacement ΔZ measured on the imaging surface of the horizontal camera 200 and the actual displacement Δz of the object plane are inversely proportional to the square. Through the geometric conversion, the detection result of the horizontal camera 200 can be accurately converted into the actual working distance offset of the objective lens, thereby improving the accuracy and controllability of automatic compensation.

[0062] As can be seen from the above embodiment, by introducing the beam splitter 300 and the horizontal camera 200 into the microscope optical path, the present application realizes real-time tracking of the focal length state of the main camera 100, and combines high frame rate sampling, multi-strip division, multi-frame comparison and definition function calculation to accurately determine the focal length state and output compensation instructions. Compared with the prior art, the present application has the advantages of simple structure, accurate determination, timely response, and good industrial application prospect.

[0063] Finally, it should be noted that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalent ones. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An autofocusing device for a microscopic scanning inspection system, characterized by, The microscope comprises a main camera, a horizontal camera and a beam splitter; The beam splitter is installed in a microscope tube and is a semi-transparent semi-reflective optical plane mirror for splitting the imaging light transmitted by the objective lens into two paths and transmitting the two paths to the main camera and the horizontal camera respectively; The imaging target surface of the main camera is perpendicular to the transmitted imaging optical axis received thereby; An included angle α is formed between the imaging target surface of the horizontal camera and the reflected imaging optical axis received thereby, wherein α≠90; The control module calculates a definition function according to the imaging data of the horizontal camera and outputs a compensation instruction of the main camera.

2. The apparatus according to claim 1, wherein The included angle α makes the images captured by the horizontal camera correspond to the lower defocus area, the in-focus area and the upper defocus area in sequence from top to bottom when the main camera is in focus.

3. A method for automatic focus tracking of a microscopic scanning detection system, applied to the apparatus of any one of claims 1-2, characterized in that, The method comprises the following steps: The main camera shoots a detection image at a frame rate N; The horizontal camera shoots a focal length tracking image at a frame rate M, and the frame rate M of the horizontal camera is greater than or equal to 3 times the frame rate N of the main camera; The evaluation object is extracted according to the multiple frames of focal length tracking images shot by the horizontal camera, the definition function value is calculated and compared to determine the focal length state and output the compensation instruction of the main camera.

4. The method of claim 3, wherein the method further comprises: The method further comprises the following steps: Before the main camera is operated, the main camera is first adjusted to be in focus, and then the horizontal camera is adjusted so that the central region thereof falls on an ideal focal plane as a reference.

5. The method of claim 3, wherein the method further comprises: The selection of the evaluation object according to the multiple frames of focal length tracking images shot by the horizontal camera specifically comprises the following steps: The detection image shot by the horizontal camera is equally divided into X strips in the vertical direction, X≥5, the strips correspond to different focal depth positions, and the layer spacing = maximum defocus amount ΔZmax / X.

6. The method of claim 3, wherein the method further comprises: In the scanning shooting mode, the horizontal camera shoots multiple focal length tracking images at high speed during the movement between two adjacent shooting points; An evaluation block is selected in the focal length tracking image, the evaluation block will appear at different positions such as the lower defocus area, the in-focus area and the upper defocus area in the multiple frames of images shot by the horizontal camera, and the position corresponding to the most clear evaluation block can reflect the size of the defocus amount of the objective lens; The image of the horizontal camera is equally divided into X strips in the vertical direction, the central strip corresponds to the in-focus area, the greater the deviation of the strip from the central region, the greater the focal depth deviation value, and the vertical distance of the strip from the central region can be used to convert the physical quantity of the focal depth deviation of the microscope objective lens.

7. The method of claim 6, wherein the method further comprises: The calculation of the definition function value and the comparison to determine the focal length state specifically comprise the following steps: At least three frames of focal length tracking images shot by the horizontal camera are selected; The selected strips are marked in the three frames of focal length tracking images in time sequence and are numbered as A, B and C respectively, A represents the upper defocus critical area, B represents the in-focus area and C represents the lower defocus critical area; The evaluation function Fclear is brought in, the better the focusing, the greater the Fclear value, and the comparison of Fclear(A), Fclear(B) and Fclear(C) is made; If normal focusing is achieved, Fclear(B)>Fclear(A)≈Fclear(C); If the object distance is defocused, Fclear(A)>Fclear(B)>Fclear(C). If the object distance is under defocus, then F clear (C)>F clear (B)>F clear (A).

8. The method of claim 7, wherein the method further comprises: The compensation instruction of the output main camera specifically comprises the steps of: When the object distance deviates from the ideal focal plane by more than 2 / 3 of the focal depth LZ, triggering the Z-direction compensation; The Z-direction compensation is only performed once between every two frames of main camera imaging, and the compensation is performed by the objective table or the objective lens. The Z-direction compensation is only performed once between every two frames of main camera imaging, and the compensation is performed by the objective table or the objective lens.

Images