A method, system and device for intelligent automatic focusing of a microscope

Abstract

The present application discloses a method, system and device for intelligent automatic focusing of a microscope, which relates to the field of microscope focusing. The method includes: performing target detection on a set of scanned images by using an adaptive search algorithm to obtain a detection result; if there is a target sample in the set of scanned images, extracting multiple frames of scanned images based on the existing position area, and calculating the clarity score value of each frame of scanned image extracted based on multiple image features; selecting the scanned image with the highest clarity score value, and marking the corresponding focal plane as the final focal plane of the path point; using the adaptive search algorithm to determine the next path point, and then returning to the step of obtaining the set of scanned images; if the result indicates that there is no target sample in the set of scanned images, using the adaptive search algorithm to determine the next path point, and then returning to the step of obtaining the set of scanned images. The present application can improve the efficiency, accuracy and adaptability of microscope automatic focusing.

Description

Technical Field

[0001] The present application relates to the field of microscope focusing, and particularly to a microscope intelligent autofocus method, system and device. Background Art

[0002] In the field of biochip analysis, the focusing accuracy requirements for sample detection under a microscope are extremely strict. Since the size of the chip sample is usually in the micron level, manual focusing under a traditional microscope is not only time-consuming and laborious, but also difficult to ensure the consistency of focusing accuracy. Especially under a high-power microscope, the depth of focus range is smaller, and the focusing difficulty is greater. A slight deviation will cause the image to be blurred, affecting the accuracy of subsequent analysis results.

[0003] Conventional autofocus technologies mainly adopt a global scanning method. There are many problems when dealing with chip samples by this method: global scanning takes a long time, reducing the detection efficiency; due to the uneven distribution of samples on the chip surface, it is easily interfered and the positioning accuracy is unstable; the lack of intelligent adaptive ability makes it difficult to cope with complex and changeable sample conditions. Under a high-power microscope, these problems are more prominent. The requirement for focusing accuracy can reach the sub-micron level, which poses a higher technical challenge to the autofocus system. Summary of the Invention

[0004] The purpose of the present application is to provide a microscope intelligent autofocus method, system and device, which can improve the efficiency, accuracy and adaptability of microscope autofocus.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In the first aspect, the present application provides a microscope intelligent autofocus method, including:

[0007] Obtain a set of scanned images; the set of scanned images is a set of scanned images obtained at any path point when a microscope component scans and images a target area along a target path; a target sample is distributed in the target area; different scanned images correspond to different focal planes;

[0008] Adopt an adaptive search algorithm to perform target detection on the set of scanned images to obtain a detection result; the detection result includes an existence result and an existence position area;

[0009] If the existence result indicates that there is a target sample in the set of scanned images, record that the number of valid points is incremented by one, extract multiple frames of scanned images based on the existence position area, and calculate the clarity score value of each frame of scanned image extracted based on multiple image features;

[0010] Select the scanned image with the highest clarity score value, and mark the corresponding focal plane as the final focal plane of the path point;

[0011] An adaptive search algorithm is adopted to adjust the point spacing according to the effective number of points, the path points, and the target path adjustment points to determine the next path point, and then return to the step of obtaining the scanned image set;

[0012] If the existence result indicates that the target sample does not exist in the scanned image set, record that the invalid number of points is incremented by one. Adopt an adaptive search algorithm to adjust the point spacing according to the invalid number of points, the path points, and the target path to determine the next path point, and then return to the step of obtaining the scanned image set.

[0013] In a second aspect, the present application provides a microscope intelligent autofocus system, including:

[0014] A scanned image acquisition module, configured to: acquire a scanned image set; the scanned image set is a set of scanned images obtained at any path point when the microscope component scans and images the target area according to the target path; the target sample is distributed in the target area; different scanned images correspond to different focal planes;

[0015] An adaptive search module, configured to: adopt an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence position area;

[0016] A sharpness scoring module, configured to: if the existence result indicates that the target sample exists in the scanned image set, record that the effective number of points is incremented by one, extract multiple frames of scanned images based on the existence position area, and calculate the sharpness scoring value of each frame of the scanned image extracted based on multiple image features;

[0017] A focal plane determination module, configured to: select the scanned image with the highest sharpness scoring value, and mark the corresponding focal plane as the final focal plane of the path point;

[0018] The adaptive search module is further configured to: adopt an adaptive search algorithm to adjust the point spacing according to the effective number of points, the path points, and the target path to determine the next path point, and then return to the scanned image acquisition module; if the existence result indicates that the target sample does not exist in the scanned image set, record that the invalid number of points is incremented by one, adopt an adaptive search algorithm to adjust the point spacing according to the invalid number of points, the path points, and the target path to determine the next path point, and then return to the scanned image acquisition module.

[0019] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the microscope intelligent autofocus method.

[0020] According to the specific embodiments provided by the present application, the present application has the following technical effects: The present application provides a method, system and device for intelligent autofocus of a microscope. An adaptive search algorithm is used to detect targets in a set of scanned images, quickly determine whether the target sample is included in the field of view at the current path point, and obtain the existence result and the existence position area, thereby avoiding wasting computing resources in invalid areas. Then, when there is a target sample, calculate the clarity score value of the scanned images within the existence position area, and determine the final focal plane based on this clarity score value, which can quickly screen the sample area and accurately locate the focal plane, ensuring the efficiency and accuracy of microscope autofocus. After there is no target sample or the focal plane has been determined, an adaptive search algorithm is used to determine the next path point. During this process, the computing overhead can be reduced and the screening efficiency can be improved. The adaptive search algorithm is used multiple times in the present application, which can avoid invalid search operations, shorten the focusing time, and improve the focusing speed. In summary, the present application can achieve efficient and accurate autofocus, significantly improve the focusing speed, accuracy and success rate, and at the same time has the advantages of high adaptability and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 It is an application environment diagram of a method for intelligent autofocus of a microscope in an embodiment of the present application.

[0023] Figure 2 It is a schematic flowchart of a method for intelligent autofocus of a microscope provided in an embodiment of the present application.

[0024] Figure 3 It is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0026] The present application provides a microscope intelligent autofocus method, system and device, which can improve the accuracy of sample recognition and positioning, optimize the focus search strategy to improve efficiency, and enhance the adaptive ability and stability of the system. In practical applications, the present application can also introduce advanced technologies such as deep learning for object detection and adaptive search algorithms, and combine with a memory optimization mechanism to significantly improve the intelligent level and automation degree of focusing, and achieve a focusing accuracy of sub-micron level. The solution of the present application also reduces manual intervention, improves the detection throughput and result reliability. Especially for the precise focusing requirements under high-magnification microscopes, it can break through the limitations of traditional technologies and provide more reliable and efficient technical support for biochip detection, which has important practical application value.

[0027] To make the above objects, features and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] The microscope intelligent autofocus method provided by the embodiment of the present application can be applied to an application environment as Figure 1 shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the scanned image set to the server 104. After receiving it, the server 104 takes a dual judgment mechanism (object detection and clarity scoring) as the core, quickly screens to obtain the sample area, and accurately focuses to obtain the final focal plane of each path point. The server 104 can feedback the final focal plane of each path point obtained to the terminal 102.

[0029] Among them, the terminal 102 can be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0030] In an exemplary embodiment, as Figure 2 shown, a microscope intelligent autofocus method is provided. This method is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or jointly executed by a terminal and a server. In the embodiment of the present application, taking this method applied to Figure 1 the server 104 in it as an example for illustration, it includes the following steps 201 to step 206.

[0031] Step 201, obtain a set of scanned images; the set of scanned images is a set of scanned images obtained at any path point when the microscope component scans and images a target area along a target path; the target area is distributed with target samples; different scanned images correspond to different focal planes.

[0032] In a practical application, the construction of the target path needs to be realized by combining a memory optimization algorithm, that is, by analyzing the historical focusing data set to optimize the current search path and range to be completed, so as to improve the overall focusing efficiency. Correspondingly, in step 201, before starting the scanning and imaging, optimize the target path according to the historical sample distribution law to preferentially process the sample-dense area and reduce the scanning of invalid areas, thereby improving the efficiency. Specifically, the determination process of the target path includes:

[0033] (11) Obtain a historical focusing data set; any historical focusing data in the historical focusing data set includes a historical path point. If the three-dimensional coordinate system of xyz is used as a reference, the historical path point is represented by (x, y), and the position of the historical final focal plane corresponding to the historical path point is represented by the z value.

[0034] (12) Uniformly divide the target area into multiple grid areas; for example, set the target area as Lx×Ly (width×height), and divide the entire target area range into Nx×Ny (horizontal×vertical) uniform grid areas according to a fixed size, and obtain the size of each grid area as: Δx = Lx / Nx, Δy = Ly / Ny.

[0035] (13) For each of the grid areas, based on the historical focusing data set, count the total number of historical path points in the grid area and the number of historical path points recorded as valid points, and calculate the sample detection rate. Specifically, the historical detection situation of the sample is counted by counting the number of path points, and then the sample detection rate Rd is calculated using the following formula: Rd = the number of historical path points recorded as valid points / the total number of historical path points × 100%.

[0036] (14) Based on the sample detection rate, determine the detection priority of the grid area; in a practical application, the marking and determination of the detection priority can be carried out according to the following settings: when Rd>80%, the grid area is a high detection rate area and also a high priority area, and these grid areas need to be scanned preferentially as the starting point of the target path; when 20%≤Rd≤80%, the grid area is a medium detection rate area and also a medium priority area, and after the high priority area is scanned, these grid areas are processed; when Rd<20%, the grid area is a low detection rate area and also a low priority area, and the scanning needs to be postponed or these grid areas can be directly skipped.

[0037] (15) Based on a preset priority scanning rule, determine a target path according to the detection priority of each grid region. When actually determining the target path, it can be processed from two aspects: path order and direction adjustment. In terms of path order, prioritize covering high-priority regions and then scan in order from high to low priority. In terms of direction adjustment, if high-priority regions are concentrated in a certain area, start scanning from these regions first to avoid redundancy in the global serpentine scanning path.

[0038] In summary, during the determination of the target path, analyze the change trend of the historical focal plane position (such as the position of the focal plane gradually moving upward or downward) based on the historical focusing dataset, and combine these trends to predict the priority direction of the current search to be completed. Especially after analysis, when the sample distribution pattern is obvious, dynamically adjust the target path, prioritize processing the sample-dense regions, and reduce the scanning of invalid regions to improve efficiency.

[0039] Step 202, use an adaptive search algorithm to perform target detection on the scanning image set to obtain a detection result; the detection result includes an existence result and an existence position region.

[0040] Combined with a memory optimization algorithm, in the step of target detection after scanning and imaging, the possible current focal plane range can be predicted according to the change trend of the historical final focal plane in the historical focusing dataset, thereby reducing the exploration of invalid z values. Among them, predicting the possible current focal plane range can obtain a preset range. The determination process of the preset range includes:

[0041] 1) Obtain a historical focusing dataset; any historical focusing data in the historical focusing dataset includes a historical path point and the corresponding historical final focal plane.

[0042] 2) Based on the historical focusing dataset, determine the change trend of the historical focal plane position and calculate the trend offset; specifically, if the historical focal plane position gradually increases, it indicates that the focal plane is moving upward, and the change trend of the historical focal plane position is an upward trend; if the historical focal plane position gradually decreases, it indicates that the focal plane is moving downward, and the change trend of the historical focal plane position is a downward trend. In practical applications, each path point corresponds to a trend offset. The trend offset corresponding to each path point (such as a historical path point) can calculate the average change amount of the z value of the corresponding historical final focal plane based on several historical path points with the same change trend of the historical focal plane position as the trend offset Δz_trend, such as taking values of ±5μm.

[0043] 3) Determine the preset range according to the trend offset, the final focal plane corresponding to the previous path point, and the preset search range. Specifically, the preset range can be represented as follows: [z_last + Δz_trend - Δz, z_last + Δz_trend + Δz], where z_last is the z value of the final focal plane of the previous path point, z_last + Δz_trend = z_focus_initial, and Δz is the preset search range set by relevant technicians. For example, it takes a small range of ±50μm. This search range can be adjusted as needed, such as adjusted to a medium range of ±150μm or a large range of ±1500μm.

[0044] After obtaining the preset range, use an adaptive search algorithm to preferentially search for the focal plane within the small range. If the best focal plane position (i.e., the final focal plane in the following text) cannot be found in the small range search, then gradually expand the search range to the medium range or the full range. By dynamically adjusting the search range in this way, the focusing efficiency can be greatly improved. In a specific application, step 202 includes the following steps (21)-(26).

[0045] (21) Extract a subset of scanned images from the set of scanned images according to the preset range and the preset scanning pitch.

[0046] (22) For the scanned images in the subset of scanned images, perform object detection on each one using an object detection algorithm to obtain a single image detection result. The object detection algorithm used can be an object detection method based on deep learning to identify sample features.

[0047] (23) If at least one of the single image detection results indicates the existence of the target sample, determine the corresponding existence position area and generate an existence result indicating the existence of the target sample in the set of scanned images.

[0048] (24) If all the single image detection results indicate the non-existence of the target sample, update the preset range, update the preset scanning pitch, and record that the update count is incremented by one.

[0049] (25) If the update count reaches the preset update count, determine that the existence position area is empty and generate an existence result indicating the non-existence of the target sample in the set of scanned images.

[0050] (26) If the update count does not reach the preset update count, return to step (21).

[0051] In another practical application, the preset range is updated, and the preset scanning pitch is updated. Specifically: the preset small range is updated to the preset medium range, and the preset small-range scanning pitch is updated to the preset medium-range scanning pitch; or, the preset medium range is updated to the preset large range, and the preset medium-range scanning pitch is updated to the preset large-range scanning pitch. The differences among the small range, the medium range, and the large range lie in the different preset search ranges Δz.

[0052] Through the above update, a step-by-step search from small to medium to large can be achieved, and the approximate z-axis position where the sample is located can be found more quickly. Specifically: within the preset small range [z_focus_initial−Δz1, z_focus_initial+Δz1], images are acquired at the preset small-range scanning pitch Δz_small (which can take a value of 0.5 μm) to obtain multiple scanned images, and the target detection algorithm is used to perform target detection one by one to determine whether there is a sample; if a sample is detected from the multiple scanned images, the approximate z-axis position of the sample is recorded, and an existence result indicating the presence of the target sample in the scanned image set is generated, and the search ends; if no sample is detected from the multiple scanned images, after the scanning within the preset small range is completed, the preset small range is updated to the preset medium range [z_focus_initial−Δz2, z_focus_initial+Δz2], and the preset small-range scanning pitch is updated to the preset medium-range scanning pitch.

[0053] Within the preset medium range, images are acquired at the preset medium-range scanning pitch Δz_medium (which can take a value of 2 μm) to obtain multiple scanned images, and the target detection algorithm is used to perform target detection one by one to determine whether there is a sample; if a sample is detected from the multiple scanned images, the approximate z-axis position of the sample is recorded, and an existence result indicating the presence of the target sample in the scanned image set is generated, and the search ends; if no sample is detected from the multiple scanned images, after the scanning within the preset medium range is completed, the preset medium range is updated to the preset large range [z_min, z_max], and the preset medium-range scanning pitch is updated to the preset large-range scanning pitch.

[0054] Within the preset large range, images are acquired at the preset large-range scanning pitch Δz_large (which can take a value of 10 μm) to obtain multiple scanned images, and the target detection algorithm is used to perform target detection one by one to determine whether there is a sample; if a sample is detected from the multiple scanned images, the approximate z-axis position of the sample is recorded, and an existence result indicating the presence of the target sample in the scanned image set is generated, and the search ends; if no sample is detected from the multiple scanned images, it is determined that the existence position area is empty, and an existence result indicating the absence of the target sample in the scanned image set is generated, which can also be shown to relevant technical personnel.

[0055] In the above-mentioned preset small range, preset medium range or preset large range, the approximate z-axis position z_focus_initial recorded when the sample is detected can be characterized by a certain z-axis value. Furthermore, in combination with the scoring criteria of the clarity scoring algorithm, the position area where the sample exists can be determined. The scoring criteria of the clarity scoring algorithm are set by relevant technical personnel. According to this scoring criteria, multiple frames of scanned images that can be collected are scored, so as to find the scanned image with the highest clarity score, and based on this, the position of the final focal plane (z_focus_final) of this path point is determined.

[0056] For example, the position area where the sample exists is set as [z_focus_initial - Δz, z_focus_initial + Δz], where Δz is a fixed scanning interval, and the value can be set to 50μm according to the scoring criteria. Within the above range, multiple frames of scanned images are collected, and the clarity scoring algorithm (such as gradient algorithm, Laplacian operator, high-frequency component analysis, etc.) in the subsequent steps is used to score each frame of scanned image to obtain the corresponding clarity score value.

[0057] Step 203, if the existence result indicates that there is a target sample in the scanned image set, then increment the recorded number of valid points by one, extract multiple frames of scanned images based on the existence position area, and calculate the clarity score value of each frame of scanned image extracted based on multiple image features.

[0058] Among them, after extracting multiple frames of scanned images corresponding to the position from the scanned image set, the clarity score value of each frame of scanned image extracted based on multiple image features is calculated. The calculation of the clarity score value can further analyze the multiple frames of scanned images selected to accurately locate the focal plane. The steps for calculating the clarity score value of each frame of scanned image extracted based on multiple image features can be calculated based on techniques such as gradient algorithm, Laplacian operator or high-frequency component analysis, and include the following steps (31)-(34).

[0059] (31) For any frame of scanned image extracted, use the gradient algorithm to calculate the gradient clarity score value. Among them, the gradient algorithm evaluates the clarity by calculating the sum of the gradient amplitudes of the pixel intensity changes in the image. The calculation formula of the gradient clarity score value is:

[0060] .

[0061] Among them, G(I) is the gradient clarity score value of the scanned image I, is the gray gradient of the pixel point (x, y) of the scanned image I in the horizontal direction, is the gray gradient of the pixel point (x, y) of the scanned image I in the vertical direction; is the accumulation of all pixel points of the scanned image I.

[0062] (32) The Laplace operator is used to calculate the Laplace sharpness score value through the second derivative; among them, the Laplace operator is the second derivative of the image and is used to detect the sharpness degree of the image edge. The calculation formula of the Laplace sharpness score value is as follows:

[0063] .

[0064] Among them, L(I) is the Laplace sharpness score value of the scanned image I, is the second derivative of the pixel point (x, y) of the scanned image I in the horizontal direction, is the second derivative of the pixel point (x, y) of the scanned image I in the vertical direction.

[0065] (33) High-frequency component analysis is adopted to calculate the high-frequency component sharpness score value through Fourier transform; among them, high-frequency component analysis is based on the Fourier transform (FFT) of the image, and the sharpness is evaluated by calculating the energy of the high-frequency region of the image. The calculation formula of the high-frequency component sharpness score value is as follows:

[0066] .

[0067] Among them, , H(I) is the high-frequency component sharpness score value of the scanned image I, F ( u , v ) is the Fourier transform result of the scanned image I in the frequency domain, is the energy of the high-frequency component of the scanned image I in the frequency domain, HF is the range of the high-frequency component (filtering the low-frequency component by setting the frequency threshold), is the Fourier transform operation, is the accumulation of the energy of the high-frequency component of the scanned image I in the frequency domain.

[0068] (34) The gradient sharpness score value, the Laplace sharpness score value and the high-frequency component sharpness score value are weighted and calculated to obtain the sharpness score value of the scanned image. The specific formula is as follows: ; among them, S(I) is the sharpness score value of the scanned image I, is the weight (adjusted according to experiments or specific application scenarios).

[0069] Step 204: Select the scanned image with the highest sharpness score value, and mark the corresponding focal plane as the final focal plane of the path point. Through the above steps (31)-(34), the gradient algorithm, Laplacian operator, high-frequency component analysis, and weighted fusion calculation can be applied to any frame of the scanned image to obtain the corresponding sharpness score value; then, compare the sharpness score values of all frames of the scanned images, and select the z value corresponding to the frame with the highest score as the final focal plane position. 。 I i is the i-th frame of the scanned image.

[0070] Step 205: Adopt an adaptive search algorithm to adjust the point spacing according to the effective number of points, the path point, and the target path adjustment point to determine the next path point, and then return to Step 201.

[0071] In this step, an adaptive search algorithm is adopted, which can efficiently detect the sample distribution and reduce the exploration of invalid regions. In a specific application, Step 205 includes: if the effective number of points is within the preset effective number of points range, reduce the point spacing, and determine the next path point according to the reduced point spacing, the path point, and the target path.

[0072] Corresponding to Step 202 above, in the application of the adaptive search algorithm in Step 205, the small-range point spacing, medium-range point spacing, and large-range point spacing are also correspondingly set. Specifically, if the sample is detected continuously for Ndetect times (i.e., the effective number of points is within the preset effective number of points range), it means that the sample distribution is relatively dense, then gradually reduce it in the order of large-range point spacing Δx_large, Δy_large → medium-range point spacing Δx_medium, Δy_medium → small-range point spacing Δx_small, Δy_small to continue the detailed scanning and improve the detection accuracy. As for the value of the point spacing, for the large range, it can be taken as 1000 μm; for the medium range, it can be taken as 500 μm; for the small range, it can be taken as 300 μm.

[0073] Step 206: If the target sample does not exist in the set of scanned images represented by the existence result, record that the number of invalid points is incremented by one, adopt an adaptive search algorithm, adjust the point spacing according to the number of invalid points, the path point, and the target path to determine the next path point, and then return to Step 201. In a specific application, an adaptive search algorithm is also adopted, and the steps for determining the next path point in Step 206 include steps (61)-(62).

[0074] If the number of invalid points is within the first preset range of invalid points, increase the point spacing, and determine the next path point based on the increased point spacing, the path points, and the target path. That is, if the sample is not detected for Nmiss consecutive times (corresponding to the first preset range of invalid points), it indicates that the current area may be an invalid area, and the point spacing needs to be increased. Corresponding to step 205 above, the increasing order can be the small-range point spacing Δx_small, Δy_small → the medium-range point spacing Δx_medium, Δy_medium → the large-range point spacing Δx_large, Δy_large.

[0075] If the number of invalid points is within the second preset range of invalid points, determine the next path point based on the preset jump amplitude, the path points, and the target path. That is, if the sample is not detected for Njump consecutive times (corresponding to the second preset range of invalid points), in the current situation, the point spacing is already the large-range point spacing, it can be determined that the path point is in an invalid area, and it can be directly skipped. After skipping, start point-by-point scanning again and restore the small-range point spacing to determine the next path point, and perform z-axis search and focusing on the next path point (see the processing in steps 201 - 204 above). Among them, the preset jump amplitude Δjump is usually a multiple of the current point spacing, such as 2×Δx_large. Among them, Njump is greater than Nmiss, that is, the lower limit value of the second preset range of invalid points is greater than or equal to the upper limit value of the first preset range of invalid points.

[0076] In another exemplary embodiment of the present application, a cell sample is known, and through observation by a microscope assembly, the characteristic distribution of the cell sample is locally dense and locally sparse. The microscope assembly includes a microscope and a sample stage. The microscope is equipped with a high-resolution camera, and the sample stage can accurately control the three-dimensional movement of (x, y, z), and supports a serpentine scanning path or a spiral scanning. Based on the above application scenario, the method of the present application can be used for intelligent automatic focusing of the microscope, including the following steps (1) to (6).

[0077] (1) Initialization.

[0078] Set the path, scanning range, the neural network model used for target detection of the scanned image set, the scoring criteria of the clarity scoring algorithm, etc. Among them, the scanning range includes the range Lx×Ly (width × height) of the target area and the scanning range of the z-axis. The neural network model used for target detection of the scanned image set is the yolov8 model. The scoring criteria of the clarity scoring algorithm are used in the calculation of the clarity scoring value. After target detection, multiple frames of scanned images within the range near the detected target position are scored.

[0079] After the settings are completed, start the control program of the sample stage and connect the image acquisition device of the microscope (i.e., the high-resolution camera). Then, load the historical focus dataset, and determine the direction adjustment and path sequence of the target path according to the historical focus dataset.

[0080] (2) Target detection.

[0081] Based on the target path, determine an (x, y) position. At this position, collect one or more frames of scanned images by performing a fast up-and-down scan along the z-axis (e.g., within the range of [z_min, z_max], which can take values of [0, 1500μm]) to obtain a set of scanned images. Then, perform target detection on the set of scanned images to determine whether there is a target sample at this (x, y) position.

[0082] If there is no target sample, skip the current (x, y) position, move to the next (x, y) position, and continue the fast scan. On this basis, if no target sample is detected at multiple consecutive points (e.g., Njump is set to 5), dynamically determine this area as an invalid area (corresponding to Njump = 5 above, and the range of the invalid area can be 300μm × 5 = 1500μm. Note that the default point spacing is 300μm), and directly skip the invalid area to reduce the number of invalid scans. The skipped range is dynamically adjusted by prior information or the sample distribution density.

[0083] If there is a target sample, stop the scan, and output the existence judgment result of the sample as an existence result. Then, extract the approximate z-axis range (e.g., z_focus_initial ± 50μm, with the default Δz being 50μm) of the area where the sample is located, that is, the existence position area. On this basis, enter the next step to calculate the clarity score value of the scanned images within the existence position area.

[0084] (3) Clarity scoring.

[0085] For the (x, y) position point, collect multiple frames of scanned images within the range of z_focus_initial ± 50μm. For each frame of scanned image collected, calculate the corresponding clarity score value using the gradient algorithm, Laplace operator, and high-frequency component analysis, and then perform weighted fusion to obtain the clarity score value of this scanned image. Compare the clarity score values of the multiple frames of scanned images collected above, and select the z value corresponding to the scanned image with the highest clarity score value as the final focal plane position (z_focus_final).

[0086] (4) Adaptive search.

[0087] After obtaining the final focal plane position at the (x, y) location, this position can be considered as a valid point, and the number of valid points is recorded and incremented by one. If multiple valid points are continuously detected, it can be considered that the target samples are densely distributed. On this basis, the point spacing at the (x, y) location can be dynamically reduced to ensure accurate detection of each sample point and optimize the efficiency. If the current point spacing is defaulted to 300μm, it can be further reduced to 100μm.

[0088] If there are no target samples at several (x, y) locations, or in other words, the sample distribution is sparse (such as during the first scan or when the sample distribution is unknown), the point spacing at the (x, y) location can be dynamically increased, gradually expanding from a small range to a medium range, and then to a large range until the specified scanning area is covered, thereby reducing unnecessary scanning times.

[0089] (5) Scanning completed.

[0090] Perform the focusing process for all (x, y) locations according to the target path and output the focusing results, including: the position of the final focal plane at each (x, y) location. On this basis, the above output can be saved in a standard format (such as JSON or CSV) for other systems to call or for users to further analyze.

[0091] (6) Memory optimization.

[0092] After the scanning of the target path is completed, record and save the valid points, the number of valid points, the invalid points, the number of invalid points, and the position of the final focal plane at each (x, y) location during this scanning process, which can be used as historical data for subsequent scans to optimize the subsequent target path.

[0093] When using the historical data to optimize the subsequent target path, analyze the change trend of the scanning area of the focal plane position, the invalid area, and the density and sparsity. The change trend is that the focal plane gradually moves up or down. According to this change trend, predict the priority range for the next focusing. For the invalid area, skip it to avoid repeated scanning of the area without samples, thereby further improving the focusing efficiency. Based on the density and sparsity, give priority to processing the areas where the samples are densely distributed to reduce the global scanning time.

[0094] In a specific practical application, corresponding to step (1) above, the data is set as follows: The range (x, y) of the target area is a 5 mm × 5 mm area, and the initial value of the scanning point spacing within the target area is 300 μm. The scanning range of the z-axis is [0, 1500 μm] to ensure coverage of the possible focal plane range. The detection threshold during target detection is 0.8. The serpentine scanning path is selected. The scoring criterion of the clarity scoring algorithm is: Compare the focal plane clarity within z_focus_initial ± 50 μm. On this basis, steps (2) - (5) are completed to conduct the experiment. It can be seen that this application demonstrates significant technical advantages in the field of microscope autofocus, mainly reflected in five aspects: scanning time, focusing accuracy, sample coverage rate, adaptability, and usage cost:

[0095] (1) Traditional microscope autofocus systems usually adopt full-range linear scanning or multi-point sampling methods. Full-range linear scanning takes a long time, requiring 10 - 20 seconds for each step-by-step scan. Although multi-point sampling and single clarity optimization methods are slightly faster, they still take 8 - 15 seconds to complete a single focus, with limited efficiency, especially performing poorly when the sample distribution is complex or the focusing range is large.

[0096] Through the synergistic effect of double judgment and adaptive search, this application can quickly screen out the sample area and accurately lock the focal plane through clarity scoring, thereby reducing ineffective operations. The total scanning time of this application is 25 minutes, saving 40% of the time compared to the 42 minutes of the traditional control group; the single focus time is shortened to 5 - 8 seconds, achieving a 50% - 60% speed increase. This performance benefits from the target detection function quickly locking the sample area to avoid ineffective scanning, while the adaptive search dynamically adjusts the search path, preferentially and precisely locating the focal plane within a small range, avoiding ineffective search operations in traditional methods, significantly shortening the focusing time, and thus significantly improving the overall focusing efficiency.

[0097] (2) Traditional technologies usually rely on a single clarity algorithm (such as the gradient algorithm), resulting in a focal plane positioning accuracy of only ±2 - 4 μm and a repeated positioning accuracy of approximately 4 μm, which is difficult to meet the requirements of high-resolution microscopic imaging. This application, through a multi-feature clarity scoring algorithm that integrates the gradient algorithm, Laplace operator, and high-frequency component analysis, can quickly and accurately determine the focal plane position, improving the focal plane positioning accuracy to ±0.5 μm and the repeated positioning accuracy to better than 1 μm, with an accuracy improvement of 50% - 75%. In addition, the stability of repeated positioning is further improved by analyzing historical data, ensuring the robustness and reliability of the system in high-precision scenarios. In addition, the sample coverage rate of this application is 100%, and all sample points are detected and focused without omission.

[0098] (3) The success rate of traditional autofocus systems is usually 85% - 90%, and the misjudgment rate is as high as 5% - 10%. Especially in the sample environment with low contrast or complex distribution, missed detections and misjudgments are quite common. Through the collaborative work of dual judgments in this application, that is, jointly judging the focal plane position through object detection and clarity scoring, the accuracy of sample recognition has been greatly improved, achieving a focusing success rate of 95% - 98% and reducing the misjudgment rate to 1% - 2%. Specifically, the sample area is quickly screened through object detection to avoid missed detections, and the calculation of the clarity scoring value significantly improves the accuracy of focal plane positioning by means of a multi-feature fusion algorithm, thus still performing excellently in a complex sample environment. Meanwhile, trend analysis using historical data effectively reduces misjudgments and ensures the stability of the focusing operation.

[0099] (4) Traditional systems mostly adopt fixed parameters and a single scanning strategy, making it difficult to meet diverse experimental requirements. They are less efficient when the sample distribution is dense and waste a large amount of scanning time under sparse distribution conditions. Through the synergistic effect of adaptive search and memory optimization in this application, it demonstrates strong adaptability and can adapt to various sample types (such as cells, tissue sections, particle samples, etc.), various experimental environments (such as different lighting conditions, microscope parameters, etc.), and different distribution characteristics (dense, sparse). With the processing of adaptive search, the scanning spacing and search range are dynamically adjusted according to the sample distribution characteristics to adapt to various sample distributions from dense to sparse; by recording the sample distribution pattern and the changing trend of the focal plane position, the scanning path is optimized and repeated operations are reduced, thus significantly improving the applicability of the system under complex experimental conditions. It has strong environmental adaptability and can handle different lighting conditions and microscope magnifications.

[0100] (5) Traditional autofocus systems are complex to operate, rely on manual parameter adjustment, and require 2 - 3 days of training time, with high labor costs and high usage thresholds. Through the full-process automated design in this application, manual intervention is effectively reduced. Specifically, adaptive search and memory optimization achieve fully automated operation from sample detection to focal plane positioning, greatly reducing the usage complexity. Compared with traditional systems, the labor cost is reduced by 70% and the training time is shortened by 60%. At the same time, the modular design also makes this invention easy to integrate and deploy, significantly reducing the user's learning cost and usage threshold.

[0101] In summary, through the collaborative innovation of dual judgments, adaptive search, and memory optimization in this application, excellent performance advantages are demonstrated in terms of focusing speed, positioning accuracy, success rate, adaptability, and usage cost. Compared with the prior art, it not only significantly improves the efficiency and reliability of microscope autofocus but also reduces the usage threshold, has broad application value and commercial potential, and has significant advantages in the field of microscope autofocus.

[0102] Based on the same inventive concept, an embodiment of the present application further provides a microscope intelligent autofocus system. The implementation solution provided by this system to solve the problem is similar to the implementation solution described in the above method. Therefore, the specific limitations in one or more system embodiments provided below can refer to the limitations on the method in the above text, and will not be repeated here. The system of the present application includes the following multiple modules.

[0103] A scanning image acquisition module, configured to: acquire a set of scanning images; the set of scanning images is a set of scanning images obtained at any path point when a microscope component scans and images a target area according to a target path; the target area is distributed with target samples; different scanning images correspond to different focal planes.

[0104] An adaptive search module, configured to: perform target detection on the set of scanning images by using an adaptive search algorithm to obtain a detection result; the detection result includes an existence result and an existence position area.

[0105] A sharpness scoring module, configured to: if the existence result indicates that there are target samples in the set of scanning images, record that the number of valid points is incremented by one, extract multiple frames of scanning images based on the existence position area, and calculate the sharpness scoring value of each frame of scanning image extracted based on multiple image features.

[0106] A focal plane determination module, configured to: select the scanning image with the highest sharpness scoring value, and mark the corresponding focal plane as the final focal plane of the path point.

[0107] The adaptive search module is further configured to: use an adaptive search algorithm to adjust the point spacing according to the number of valid points, the path point, and the target path to determine the next path point, and then return to the scanning image acquisition module; if the existence result indicates that there are no target samples in the set of scanning images, record that the number of invalid points is incremented by one, use an adaptive search algorithm to adjust the point spacing according to the number of invalid points, the path point, and the target path to determine the next path point, and then return to the scanning image acquisition module.

[0108] The system of the present application can achieve modular design, is easy to integrate, and is applicable to various application scenarios of microscope autofocus, such as life science, material research, and industrial inspection. The system of the present application has high efficiency and robustness, can adapt to the diverse characteristics of complex samples, ensure the accurate selection of the focal plane; can quickly screen the sample area and accurately locate the focal plane, significantly improving the efficiency and accuracy of autofocus.

[0109] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 3As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements the microscope intelligent autofocus method.

[0110] Those skilled in the art can understand that Figure 3 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0111] In an exemplary embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. The processor executes the computer program to implement the microscope intelligent autofocus method. This computer device is of great significance to the autofocus technology and can effectively improve the automation level of sample detection.

[0112] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above method embodiments.

[0113] In an exemplary embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in the above method embodiments.

[0114] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0116] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0117] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0118] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A microscope intelligent automatic focusing method, characterized in that: Methods include: Acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area along the target path; Target samples are distributed in the target area; Different scanned images correspond to different focal planes; The process of determining the target path includes: obtaining a historical focus data set; any historical focus data in the historical focus data set includes a historical path point; evenly dividing the target area into a plurality of grid areas; for each of the grid areas, based on the historical focus data set, counting the total number of historical path points in the grid area and the number of historical path points recorded as valid points, and calculating a sample detection rate; based on the sample detection rate, determining a detection priority of the grid area; based on a preset priority scanning rule, determining the target path according to the detection priority of each of the grid areas; Adopting an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area; If the existence result indicates that the target sample exists in the scanned image set, the effective point number is recorded plus one, multiple frames of scanned images are extracted based on the existence position area, and the clarity score value of each frame of the scanned image extracted is calculated based on the image multi-features; Select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point; Adopting an adaptive search algorithm, adjusting the point spacing according to the number of valid points, the path points and the target path to determine the next path point, and then returning to the step of obtaining the scanned image set; If the existence result indicates that the target sample does not exist in the scanned image set, the number of invalid points is recorded plus one, and an adaptive search algorithm is used to adjust the point spacing according to the number of invalid points, the path points and the target path to determine the next path point, and then return to the step of obtaining the scanned image set.

2. The microscope intelligent autofocus method according to claim 1, characterized in that: Adopting an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result, including: Extracting a scan image subset from the scan image set according to a preset range and a preset scan interval; For the scanned images in the scanned image subset, using a target detection algorithm to perform target detection one by one to obtain a single image detection result; If at least one of the single image detection results indicates the presence of the target sample, determining the corresponding presence location area, and generating a presence result indicating the presence of the target sample in the scanned image set; If all the single image detection results indicate that the target sample does not exist, the preset range is updated, the preset scanning interval is updated, and the number of updates is recorded plus one; If the number of updates reaches the preset number of updates, it is determined that the existing position area is empty, and an existence result indicating that the target sample does not exist in the scanned image set is generated; If the update number does not reach the preset update number, the method returns to the step of extracting a scan image subset from the scan image set according to a preset range and a preset scanning interval.

3. The microscope intelligent autofocus method according to claim 2, characterized in that: The preset range is updated, and the preset scanning interval is updated, specifically: the preset small range is updated to the preset medium range, and the preset small range scanning interval is updated to the preset medium range scanning interval; Alternatively, the preset range is updated, and the preset scanning interval is updated, specifically: the preset medium range is updated to the preset large range, and the preset medium range scanning interval is updated to the preset large range scanning interval.

4. The microscope intelligent autofocus method according to claim 2, characterized in that: The process of determining the preset range includes: Acquire a historical focus data set; any historical focus data in the historical focus data set includes a historical path point and a corresponding historical final focal plane; Based on the historical focus data set, determining a historical focus position change trend and calculating a trend offset; The preset range is determined according to the trend offset, the final focal plane corresponding to the previous path point and the preset search range.

5. The microscope intelligent autofocus method according to claim 1, characterized in that: The clarity score of each scanned image frame is calculated and extracted based on multiple image features, including: For any frame scanned image extracted, a gradient algorithm is used to calculate the gradient clarity score value; The Laplace operator is used to calculate the Laplace clarity score through the second-order derivative; High-frequency component analysis was used to calculate the high-frequency component clarity score through Fourier transform; The gradient clarity score value, the Laplace clarity score value and the high-frequency component clarity score value are weightedly calculated to obtain a clarity score value of the scanned image.

6. The microscope intelligent autofocus method according to claim 5, characterized in that: The calculation formula of the gradient clarity score is: ； Where G(I) is the gradient clarity score of the scanned image I, is the grayscale gradient of the pixel (x, y) of the scanned image I in the horizontal direction, is the grayscale gradient of the pixel (x, y) of the scanned image I in the vertical direction; To accumulate all the pixels of the scanned image I; The calculation formula of the Laplace clarity score is: ； Where L(I) is the Laplace clarity score of the scanned image I, is the second-order derivative of the pixel (x, y) of the scanned image I in the horizontal direction, is the second-order derivative of the pixel (x, y) of the scanned image I in the vertical direction; The calculation formula of the high-frequency component clarity score is: ； in, , H(I) is the high-frequency component clarity score of the scanned image I, F ( u , v ) is the Fourier transform result of the scanned image I in the frequency domain, is the energy of the high-frequency component of the scanned image I in the frequency domain, HF is the range of the high-frequency component, is the Fourier transform operation, The energy of the high-frequency components of the scanned image I in the frequency domain is accumulated.

7. The microscope intelligent autofocus method according to claim 1, characterized in that: Adopting an adaptive search algorithm, adjusting the point spacing according to the number of valid points, the path points and the target path to determine the next path point, including: If the valid points are within a preset valid point range, the point spacing is reduced, and the next path point is determined according to the reduced point spacing, the path point and the target path; Adopting an adaptive search algorithm, adjusting the point spacing according to the number of invalid points, the path points and the target path to determine the next path point, including: If the invalid points are within a first preset invalid points range, increasing the point spacing, and determining the next path point according to the increased point spacing, the path point and the target path; If the invalid points are within a second preset invalid points range, a next path point is determined based on a preset jump amplitude, the path point and the target path.

8. An intelligent automatic focusing system for a microscope, characterized in that: The system includes: The scanning image acquisition module is used to: acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area according to the target path; Target samples are distributed in the target area; Different scanned images correspond to different focal planes; The process of determining the target path includes: obtaining a historical focus data set; any historical focus data in the historical focus data set includes a historical path point; evenly dividing the target area into a plurality of grid areas; for each of the grid areas, based on the historical focus data set, counting the total number of historical path points in the grid area and the number of historical path points recorded as valid points, and calculating a sample detection rate; based on the sample detection rate, determining a detection priority of the grid area; based on a preset priority scanning rule, determining the target path according to the detection priority of each of the grid areas; An adaptive search module, used to: use an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area; A clarity scoring module is used to: if the existence result indicates that the target sample exists in the scanned image set, record the valid points plus one, extract multiple frames of scanned images based on the existence position area, and calculate the clarity score value of each frame of the scanned image extracted based on multiple features of the image; A focal plane determination module, configured to: select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point; The adaptive search module is also used to: use an adaptive search algorithm to adjust the point spacing according to the valid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module; if the existence result indicates that the target sample does not exist in the scanning image set, then record the invalid points plus one, use an adaptive search algorithm to adjust the point spacing according to the invalid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the microscope intelligent autofocus method according to any one of claims 1 to 7.

Images