Microscope intelligent automatic focusing method, system and equipment

Abstract

The invention discloses an intelligent automatic focusing method, system and device for a microscope, and relates to the field of microscope focusing, and the method comprises the steps: carrying out the target detection of a scanning image set through an adaptive search algorithm, so as to obtain a detection result; if the target sample exists in the scanning image set, extracting multiple frames of scanning images based on an existing position area, and calculating a definition score value of each frame of scanning image obtained through extraction based on image multi-feature calculation; selecting the scanning image with the highest definition score value, and marking the corresponding focal plane as the final focal plane of the path point; determining a next path point by adopting a self-adaptive search algorithm, and then returning to the step of acquiring the scanning image set; and if the result shows that the target sample does not exist in the scanning image set, determining the next path point by adopting the adaptive search algorithm, and then returning to the step of acquiring the scanning image set. According to the invention, the automatic focusing efficiency, precision and adaptability of the microscope can be improved.

Description

Technical Field

[0001] The present application relates to the field of microscope focusing, and in particular to a microscope intelligent automatic focusing method, system and equipment. Background Art

[0002] In the field of biochip analysis, the focus accuracy requirements for sample detection under a microscope are extremely strict. Since the chip sample size is usually at the micron level, manual focusing under a traditional microscope is not only time-consuming and labor-intensive, but also difficult to ensure the consistency of focus accuracy. Especially under high-power microscopes, the depth of focus range is smaller and the focusing difficulty is greater. A slight deviation will cause image blur, affecting the accuracy of subsequent analysis results.

[0003] Conventional autofocus technology mainly uses global scanning, which has many problems when processing chip samples: global scanning takes a long time, reducing detection efficiency; due to the uneven distribution of samples on the chip surface, it is easy to be disturbed, resulting in unstable positioning accuracy; lack of intelligent adaptive capabilities, it is difficult to cope with complex and changing sample conditions. These problems are more prominent under high-power microscopes, and the focus accuracy requirements can reach sub-micron levels, which poses higher technical challenges to the autofocus system. Summary of the invention

[0004] The purpose of this application is to provide a microscope intelligent autofocus method, system and equipment, which can improve the efficiency, accuracy and adaptability of microscope autofocus.

[0005] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a microscope intelligent autofocus method, comprising: Acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area according to the target path; the target area is distributed with target samples; different scanning images correspond to different focal planes; Adopting an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area; If the existence result indicates that the target sample exists in the scanned image set, the effective point number is recorded plus one, multiple frames of scanned images are extracted based on the existence position area, and the clarity score value of each frame of the scanned image extracted is calculated based on the image multi-features; Select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point; Adopting an adaptive search algorithm, adjusting the point spacing according to the number of valid points, the path points and the target path to determine the next path point, and then returning to the step of obtaining the scanned image set; If the existence result indicates that the target sample does not exist in the scanned image set, the number of invalid points is recorded plus one, and an adaptive search algorithm is used to adjust the point spacing according to the number of invalid points, the path points and the target path to determine the next path point, and then return to the step of obtaining the scanned image set.

[0006] In a second aspect, the present application provides a microscope intelligent autofocus system, comprising: The scanning image acquisition module is used to: acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area according to the target path; the target area is distributed with target samples; different scanning images correspond to different focal planes; An adaptive search module, used to: use an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area; A clarity scoring module is used to: if the existence result indicates that the target sample exists in the scanned image set, record the valid points plus one, extract multiple frames of scanned images based on the existence position area, and calculate the clarity score value of each frame of the scanned image extracted based on multiple features of the image; A focal plane determination module, configured to: select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point; The adaptive search module is also used to: use an adaptive search algorithm to adjust the point spacing according to the valid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module; if the existence result indicates that the target sample does not exist in the scanning image set, then record the invalid points plus one, use an adaptive search algorithm to adjust the point spacing according to the invalid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module.

[0007] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an intelligent autofocus method for a microscope.

[0008] According to the specific embodiments provided by the present application, the present application has the following technical effects: the present application provides a microscope intelligent automatic focusing method, system and device, which uses an adaptive search algorithm to perform target detection on a scanned image set, quickly determines whether the target sample is included in the field of view of the current path point, and obtains the existence result and the existence position area, thereby avoiding wasting computing resources in invalid areas. Then, when the target sample exists, the clarity score value of the scanned image in the existence position area is calculated, and the final focal plane is determined according to the clarity score value, which can quickly screen the sample area and accurately locate the focal plane, ensuring the efficiency and accuracy of the microscope automatic focusing. When there is no target sample or the focal plane has been determined, the adaptive search algorithm is used to determine the next path point. In this process, the computational overhead can be reduced and the screening efficiency can be improved. The adaptive search algorithm is used multiple times in the present application to avoid invalid search operations, shorten the focusing time, and improve the focusing speed. In summary, the present application can achieve efficient and accurate automatic focusing, significantly improve the focusing speed, accuracy and success rate, and have the advantages of high adaptability and low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 This is a diagram of the application environment of an intelligent autofocus method for a microscope in one embodiment of the present application.

[0011] Figure 2 A schematic flow chart of an intelligent autofocus method for a microscope provided in one embodiment of the present application.

[0012] Figure 3 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The present application provides a microscope intelligent autofocus method, system and equipment, which can improve the accuracy of sample identification and positioning, optimize the focus search strategy to improve efficiency, and enhance the system's adaptability and stability. In practical applications, the present application can also significantly improve the intelligence level and automation of focusing by introducing deep learning to achieve target detection, adaptive search algorithms and other advanced technologies, combined with memory optimization mechanisms, and achieve sub-micron focusing accuracy. The present application also reduces manual intervention, improves detection throughput and result reliability. Especially in terms of the precise focusing requirements under high-power microscopes, it can break through the limitations of traditional technologies, provide more reliable and efficient technical support for biochip detection, and has important practical application value.

[0015] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0016] The microscope intelligent autofocus method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the scanned image set to the server 104. After the server 104 receives it, it uses the dual judgment mechanism (target detection and clarity scoring) as the core to quickly screen the sample area and accurately focus on the final focal plane of each path point. The server 104 can feedback the final focal plane of each path point to the terminal 102.

[0017] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0018] In an exemplary embodiment, Figure 2 As shown, a microscope intelligent autofocus method is provided, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 206.

[0019] Step 201, obtaining a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area along the target path; target samples are distributed in the target area; different scanning images correspond to different focal planes.

[0020] In a practical application, the construction of the target path needs to be implemented in combination with a memory optimization algorithm, that is, by analyzing the historical focus data set, the current search path and range to be completed are optimized to improve the overall focus efficiency. Correspondingly, in step 201, before starting scanning imaging, the target path is optimized according to the historical sample distribution law to give priority to processing sample-dense areas and reduce the scanning of invalid areas, thereby improving efficiency. Specifically, the process of determining the target path includes: (11) Obtaining a historical focus data set; any historical focus data in the historical focus data set includes a historical path point. If the three-dimensional coordinate system of xyz is used as a reference, the historical path point is represented by (x, y), and the position of the historical final focus plane corresponding to the historical path point is represented by the z value.

[0021] (12) The target area is uniformly divided into a plurality of grid areas; for example, the target area is set to Lx×Ly (width×height), and the entire target area is divided into Nx×Ny (horizontal×vertical) uniform grid areas according to a fixed size, so that the size of each grid area is: Δx=Lx / Nx, Δy=Ly / Ny.

[0022] (13) For each of the grid areas, based on the historical focus data set, the total number of historical path points in the grid area and the number of historical path points recorded as valid points are counted, and the sample detection rate is calculated. Specifically, the statistics of the sample historical detection situation are realized by counting the number of path points, and then the sample detection rate R_detect is calculated using the following formula: R_detect = the number of historical path points recorded as valid points / the total number of historical path points × 100%.

[0023] (14) Based on the sample detection rate, determine the detection priority of the grid area; in an actual application, the detection priority can be marked and determined according to the following settings: when R_detect>80%, the grid area is a high detection rate area, also a high priority area, and it is necessary to scan these grid areas first as the starting point of the target path; when 20%≤R_detect≤80%, the grid area is a medium detection rate area, also a medium priority area, and these grid areas are processed after the high priority area is scanned; when R_detect<20%, the grid area is a low detection rate area, also a low priority area, and it is necessary to postpone scanning or directly skip these grid areas.

[0024] (15) Based on the preset priority scanning rules, the target path is determined according to the detection priority of each grid area. When actually determining the target path, it can be processed from two aspects: path order and direction adjustment. In terms of path order, high priority areas are covered first, and then scanned in order from high to low priority. In terms of direction adjustment, if high priority areas are concentrated in a certain area, scanning is started from these areas first to avoid redundancy of the global serpentine scanning path.

[0025] In summary, in the process of determining the target path, the historical focus position change trend (such as the focus position gradually moving up or down) is analyzed based on the historical focus data set, and the priority direction of the current search to be completed is predicted based on these trends. Especially after the analysis, when the sample distribution pattern is obvious, the target path is dynamically adjusted to prioritize the sample-dense areas and reduce the scanning of invalid areas to improve efficiency.

[0026] Step 202, using an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area.

[0027] Combined with the memory optimization algorithm, in the target detection step after scanning imaging, the current possible focus range can be predicted based on the change trend of the historical final focus in the historical focus data set, thereby reducing the exploration of invalid z values. Among them, predicting the current possible focus range can obtain the preset range, and the process of determining the preset range includes: 1) Obtaining a historical focus data set; any historical focus data in the historical focus data set includes a historical path point and a corresponding historical final focus plane.

[0028] 2) Based on the historical focus data set, determine the historical focus position change trend and calculate the trend offset; specifically, if the historical focus position gradually increases, it means that the focus is moving upward, and the historical focus position change trend is upward; if the historical focus position gradually decreases, it means that the focus is moving downward, and the historical focus position change trend is downward. In practical applications, each path point corresponds to a trend offset, and the trend offset corresponding to each path point (such as a historical path point) can be calculated based on several consecutive historical path points with the same historical focus position change trend, and the average change of the z value of the corresponding historical final focus is used as the trend offset Δz_trend, such as a value of ±5μm.

[0029] 3) Determine the preset range according to the trend offset, the final focal plane corresponding to the previous path point, and the preset search range. Specifically, the preset range can be expressed as follows: [z_last+Δz_trend−Δz,z_last+Δz_trend+Δz], where z_last is the z value of the final focal plane of the previous path point, z_last+Δz_trend=z_focus_initial, and Δz is the preset search range set by relevant technical personnel, such as a small range of ±50μm. The search range can be adjusted as needed, such as a medium range of ±150μm and a large range of ±1500μm.

[0030] After obtaining the preset range, an adaptive search algorithm is used to preferentially search for the focal plane within a small range. If the small range search fails to find the best focal plane position (i.e., the final focal plane in the following text), the search range is gradually expanded to a medium range or a full range. By dynamically adjusting the search range, the focusing efficiency can be greatly improved. In a specific application, step 202 includes the following steps (21)-(26).

[0031] (21) Extracting a scan image subset from the scan image set according to a preset range and a preset scan interval.

[0032] (22) For the scanned images in the scanned image subset, a target detection algorithm is used to perform target detection one by one to obtain a single image detection result. The target detection algorithm used may be a target detection method based on deep learning to identify sample features.

[0033] (23) If at least one of the single image detection results indicates the presence of the target sample, a corresponding existence location area is determined, and an existence result indicating the presence of the target sample in the scanned image set is generated.

[0034] (24) If all of the single image detection results indicate that the target sample does not exist, the preset range is updated, the preset scanning interval is updated, and the number of updates is recorded plus one.

[0035] (25) If the update number reaches the preset update number, it is determined that the existence position area is empty, and an existence result is generated indicating that the target sample does not exist in the scanned image set.

[0036] (26) If the update number does not reach the preset update number, return to step (21).

[0037] In another practical application, the preset range is updated, and the preset scanning interval is updated, specifically: the preset small range is updated to the preset medium range, and the preset small range scanning interval is updated to the preset medium range scanning interval; or the preset medium range is updated to the preset large range, and the preset medium range scanning interval is updated to the preset large range scanning interval. The difference between the small range, the medium range, and the large range is the difference in the preset search range Δz.

[0038] Through the above update, a step-by-step search from small to medium to large can be achieved, and the approximate z-axis position of the sample can be found more quickly. Specifically, within the preset small range [z_focus_initial−Δz1, z_focus_initial+Δz1], images are collected according to the preset small range scanning spacing Δz_small (the value can be 0.5 μm), and multiple scanned images are obtained. The target detection algorithm is used to perform target detection one by one to determine whether there is a sample; if a sample is detected from multiple scanned images, the approximate z-axis position of the sample is recorded, and a result representing the existence of the target sample in the scanned image set is generated, and the search is ended; if no sample is detected from multiple scanned images, then after completing the scan within the preset small range, the preset small range is updated to the preset medium range [z_focus_initial−Δz2, z_focus_initial+Δz2], and the preset small range scanning spacing is updated to the preset medium range scanning spacing.

[0039] Within the preset medium range, images are collected according to the preset medium range scanning spacing Δz_medium (the value can be 2 μm) to obtain multiple scanned images, and the target detection algorithm is used to perform target detection one by one to determine whether there is a sample; if a sample is detected from multiple scanned images, the approximate z-axis position of the sample is recorded, and a result representing the presence of the target sample in the scan image set is generated, and the search is ended; if no sample is detected from multiple scanned images, after completing the scan within the preset medium range, the preset medium range is updated to the preset large range [z_min, z_max], and the preset medium range scanning spacing is updated to the preset large range scanning spacing.

[0040] Within a preset large range, images are collected according to a preset large-range scanning spacing Δz_large (which can take a value of 10 μm) to obtain multiple scanned images, and a target detection algorithm is used to perform target detection one by one to determine whether there is a sample; if a sample is detected from multiple scanned images, the approximate z-axis position of the sample is recorded, and an existence result is generated to characterize the existence of the target sample in the scanned image set, and the search is ended; if no sample is detected from the multiple scanned images, it is determined that the existence position area is empty, and a existence result is generated to characterize the absence of the target sample in the scanned image set, which can also be displayed to relevant technical personnel.

[0041] In the above preset small range, preset medium range or preset large range, the approximate z-axis position z_focus_initial recorded when the sample is detected can be characterized by a certain z-axis value, and then the existence position area can be determined in combination with the scoring criteria of the clarity scoring algorithm. Among them, the scoring criteria of the clarity scoring algorithm are set by relevant technical personnel, and according to the scoring criteria, the multiple frames of scanned images that can be collected are scored, so as to find the scanned image with the highest clarity score, and the final focus position (z_focus_final) of the path point is determined accordingly.

[0042] For example, the existing position area is set as [z_focus_initial−Δz, z_focus_initial+Δz], where Δz is a fixed scanning spacing, which can be set to 50μm according to the scoring standard. Within the above range, multiple frames of scanned images are collected, and the clarity scoring algorithm in the subsequent steps (such as gradient algorithm, Laplace operator, high-frequency component analysis, etc.) is used to score each frame of the scanned image, and the corresponding clarity score value is to be tested.

[0043] Step 203: If the existence result indicates that the target sample exists in the scanned image set, the effective point number is recorded plus one, multiple frames of scanned images are extracted based on the existence position area, and the clarity score value of each frame of the scanned image extracted is calculated based on the image multi-features.

[0044] Among them, after extracting multiple frames of scanned images at corresponding positions from the scanned image set, the clarity score value of each frame of the scanned image is extracted based on the image multi-feature calculation. The calculation of the clarity score value can further analyze the selected multiple frames of scanned images to accurately locate the focal plane. The step of extracting the clarity score value of each frame of the scanned image based on the image multi-feature calculation can be calculated based on a gradient algorithm, a Laplace operator or a high-frequency component analysis technology, and includes the following steps (31)-(34).

[0045] (31) For any frame scanned image extracted, a gradient algorithm is used to calculate a gradient clarity score. The gradient algorithm evaluates clarity by calculating the sum of the gradient amplitudes of the pixel intensity changes in the image. The calculation formula for the gradient clarity score is: .

[0046] Where G(I) is the gradient clarity score of the scanned image I, is the grayscale gradient of the pixel (x, y) of the scanned image I in the horizontal direction, is the grayscale gradient of the pixel (x, y) of the scanned image I in the vertical direction; To accumulate all the pixels of the scanned image I.

[0047] (32) The Laplace operator is used to calculate the Laplace clarity score value through the second-order derivative; wherein the Laplace operator is the second-order derivative of the image and is used to detect the sharpness of the image edge. The calculation formula of the Laplace clarity score value is: .

[0048] Where L(I) is the Laplace clarity score of the scanned image I, is the second-order derivative of the pixel (x, y) of the scanned image I in the horizontal direction, is the second-order derivative of the pixel (x, y) of the scanned image I in the vertical direction.

[0049] (33) High-frequency component analysis is used to calculate the high-frequency component clarity score value through Fourier transform; wherein the high-frequency component analysis is based on the Fourier transform (FFT) of the image and evaluates the clarity by calculating the energy of the high-frequency area of ​​the image. The calculation formula of the high-frequency component clarity score value is: .

[0050] in, , H(I) is the high-frequency component clarity score of the scanned image I, F ( u , v ) is the Fourier transform result of the scanned image I in the frequency domain, is the energy of the high-frequency components of the scanned image I in the frequency domain, HF is the range of the high-frequency components (low-frequency components are filtered by setting a frequency threshold), is the Fourier transform operation, The energy of the high-frequency components of the scanned image I in the frequency domain is accumulated.

[0051] (34) The gradient clarity score value, the Laplace clarity score value and the high-frequency component clarity score value are weighted to obtain a clarity score value of the scanned image, specifically using the following formula: ; Where S(I) is the clarity score of the scanned image I, is the weight (adjusted according to experiments or specific application scenarios).

[0052] Step 204, select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point. Through the above steps (31)-(34), the gradient algorithm, Laplace operator, high-frequency component analysis, and weighted fusion calculation can be applied to any frame scanned image to obtain the corresponding clarity score; then compare the clarity scores of all frame scanned images, and select the z value corresponding to the frame with the highest score as the final focal plane position. . I i Scan the image for the i-th frame.

[0053] Step 205 , using an adaptive search algorithm, adjusting the point spacing according to the valid points, the path points and the target path to determine the next path point, and then returning to step 201 .

[0054] In this step, an adaptive search algorithm is used to efficiently detect sample distribution and reduce exploration of invalid areas. In a specific application, step 205 includes: if the number of valid points is within a preset valid point range, the point spacing is reduced, and the next path point is determined based on the reduced point spacing, the path point and the target path.

[0055] Corresponding to step 202 above, in the application of the adaptive search algorithm in step 205, the small range point spacing, medium range point spacing and large range point spacing are also set accordingly. Specifically, if the sample is detected for N consecutive times (that is, the number of valid points is within the preset valid point number range), it means that the sample distribution is dense, and then the order of large range point spacing Δx_large, Δy_large→medium range point spacing Δx_medium, Δy_medium→small range point spacing Δx_small, Δy_small is gradually reduced to continue detailed scanning and improve detection accuracy. As for the value of the point spacing, corresponding to the large range, the value can be 1000 μm; corresponding to the medium range, the value can be 500 μm; corresponding to the small range, the value can be 300 μm.

[0056] In step 206, if the existence result indicates that the target sample does not exist in the scanned image set, the number of invalid points is recorded plus one, and an adaptive search algorithm is used to adjust the point spacing according to the number of invalid points, the path points and the target path to determine the next path point, and then return to step 201. In a specific application, an adaptive search algorithm is also used, and the step of determining the next path point in step 206 includes steps (61)-(62).

[0057] (61) If the number of invalid points is within the first preset invalid point range, the point spacing is increased, and the next path point is determined based on the increased point spacing, the path point, and the target path. That is, if no sample is detected for Nmiss times (corresponding to the first preset invalid point range), it means that the current area may be an invalid area and the point spacing needs to be increased. Corresponding to step 205 above, the order of increase can be small range point spacing Δx_small, Δy_small → medium range point spacing Δx_medium, Δy_medium → large range point spacing Δx_large, Δy_large.

[0058] (62) If the number of invalid points is within the second preset invalid point range, the next path point is determined based on the preset jump amplitude, the path point and the target path. That is, if no sample is detected for Njump (corresponding to the second preset invalid point range) times, in the current case, the point spacing is already a large-range point spacing, and the path point can be determined to be in an invalid area. It can be skipped directly, and point-by-point scanning is restarted after skipping, and the small-range point spacing is restored to determine the next path point, and the next path point is searched and focused on the z-axis (see the processing of steps 201-204 above). Among them, the preset jump amplitude Δjump is usually a multiple of the current point spacing, such as 2×Δx_large. Among them, Njump is greater than Nmiss, that is, the lower limit value of the second preset invalid point range is greater than or equal to the upper limit value of the first preset invalid point range.

[0059] In another exemplary embodiment of the present application, a known cell sample is observed through a microscope assembly, and the distribution of cell sample characteristics is locally dense and locally sparse. The microscope assembly includes a microscope and a sample stage. The microscope is equipped with a high-resolution camera. The sample stage can accurately control (x, y, z) three-dimensional movement and support serpentine scanning paths or spiral scanning. Based on the above application scenarios, the method of the present application can be used to perform intelligent autofocus of the microscope, including the following steps (i) to (vi).

[0060] 1. Initialization

[0061] Set the path, scanning range, neural network model used for target detection on the scanned image set, and scoring criteria for the clarity scoring algorithm. The scanning range includes the range of the target area Lx×Ly (width×height) and the scanning range of the z axis. The neural network model used for target detection on the scanned image set is the yolov8 model. The scoring criteria of the clarity scoring algorithm are used in the calculation of the clarity score value. After target detection, multiple frames of scanned images in the vicinity of the detected target position are scored.

[0062] After the setup is completed, the control program of the sample stage is started, and the image acquisition device (i.e., high-resolution camera) of the microscope is connected. Then, the historical focus data set is loaded, and the direction adjustment and path sequence of the target path are determined based on the historical focus data set.

[0063] (B) Target detection

[0064] According to the target path, an (x, y) position is determined. At this position, one or more scanned images are collected by fast up and down scanning of the z axis (e.g., within the range of [z_min, z_max], the value can be [0, 1500μm]) to obtain a scanned image set. Then, target detection is performed on the scanned image set to determine whether there is a target sample at the (x, y) position.

[0065] If the target sample does not exist, the current (x, y) position is skipped, and the next (x, y) position is moved to continue the fast scan. On this basis, if the target sample is not detected at multiple consecutive points (such as Njump value is 5), the area is dynamically judged as an invalid area (corresponding to the Njump value of 5 above, the range of the invalid area can be 300μm×5=1500μm, and the default point spacing is 300μm), and the invalid area is directly skipped to reduce the number of invalid scans. The skipped range is dynamically adjusted by prior information or sample distribution density.

[0066] If the target sample exists, the scan is stopped, and the sample existence judgment result is output as the existence result. Then the approximate z-axis range of the sample area (such as z_focus_initial±50μm, the default Δz is 50μm) is extracted, that is, the existence position area. On this basis, the next step is to calculate the clarity score value of the scanned image in the existence position area.

[0067] (3) Clarity rating

[0068] For the (x, y) position point, multiple frames of scanned images are collected within the range of z_focus_initial±50μm. For each frame of scanned image collected, the corresponding clarity score value is calculated using the gradient algorithm, Laplace operator, and high-frequency component analysis, and then weighted fusion is performed to obtain the clarity score value of the scanned image. Compare the clarity scores of the multiple frames of scanned images collected above, and select the z value corresponding to the scanned image with the highest clarity score as the final focal position (z_focus_final).

[0069] 4. Adaptive search.

[0070] After obtaining the final focal plane position of the (x, y) position, the position can be considered as a valid point, and the number of valid points plus one is recorded. If multiple valid points are found in succession, it can be considered that the target samples are densely distributed. On this basis, the point spacing of the (x, y) position can be dynamically reduced to ensure accurate detection of each sample point to optimize efficiency. If the current point spacing is 300μm by default, it can be further reduced to 100μm.

[0071] If there are several (x, y) positions where there are no target samples, or the sample distribution is sparse (such as the first scan or the sample distribution is unknown), the point spacing of the (x, y) positions can be dynamically increased, gradually expanding from a small range to a medium range, and then to a large range until the specified scanning area is covered, thereby reducing the number of unnecessary scans.

[0072] 5. Scanning completed.

[0073] Complete the focus processing of all (x,y) positions according to the target path, and output the focus results, including: the position of the final focal plane at each (x,y) position. On this basis, the above output can be saved in a standard format (such as JSON or CSV) for other system calls or users to further analyze.

[0074] (6) Memory optimization.

[0075] After the target path is scanned, the valid points, number of valid points, invalid points, number of invalid points, and the position of the final focal plane at each (x, y) position during the scanning process are recorded and saved, which can be used as historical data for subsequent scanning to optimize the subsequent target path.

[0076] When optimizing the subsequent target path as historical data, analyze the change trend, invalid area and density and sparseness of the focus position scanning area. The change trend is that the focus gradually moves up or down, and the priority range for the next focus is predicted based on the change trend. For invalid areas, skip them to avoid repeated scanning of areas without samples, thereby further improving the focus efficiency. Based on the density and sparseness, give priority to areas with dense sample distribution to reduce the global scanning time.

[0077] In a specific practical application, corresponding to step (I) above, the data settings are as follows: the range (x, y) of the target area is 5mm×5mm, and the initial value of the scanning point spacing in the target area is 300μm, and the scanning range of the z-axis is [0,1500μm] to ensure that the possible focal plane range is covered; the detection threshold for target detection is 0.8; the path selects the serpentine scanning path; the scoring standard of the clarity scoring algorithm is: compare the focus plane clarity within z_focus_initial±50 μm. On this basis, completing steps (II) to (V) to conduct experiments, it can be seen that this application has shown significant technical advantages in the field of microscope autofocus, mainly in five aspects: scanning time, focus accuracy, sample coverage, adaptability and use cost: (1) Traditional microscope autofocus systems usually use full-range linear scanning or multi-point sampling methods. Full-range linear scanning takes a long time, requiring 10-20 seconds for each step-by-step scan. Although multi-point sampling and single clarity optimization methods are slightly faster, they still take 8-15 seconds to complete a single focus, which is limited in efficiency, especially when the sample distribution is complex or the focus range is large.

[0078] Through the synergy of dual judgment and adaptive search, this application can quickly screen out the sample area and accurately lock the focal plane through clarity scoring, thereby reducing invalid operations. The total scanning time of this application is 25 minutes, which saves 40% of the 42 minutes of the traditional control group; the single focus time is shortened to 5-8 seconds, achieving a speed increase of 50%-60%. This performance is due to the target detection function that quickly locks the sample area to avoid invalid scanning. At the same time, the adaptive search dynamically adjusts the search path, giving priority to accurately locating the focal plane within a small range, avoiding invalid search operations in traditional methods, greatly shortening the focus time, and significantly improving the overall focus efficiency.

[0079] (2) Traditional technologies usually rely on a single clarity algorithm (such as a gradient algorithm), resulting in a focal plane positioning accuracy of only ±2-4μm and a repeat positioning accuracy of approximately 4μm, which is difficult to meet the needs of high-resolution microscopic imaging. This application uses a multi-feature clarity scoring algorithm that integrates a gradient algorithm, a Laplace operator, and high-frequency component analysis to quickly and accurately determine the focal plane position, thereby increasing the focal plane positioning accuracy to ±0.5μm, and the repeat positioning accuracy to better than 1μm, with an accuracy improvement of 50%-75%. In addition, the stability of repeat positioning is further improved by analyzing historical data, ensuring the robustness and reliability of the system in high-precision scenarios. In addition, the sample coverage of this application is 100%, and all sample points are detected and focused without omissions.

[0080] (3) The success rate of traditional autofocus systems is usually 85%-90%, and the misjudgment rate is as high as 5%-10%. Especially in low-contrast or complex sample environments, missed detections and misjudgments are more common. This application greatly improves the accuracy of sample identification through the collaborative work of dual judgments, that is, the focus position is jointly judged by target detection and clarity scoring, achieving a 95%-98% focus success rate and reducing the misjudgment rate to 1%-2%. Specifically, target detection is used to quickly screen sample areas to avoid missed detection, and the calculation of clarity scores significantly improves the accuracy of focus positioning with the help of multi-feature fusion algorithms, thereby still performing well in complex sample environments. At the same time, the use of historical data for trend analysis effectively reduces misjudgments and ensures the stability of the focus operation.

[0081] (4) Traditional systems mostly use fixed parameters and a single scanning strategy, which makes it difficult to cope with diverse experimental needs. They are less efficient when the samples are densely distributed, and waste a lot of scanning time under sparse distribution conditions. This application demonstrates strong adaptability through the synergy of adaptive search and memory optimization, and can adapt to a variety of sample types (such as cells, tissue sections, particle samples, etc.), a variety of experimental environments (such as different lighting conditions, microscope parameters, etc.) and different distribution characteristics (dense, sparse). Adaptive search processing is used to dynamically adjust the scanning spacing and search range according to the sample distribution characteristics to adapt to a variety of sample distributions from dense to sparse; by recording the sample distribution law and the trend of focal position changes, the scanning path is optimized and repeated operations are reduced, thereby significantly improving the applicability of the system under complex experimental conditions. It has strong environmental adaptability and can cope with different lighting conditions and microscope magnifications.

[0082] (5) Traditional autofocus systems are complex to operate, rely on manual parameter adjustment, and require 2-3 days of training, with high labor costs and a high threshold for use. This application effectively reduces manual intervention through full-process automation design. Specifically, adaptive search and memory optimization achieves fully automated operations from sample detection to focal plane positioning, greatly reducing the complexity of use. Compared with traditional systems, labor costs are reduced by 70% and training time is shortened by 60%. At the same time, the modular design also makes the present invention easy to integrate and deploy, significantly reducing user learning costs and usage barriers.

[0083] In summary, this application has demonstrated outstanding performance advantages in focusing speed, positioning accuracy, success rate, adaptability and cost of use through the collaborative innovation of dual judgment, adaptive search and memory optimization. Compared with the existing technology, it not only significantly improves the efficiency and reliability of microscope autofocus, but also lowers the threshold for use. It has wide application value and commercial potential, and has significant advantages in the field of microscope autofocus.

[0084] Based on the same inventive concept, the embodiment of the present application also provides an intelligent autofocus system for a microscope. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more system embodiments provided below can refer to the limitations of the method above, and will not be repeated here. The system of the present application includes the following multiple modules.

[0085] The scanning image acquisition module is used to: acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area along the target path; the target area is distributed with target samples; different scanning images correspond to different focal planes.

[0086] The adaptive search module is used to: adopt an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area.

[0087] The clarity scoring module is used to: if the existence result represents the existence of the target sample in the scanned image set, then record the valid point plus one, extract multiple frames of scanned images based on the existence position area, and calculate the clarity score value of each frame of the scanned image extracted based on multiple image features.

[0088] The focal plane determination module is used to: select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point.

[0089] The adaptive search module is also used to: use an adaptive search algorithm to adjust the point spacing according to the valid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module; if the existence result indicates that the target sample does not exist in the scanning image set, then record the invalid points plus one, use an adaptive search algorithm to adjust the point spacing according to the invalid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module.

[0090] The system of this application can be modularly designed and easily integrated, and is suitable for a variety of application scenarios of microscope autofocus, such as life sciences, material research, and industrial testing. The system of this application is efficient and robust, and can adapt to the diverse characteristics of complex samples to ensure the precise selection of the focal plane; it can quickly screen the sample area and accurately locate the focal plane, significantly improving the efficiency and accuracy of autofocus.

[0091] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the microscope intelligent automatic focusing method is realized.

[0092] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0093] In an exemplary embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a microscope intelligent autofocus method. The computer device is of great significance to the autofocus technology and can effectively improve the automation level of sample detection.

[0094] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0095] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0096] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0098] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0099] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A microscope intelligent automatic focusing method, characterized in that: Methods include: Acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area along the target path; Target samples are distributed in the target area; different scanned images correspond to different focal planes; Adopting an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area; If the existence result indicates that the target sample exists in the scanned image set, the effective point number is recorded plus one, multiple frames of scanned images are extracted based on the existence position area, and the clarity score value of each frame of the scanned image extracted is calculated based on the image multi-features; Select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point; Adopting an adaptive search algorithm, adjusting the point spacing according to the number of valid points, the path points and the target path to determine the next path point, and then returning to the step of obtaining the scanned image set; If the existence result indicates that the target sample does not exist in the scanned image set, the number of invalid points is recorded plus one, and an adaptive search algorithm is used to adjust the point spacing according to the number of invalid points, the path points and the target path to determine the next path point, and then return to the step of obtaining the scanned image set.

2. The microscope intelligent autofocus method according to claim 1, characterized in that: The process of determining the target path includes: Acquire a historical focus data set; any historical focus data in the historical focus data set includes a historical path point; Evenly divide the target area into a plurality of grid areas; For each of the grid areas, based on the historical focus data set, the total number of historical path points in the grid area and the number of historical path points recorded as valid points are counted, and a sample detection rate is calculated; Based on the sample detection rate, determining the detection priority of the grid area; Based on the preset priority scanning rules, the target path is determined according to the detection priority of each of the grid areas.

3. The microscope intelligent autofocus method according to claim 1, characterized in that: Adopting an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result, including: Extracting a scan image subset from the scan image set according to a preset range and a preset scan interval; For the scanned images in the scanned image subset, using a target detection algorithm to perform target detection one by one to obtain a single image detection result; If at least one of the single image detection results indicates the presence of the target sample, determining the corresponding presence location area, and generating a presence result indicating the presence of the target sample in the scanned image set; If all the single image detection results indicate that the target sample does not exist, the preset range is updated, the preset scanning interval is updated, and the number of updates is recorded plus one; If the number of updates reaches the preset number of updates, it is determined that the existing position area is empty, and an existence result indicating that the target sample does not exist in the scanned image set is generated; If the update number does not reach the preset update number, the method returns to the step of extracting a scan image subset from the scan image set according to a preset range and a preset scanning interval.

4. The microscope intelligent autofocus method according to claim 3, characterized in that: The preset range is updated, and the preset scanning interval is updated, specifically: the preset small range is updated to the preset medium range, and the preset small range scanning interval is updated to the preset medium range scanning interval; Alternatively, the preset range is updated, and the preset scanning interval is updated, specifically: the preset medium range is updated to the preset large range, and the preset medium range scanning interval is updated to the preset large range scanning interval.

5. The microscope intelligent autofocus method according to claim 3, characterized in that: The process of determining the preset range includes: Acquire a historical focus data set; any historical focus data in the historical focus data set includes a historical path point and a corresponding historical final focal plane; Based on the historical focus data set, determining a historical focus position change trend and calculating a trend offset; The preset range is determined according to the trend offset, the final focal plane corresponding to the previous path point and the preset search range.

6. The microscope intelligent autofocus method according to claim 1, characterized in that: The clarity score of each scanned image frame is calculated and extracted based on multiple image features, including: For any frame scanned image extracted, a gradient algorithm is used to calculate the gradient clarity score value; The Laplace operator is used to calculate the Laplace clarity score through the second-order derivative; High-frequency component analysis was used to calculate the high-frequency component clarity score through Fourier transform; The gradient clarity score value, the Laplace clarity score value and the high-frequency component clarity score value are weightedly calculated to obtain a clarity score value of the scanned image.

7. The microscope intelligent autofocus method according to claim 6, characterized in that: The calculation formula of the gradient clarity score is: ； Where G(I) is the gradient clarity score of the scanned image I, is the grayscale gradient of the pixel (x, y) of the scanned image I in the horizontal direction, is the grayscale gradient of the pixel (x, y) of the scanned image I in the vertical direction; To accumulate all the pixels of the scanned image I; The calculation formula of the Laplace clarity score is: ； Where L(I) is the Laplace clarity score of the scanned image I, is the second-order derivative of the pixel (x, y) of the scanned image I in the horizontal direction, is the second-order derivative of the pixel (x, y) of the scanned image I in the vertical direction; The calculation formula of the high-frequency component clarity score is: ； in, , H(I) is the high-frequency component clarity score of the scanned image I, F ( u , v ) is the Fourier transform result of the scanned image I in the frequency domain, is the energy of the high-frequency component of the scanned image I in the frequency domain, HF is the range of the high-frequency component, is the Fourier transform operation, The energy of the high-frequency components of the scanned image I in the frequency domain is accumulated.

8. The microscope intelligent autofocus method according to claim 1, characterized in that: Adopting an adaptive search algorithm, adjusting the point spacing according to the number of valid points, the path points and the target path to determine the next path point, including: If the valid points are within a preset valid point range, the point spacing is reduced, and the next path point is determined according to the reduced point spacing, the path point and the target path; Adopting an adaptive search algorithm, adjusting the point spacing according to the number of invalid points, the path points and the target path to determine the next path point, including: If the invalid points are within a first preset invalid points range, increasing the point spacing, and determining the next path point according to the increased point spacing, the path point and the target path; If the invalid points are within a second preset invalid points range, a next path point is determined based on a preset jump amplitude, the path point and the target path.

9. An intelligent automatic focusing system for a microscope, characterized in that: The system includes: The scanning image acquisition module is used to: acquire a scanning image set; the scanning image set is a set of scanning images obtained at any path point when the microscope component scans and images the target area according to the target path; the target area is distributed with target samples; different scanning images correspond to different focal planes; An adaptive search module, used to: use an adaptive search algorithm to perform target detection on the scanned image set to obtain a detection result; the detection result includes an existence result and an existence location area; A clarity scoring module is used to: if the existence result indicates that the target sample exists in the scanned image set, record the valid points plus one, extract multiple frames of scanned images based on the existence position area, and calculate the clarity score value of each frame of the scanned image extracted based on multiple features of the image; A focal plane determination module, configured to: select the scanned image with the highest clarity score, and mark the corresponding focal plane as the final focal plane of the path point; The adaptive search module is also used to: use an adaptive search algorithm to adjust the point spacing according to the valid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module; if the existence result indicates that the target sample does not exist in the scanning image set, then record the invalid points plus one, use an adaptive search algorithm to adjust the point spacing according to the invalid points, the path points and the target path to determine the next path point, and then return to the scanning image acquisition module.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the microscope intelligent autofocus method according to any one of claims 1 to 8.

Images