Digital pathological section automatic focusing method and device based on five-point heterogeneous constraint
By employing an autofocus method based on five-point heterogeneous constraints, combining coarse and fine focusing, the problem of low efficiency in existing technologies is solved, enabling efficient and adaptable digital pathological slide scanning and ensuring high-quality imaging of pathological slides.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN SHENGQIANG TECH
- Filing Date
- 2026-04-10
- Publication Date
- 2026-05-12
AI Technical Summary
Existing digital pathology slide scanning technology is inefficient when increasing the number of focus points, struggles to adapt to differences in different pathological tissue types, and suffers from interference from local abnormal areas during scanning, resulting in uneven image quality.
An autofocus method based on five-point heterogeneous constraints is adopted. By selecting five focus points that take into account the overall focal plane reference, scanning direction trend, lateral boundary prediction accuracy and local focal plane change compensation in coarse and fine focus, a coarse focal plane model is constructed and a fine focal plane model is corrected in high-risk areas to optimize the autofocus path.
It improves the efficiency and imaging quality of digital pathology slide scanning, ensures adaptability to different pathological tissue categories, reduces unnecessary fine modeling operations, and increases the throughput and imaging clarity of whole-slide scans.
Smart Images
Figure CN122018137A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of digital pathological slide scanning, and in particular to a method and apparatus for automatic focusing of digital pathological slides based on five-point heterogeneous constraints. Background Technology
[0002] The core objective of digital pathology slide scanning is to convert pathological tissue samples on glass slides into high-resolution, highly consistent panoramic digital images using a high-magnification microscopic optical system. In clinically common scanning scenarios at 20x, 40x, and higher magnifications, the depth of field of the microscope objective is extremely shallow, typically only at the level of a few micrometers. Numerous factors, such as differences in the thickness of the pathological tissue, coverslip tilt, uneven distribution of the mounting medium, slide clamping errors, flatness deviations of the motion platform, and thermal drift during equipment operation, can all cause significant variations in the optimal focal height at different spatial positions of the same slide.
[0003] Currently, the mainstream technical solutions in this field mainly include four categories: single-point focusing, multi-point planar fitting, multi-point curved surface fitting, and continuous focusing during scanning. However, these solutions have the following shortcomings in practical applications: First, simply increasing the number of focus points can improve fitting accuracy, but it will significantly increase focusing time and affect the overall scanning throughput. Second, ordinary multi-point fitting usually assumes that all focus points are equally reliable, which is easily affected by blank areas, wrinkled areas, bubble areas, unevenly stained areas, and warped areas at tissue edges. Third, the texture features and focusing reliability of different pathological tissue categories vary significantly. If a uniform weight is used, low-reliability tissue areas can easily overly affect the focal plane modeling results. Fourth, some solutions only perform a one-time global fitting, making it difficult to take into account the coexistence of overall gradual changes and local abrupt changes in the tissue surface. Fifth, historical focal plane information in the scanning direction is not fully utilized, resulting in the need to repeatedly model each local area, which is inefficient.
[0004] Therefore, there is a need for an autofocus method that can achieve high robustness and efficiency with a smaller number of focus points and can adapt to the differences in different pathological tissue categories. Summary of the Invention
[0005] This application provides a digital pathological slide autofocus method and device based on five-point heterogeneous constraints. By selecting five focus points that take into account the overall focal plane reference, scanning direction trend, lateral boundary prediction accuracy and local focal plane mutation compensation between coarse and fine focusing, it makes up for the insufficient adaptation of low-order surface fitting to local abnormal areas and achieves accurate acquisition of the autofocus path.
[0006] In a first aspect, embodiments of this application provide an automatic focusing method for digital pathological slides based on five-point heterogeneity constraints, the method comprising:
[0007] A low-magnification preview image of the target slice is obtained using a low-magnification objective lens that is lower than the target magnification, and the tissue region in the low-magnification preview image is segmented out, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements; A low-magnification preview image of the target slice is obtained using a low-magnification objective lens that is lower than the target magnification, and the tissue region in the low-magnification preview image is segmented out, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements; Five coarse focus points are selected on the low-magnification preview image. The five coarse focus points are then focused at the five coarse focus points using a low-magnification objective lens to obtain the coordinates of the five coarse focus points. The coarse focus plane model is obtained by fitting the coordinates of the five coarse focus points. The low-magnification preview image is divided into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-region containing the tissue region is taken as a local modeling unit. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling unit corresponding to the local focal plane that meets the high-risk area screening rules is taken as the high-risk area, and the remaining local modeling units are non-high-risk areas. Five fine focus points are selected in each high-risk area. The objective lens at the target magnification is used to focus at the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus surface model of the corresponding high-risk area. In the coarse focal plane model, the focal plane of each non-high-risk area in the set of focus sub-regions is obtained as the first focal plane set. In the corresponding fine focal plane model, the focal plane of each high-risk area in the set of focus sub-regions is obtained as the second focal plane set. The autofocus path of the target slice is obtained based on the first focal plane set and the second focal plane set.
[0008] Secondly, embodiments of this application provide an automatic focusing device for digital pathological slides based on five-point heterogeneous constraints, comprising: The acquisition module is used to acquire a low-magnification preview image of the target slice with a low-magnification objective lens that is lower than the target magnification, and to segment the tissue region in the low-magnification preview image, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements. The coarse focus module is used to select five coarse focus points on the low-magnification preview image, perform focusing at the five coarse focus points with the low-magnification objective lens to obtain the coordinates of the five coarse focus points, and fit the coordinates of the five coarse focus points to obtain the coarse focus surface model. The risk assessment module is used to divide the low-magnification preview image into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-regions containing tissue areas in the focus sub-region set are used as local modeling units. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling units corresponding to the local focal planes that meet the high-risk area screening rules are taken as high-risk areas, and the remaining local modeling units are non-high-risk areas. The fine focus module is used to select five fine focus points in each high-risk area. The objective lens at the target magnification is used to focus on the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus plane model of the corresponding high-risk area. The autofocus path acquisition module obtains the focal plane of each non-high-risk area in the focus sub-region set as the first focal plane set in the coarse focal plane model, and obtains the focal plane of each high-risk area in the focus sub-region set as the second focal plane set in the corresponding fine focal plane model. Based on the first focal plane set and the second focal plane set, the autofocus path of the target slice is obtained.
[0009] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a digital pathological slide autofocusing method based on five-point heterogeneous constraints.
[0010] Fourthly, embodiments of this application provide a readable storage medium storing a computer program, which, when executed by a processor, implements a method for automatic focusing of digital pathological slides based on five-point heterogeneous constraints.
[0011] The main contributions and innovations of this invention are as follows: This application's embodiments compensate for the insufficient adaptation of low-order surface fitting to local abnormal regions by selecting five focus points that take into account the overall focal plane reference, scanning direction trend, lateral boundary prediction accuracy, and local focal plane abrupt change compensation between coarse and fine focusing. This application's embodiments also avoid the high time consumption of full-area fine focusing by using a layered modeling approach: first, globally constructing a coarse focal plane model, and then locally correcting the fine focal plane model in high-risk areas. This approach considers both gradual changes in the overall tissue surface and local abrupt changes, improving the overall scanning throughput while ensuring focusing accuracy. Furthermore, this application's embodiments combine focus confidence and fitting residuals... By analyzing the focus height gradient, tissue type, and tissue-background boundary, this application accurately identifies areas with low focusing reliability and prone to defocusing. This ensures that fine focusing is applied only to the truly needed areas, reducing ineffective fine modeling operations. Simultaneously, the tissue-background boundary is included in the high-risk area to guarantee image clarity in that region. Furthermore, this application's embodiments utilize the characteristic of continuously and gradually changing focal planes in adjacent high-risk areas along the scanning direction to apply prior constraints to the fitting of fine focal plane models from the preceding high-risk area as fitting priors for the next adjacent area. This reduces the time spent on repeated complete modeling and improves scanning efficiency.
[0012] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description
[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of an automatic focusing method for digital pathological slides based on five-point heterogeneous constraints, according to an embodiment of this application. Figure 2 This is a schematic diagram of a process for obtaining five fine focus points according to an embodiment of this application; Figure 3 This is a structural block diagram of a digital pathological slide autofocus device based on five-point heterogeneous constraints according to an embodiment of this application; Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0014] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.
[0015] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.
[0016] Example 1 This application provides an automatic focusing method for digital pathological slides based on five-point heterogeneous constraints. By selecting five focus points that balance overall focal plane reference, scanning direction trend, lateral boundary prediction accuracy, and compensation for local focal plane abrupt changes between coarse and fine focusing, it compensates for the insufficient adaptation of low-order surface fitting to local abnormal regions, achieving accurate acquisition of the automatic focusing path. Specifically, refer to... Figure 1 The method includes: A low-magnification preview image of the target slice is obtained using a low-magnification objective lens that is lower than the target magnification, and the tissue region in the low-magnification preview image is segmented out, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements; Five coarse focus points are selected on the low-magnification preview image. The five coarse focus points are then focused at the five coarse focus points using a low-magnification objective lens to obtain the coordinates of the five coarse focus points. The coarse focus plane model is obtained by fitting the coordinates of the five coarse focus points. The low-magnification preview image is divided into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-region containing the tissue region is taken as a local modeling unit. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling unit corresponding to the local focal plane that meets the high-risk area screening rules is taken as the high-risk area, and the remaining local modeling units are non-high-risk areas. Five fine focus points are selected in each high-risk area. The objective lens at the target magnification is used to focus at the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus surface model of the corresponding high-risk area. In the coarse focal plane model, the focal plane of each non-high-risk area in the set of focus sub-regions is obtained as the first focal plane set. In the corresponding fine focal plane model, the focal plane of each high-risk area in the set of focus sub-regions is obtained as the second focal plane set. The autofocus path of the target slice is obtained based on the first focal plane set and the second focal plane set.
[0017] In the current embodiment, a low-magnification objective refers to an objective lens with a microscopic imaging magnification lower than the target magnification used to perform pre-scanning of the target slide. In pathological slide scanning, 20x, 40x or higher magnification objectives are generally used to scan the target slide to meet clinical clarity requirements. Therefore, the low-magnification objective lens in this solution is a 2.5x or 10x objective lens.
[0018] In other words, this method first uses a 10x objective lens to scan the target slice to obtain a low-magnification preview image.
[0019] In the current embodiment, a pre-trained segmentation model is used to segment the tissue region, background blank region, and low-reliability regions such as wrinkles, bubbles, contamination, and coverslip edges in the low-magnification preview image.
[0020] In the current embodiment, the five coarse focus points are the center representative point, the forward scan prediction point, the first lateral boundary point, the second lateral boundary point, and the risk compensation point on the low-magnification preview image. The five fine focus points are the center representative point, the forward scan prediction point, the first lateral boundary point, the second lateral boundary point, and the risk compensation point on the high-risk area. The center representative point is a point located within the geometric center of the corresponding image and situated on the tissue region. The forward scan prediction point is a point within the tissue region with the highest tissue continuity in the scanning direction of the corresponding image. The first lateral boundary point is a point with the smallest X-axis coordinate in the corresponding image and situated within the tissue region. The second lateral boundary point is a point with the largest X-axis coordinate in the corresponding image and situated within the tissue region. The risk compensation point is a point within the tissue region with the highest probability of focal plane anomalies. The corresponding image is the corresponding low-magnification preview image or the high-risk area.
[0021] Specifically, the central representative point represents the overall average focal plane level of the low-magnification preview image, provides a benchmark value for focal plane fitting, and is the point with the highest fitting weight priority.
[0022] Specifically, the forward prediction point is used to capture the focal plane change trend in the scanning direction and constrain the slope of the coarse focal plane model in the scanning direction. Specifically, the first and second lateral boundary points are used to constrain the tilt and change trend in the X-axis direction, thereby compensating for the inaccuracy of lateral boundary prediction caused by sampling in the central region and ensuring the focal plane prediction accuracy of the entire lateral range of the target area.
[0023] Specifically, the risk compensation point is an anomaly compensation anchor point used to improve the robustness of the model. It is used to capture local focal plane mutation information in the target area, thereby making up for the insufficient adaptation of low-order surface fitting to local anomaly areas and reducing the overall fitting deviation caused by local focal plane mutation.
[0024] Furthermore, a pre-trained artificial intelligence model is used to select pixels in the tissue region that have obvious texture abrupt changes, large tissue thickness variations, or high edge warping probability as risk compensation points.
[0025] Specifically, low-magnification focusing is performed on each of the acquired coarse focus points to obtain the coarse focus coordinates of the five coarse focus points under low-magnification imaging conditions.
[0026] Specifically, this scheme uses a first-order inclined plane fitting or low-order surface fitting to fit the coordinates of the five coarse focus points. When the overall undulation of the slice is small, the coarse focus surface model can be obtained by fitting the horizontal coordinates, vertical coordinates and focus height of the five coarse focus points using the least squares method. In addition, when the overall undulation of the slice is complex, a low-order residual compensation term can be superimposed on the coarse focus surface model.
[0027] Specifically, the coarse focal plane model obtained by fitting is used to describe the overall trend of focal point change with plane position within the whole area or a large partition, and serves as the focal point prior for subsequent high-magnification scanning.
[0028] Furthermore, the focus height value of any pixel in the low-magnification preview image can be obtained based on the coarse focus plane model, thereby obtaining multiple consecutive coarse focus points in the local modeling unit in the subsequent process.
[0029] In the current embodiment, a local modeling unit is a range containing a tissue region under the field of view of the target magnification objective lens. That is, the local modeling unit corresponds to one or more high-magnification tiles, or to a region composed of several continuous scan lines under a linear array scanning model.
[0030] Specifically, the size of the local modeling unit can be preset according to the objective lens magnification, field of view, tissue continuity, and focusing time. For example, it can be set to cover an area of 2×2, 3×3, or more high-magnification tiles. Each local modeling unit is arranged adjacent to the other end along the scanning direction, side by side, to collectively cover the entire tissue area to be scanned.
[0031] In the current embodiment, the local focal plane is input into a pre-trained focus confidence prediction model to obtain the focus confidence of each coarse focus point; the difference in focus height between each coarse focus point in the local focal plane and the five coarse focus points in the low-magnification preview image is calculated respectively, and the result with the largest difference is selected as the fitting residual of the corresponding coarse focus point; the maximum focus height difference between the current local focal plane and the adjacent local focal plane is used as the focus height change gradient of the current local focal plane.
[0032] Specifically, low focus confidence, high fitting residual, and high gradient of focus height change indicate that the fitting accuracy of the corresponding local focal plane is not high. That is, the coarse focus point quality predicted by the coarse focal plane model within the corresponding local focal plane is poor, so it is regarded as a high-risk area.
[0033] In the current embodiment, the tissue category of each local modeling unit is classified based on the tissue category identification model, and local modeling units containing tissue categories such as necrotic area, mucus area and fat area are designated as high-risk areas based on the classification results; local modeling units containing the boundary between the tissue area and the background area are also designated as high-risk areas.
[0034] Specifically, the tissue region is divided into one or more categories based on the tissue category recognition model, such as cell-dense area, interstitial fiber area, necrotic area, mucus area, fat area, hemorrhage area, and background impurity area.
[0035] Specifically, the organization category identification model can be obtained based on a pre-trained classification model using color, texture, and structural features.
[0036] Specifically, necrotic areas, mucus areas, and fat areas have inherently lower focusing reliability, and due to their greater tissue thickness and susceptibility to tissue mutations, they are considered high-risk areas. On the other hand, the tissue morphology at the boundary between the tissue area and the background can help determine the extent of tissue infiltration. Fine focusing at the boundary between the tissue area and the background can ensure image clarity, preserve all pathological information of the slide, and allow the final digital pathology slide to meet the needs of all scenarios, including clinical diagnosis.
[0037] In the current embodiment, when a local modeling unit is determined to be high-risk, the system reselects five fine focus points within that unit for local correction. Therefore, if there are two independent local modeling units that are both determined to be high-risk, they will, in principle, correspond to two sets of fine focus points, for a total of ten fine focus points; if only one local modeling unit is high-risk, then only one set of high-magnification five-point correction is performed within that unit.
[0038] Specifically, by refocusing five fine focus points in the high-risk area to correct the focus points in the high-risk area, we can pay more attention to the local focal plane undulations in the high-risk area, so that the secondary modeling is focused only on the area most likely to be out of focus, thereby reducing the overall scanning time while ensuring accuracy.
[0039] In the current embodiment, when acquiring the coordinates of each fine focus point, the target magnification objective lens moves along the Z-axis at each fine focus point with a preset step distance and acquires multiple intermediate images. The sharpness of each intermediate image is calculated, and the coordinate position corresponding to the intermediate image with the highest sharpness is taken as the fine focus point coordinate.
[0040] Specifically, in the step of moving along the Z-axis at each fine focus point with a preset step distance and acquiring multiple intermediate images, a coarse search is first performed within the depth of focus range using the first step distance to obtain a coarse search range, and then a fine search is performed within the coarse search range using the second step distance, and the images acquired during the fine search are acquired as intermediate images, wherein the first step distance is greater than the second step distance.
[0041] Specifically, gradient energy, Laplacian response, high-frequency component energy, Brenner function, or a combination of the above methods can be used to calculate image sharpness.
[0042] In the current embodiment, a flowchart illustrating the process of acquiring five fine focus points is shown below. Figure 2 As shown, when the fine focus point is located in a contaminated or bubble-interfered tissue area, or when the fine focus point is not located in a tissue area, or when the confidence level of the fine focus point is lower than the set threshold, the corresponding fine focus point is regarded as an abnormal fine focus point, and a new fine focus point is reacquired in the neighborhood of the abnormal fine focus point.
[0043] Furthermore, the focus confidence is calculated based on the imaging results of the target magnification objective lens at the abnormal fine focus point. Specifically, the confidence of each abnormal fine focus point is determined by one or more of the following factors: peak sharpness, difference between peak and subpeak values, curve monotonicity, tissue coverage, and local texture stability, to ensure that the image acquired based on the fine focus point coordinates meets the sharpness requirements.
[0044] Specifically, the method for obtaining new fine focus points is the same as that for the corresponding abnormal fine focus points. For example, when a risk compensation point in a high-risk area is identified as an abnormal fine focus point, a new risk compensation point is obtained in the same way.
[0045] Furthermore, the new fine focus point prioritizes locations with continuous tissue, stable texture, and high correlation with the current scanning direction, and re-performs fine focusing until a usable alternative point is obtained.
[0046] In the current embodiment, the specific implementation method of fitting the five fine focal coordinates to obtain the fine focal plane model corresponding to the high-risk area is the same as that of fitting the five coarse focal coordinates.
[0047] In the current embodiment, the fine focal plane model of the current high-risk area is used as the fitting prior for the next adjacent high-risk area. The adjacency relationship between high-risk areas is determined based on the scanning direction. The fitting prior is used to constrain the fitting process of the fine focal plane model of the next adjacent high-risk area, making the fitting result more accurate and stable.
[0048] Specifically, when fitting the coordinates of the five fine focal points of the next adjacent high-risk area, the parameters and characteristics of the fine focal plane model of the current high-risk area, such as slope and intercept, are referenced and incorporated into the fitting process as prior knowledge.
[0049] Specifically, different local modeling units are usually adjacent to each other in space along the scanning direction and side by side in the horizontal direction. The tissue surface height between adjacent units usually has a certain continuity. Therefore, the local focal plane parameters of the previous unit can be used as an effective prior for the next unit. The principle is that although there are local mutations on the surface of pathological tissue, in most cases the focal plane changes of adjacent local areas are continuous and gradual. Therefore, the focal plane function that has been fitted by the current unit can provide the next unit with an initial Z-axis distribution prediction that is closer to the true value.
[0050] For example, the target focal plane function of the current high-risk area is first extrapolated along the scanning direction to the center and boundary positions of the next high-risk area to obtain the predicted focal height of each predetermined verification point in the next high-risk area; then, the center representative point, the forward prediction point, and the risk compensation point are verified first in the next high-risk area, and the difference between the actual best focal point and the predicted value of the verification point is calculated; if the difference is less than the first preset threshold, the parameters of the current high-risk area are directly used or only a small translation correction is made; if the difference is between the first threshold and the second threshold, a local correction is performed on the basis of retaining the original trend term; if the difference is greater than the second threshold, it is considered that a significant local mutation has occurred, and the complete five-point modeling is re-executed for the next high-risk area.
[0051] Specifically, using the fine focal plane model of the current high-risk area as the fitting prior for the next adjacent high-risk area can make full use of the continuity on the scanning path, reduce the time required for each local modeling unit to start fine modeling from scratch, and improve scanning efficiency; at the same time, it can promptly detect local mutations through the verification mechanism, balancing speed and accuracy.
[0052] In the current embodiment, the non-high-risk area focal plane information in the first focal plane set is spatially fused with the high-risk area fine focal plane model in the second focal plane set to obtain the autofocus path.
[0053] Specifically, for the first focal plane subset, since the coarse focal plane model can already describe its focal point change trend well, imaging can be performed relatively smoothly according to the predetermined focus height during scanning. For the second focal plane subset, because it has undergone fine focal plane model correction, it can more accurately cope with the complex changes in local focal planes and effectively avoid image blurring caused by abrupt changes in focal planes.
[0054] In the current embodiment, the entire slide is scanned at high speed based on the autofocus path to obtain a clear digital pathological image, and the scanned image and related focus parameters are output to the subsequent browsing, analysis or archiving module.
[0055] Furthermore, when focusing along the autofocus path to acquire digital pathological images, local compensation is performed for each pair of focal points. Specifically, during actual high-magnification scanning, the sharpness index of the current tile, line segment, or sampled image block is calculated in real-time or near real-time, and this index is compared with the expected sharpness threshold. When the sharpness index is lower than the threshold but not to the point where remodeling is required, one or more small-step tentative offsets are performed near the current target Z-value. For example, one or more small Z-steps are moved in both the forward and reverse directions, a small number of verification images are reacquired, and the Z-value corresponding to the highest sharpness is selected as the micro-compensation result for the tile or line segment.
[0056] Specifically, the amount of micro-compensation is preferably limited to a preset small range to avoid disrupting the focal plane trend of the main subject.
[0057] Example 2 Based on the same concept, referencing Figure 3 This application also proposes a digital pathological slide autofocusing device based on five-point heterogeneous constraints, comprising: The acquisition module is used to acquire a low-magnification preview image of the target slice with a low-magnification objective lens that is lower than the target magnification, and to segment the tissue region in the low-magnification preview image, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements. The coarse focus module is used to select five coarse focus points on the low-magnification preview image, perform focusing at the five coarse focus points with the low-magnification objective lens to obtain the coordinates of the five coarse focus points, and fit the coordinates of the five coarse focus points to obtain the coarse focus surface model. The risk assessment module is used to divide the low-magnification preview image into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-regions containing tissue areas in the focus sub-region set are used as local modeling units. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling units corresponding to the local focal planes that meet the high-risk area screening rules are taken as high-risk areas, and the remaining local modeling units are non-high-risk areas. The fine focus module is used to select five fine focus points in each high-risk area. The objective lens at the target magnification is used to focus on the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus plane model of the corresponding high-risk area. The autofocus path acquisition module obtains the focal plane of each non-high-risk area in the focus sub-region set as the first focal plane set in the coarse focal plane model, and obtains the focal plane of each high-risk area in the focus sub-region set as the second focal plane set in the corresponding fine focal plane model. Based on the first focal plane set and the second focal plane set, the autofocus path of the target slice is obtained.
[0058] Example 3 This embodiment also provides an electronic device, see reference. Figure 4 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.
[0059] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0060] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.
[0061] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.
[0062] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the digital pathological slide autofocusing methods based on five-point heterogeneous constraints in the above embodiments.
[0063] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.
[0064] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.
[0065] Input / output device 408 is used to input or output information. In this embodiment, the input information may be a low-magnification preview image, coarse focus point, etc., and the output information may be an autofocus path.
[0066] Optionally, in this embodiment, the processor 402 can be configured to perform the following steps via a computer program: A low-magnification preview image of the target slice is obtained using a low-magnification objective lens that is lower than the target magnification, and the tissue region in the low-magnification preview image is segmented out, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements; Five coarse focus points are selected on the low-magnification preview image. The five coarse focus points are then focused at the five coarse focus points using a low-magnification objective lens to obtain the coordinates of the five coarse focus points. The coarse focus plane model is obtained by fitting the coordinates of the five coarse focus points. The low-magnification preview image is divided into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-region containing the tissue region is taken as a local modeling unit. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling unit corresponding to the local focal plane that meets the high-risk area screening rules is taken as the high-risk area, and the remaining local modeling units are non-high-risk areas. Five fine focus points are selected in each high-risk area. The objective lens at the target magnification is used to focus at the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus surface model of the corresponding high-risk area. In the coarse focal plane model, the focal plane of each non-high-risk area in the set of focus sub-regions is obtained as the first focal plane set. In the corresponding fine focal plane model, the focal plane of each high-risk area in the set of focus sub-regions is obtained as the second focal plane set. The autofocus path of the target slice is obtained based on the first focal plane set and the second focal plane set.
[0067] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.
[0068] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.
[0069] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 4 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.
[0070] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0071] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for automatic focusing of digital pathological slides based on five-point heterogeneous constraints, characterized in that, Includes the following steps: A low-magnification preview image of the target slice is obtained using a low-magnification objective lens that is lower than the target magnification, and the tissue region in the low-magnification preview image is segmented out, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements; Five coarse focus points are selected on the low-magnification preview image. The five coarse focus points are then focused at the five coarse focus points using a low-magnification objective lens to obtain the coordinates of the five coarse focus points. The coarse focus plane model is obtained by fitting the coordinates of the five coarse focus points. The low-magnification preview image is divided into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-region containing the tissue region is taken as a local modeling unit. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling unit corresponding to the local focal plane that meets the high-risk area screening rules is taken as the high-risk area, and the remaining local modeling units are non-high-risk areas. Five fine focus points are selected in each high-risk area. The objective lens at the target magnification is used to focus at the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus surface model of the corresponding high-risk area. In the coarse focal plane model, the focal plane of each non-high-risk area in the set of focus sub-regions is obtained as the first focal plane set. In the corresponding fine focal plane model, the focal plane of each high-risk area in the set of focus sub-regions is obtained as the second focal plane set. The autofocus path of the target slice is obtained based on the first focal plane set and the second focal plane set.
2. The automatic focusing method for digital pathological slides based on five-point heterogeneous constraints according to claim 1, characterized in that, The high-risk area screening rules are as follows: the focus confidence of the coarse focus point of the local focal plane is lower than the first threshold, or the focus fitting residual of the coarse focus point of the local focal plane is greater than the second threshold, or the focus height change gradient between the local focal plane and the adjacent local focal plane is greater than the third threshold.
3. The automatic focusing method for digital pathological slides based on five-point heterogeneous constraints according to claim 2, characterized in that, The local focal plane is input into a pre-trained focus confidence prediction model to obtain the focus confidence of each coarse focus point; the difference in focus height between each coarse focus point in the local focal plane and the five coarse focus points in the low-magnification preview image is calculated, and the result with the largest difference is selected as the fitting residual of the corresponding coarse focus point; the maximum focus height difference between the current local focal plane and the adjacent local focal plane is used as the focus height change gradient of the current local focal plane.
4. The automatic focusing method for digital pathological slides based on five-point heterogeneous constraints according to claim 1, characterized in that, Based on the tissue category identification model, the tissue category of each local modeling unit is classified, and based on the classification results, local modeling units containing tissue categories such as necrotic area, mucus area and fat area are designated as high-risk areas; local modeling units containing the boundary between tissue area and background part are also designated as high-risk areas.
5. The automatic focusing method for digital pathological slides based on five-point heterogeneous constraints according to claim 1, characterized in that, The five coarse focus points are the center representative point, the forward scan prediction point, the first lateral boundary point, the second lateral boundary point, and the risk compensation point on the low-magnification preview image. The five fine focus points are the center representative point, the forward scan prediction point, the first lateral boundary point, the second lateral boundary point, and the risk compensation point on the high-risk area. Among them, the center representative point is the point within the geometric center of the corresponding image and located on the tissue region; the forward scan prediction point is the point within the tissue region with the highest tissue continuity in the scan direction of the corresponding image; the first lateral boundary point is the point with the smallest X-axis coordinate in the corresponding image and located within the tissue region; the second lateral boundary point is the point with the largest X-axis coordinate in the corresponding image and located within the tissue region; and the risk compensation point is the point in the tissue region with the highest probability of focal plane anomaly.
6. The automatic focusing method for digital pathological slides based on five-point heterogeneous constraints according to claim 1, characterized in that, The target magnification objective lens is moved along the Z-axis at each fine focus point with a preset step distance and multiple intermediate images are acquired. The sharpness of each intermediate image is calculated, and the coordinate position corresponding to the intermediate image with the highest sharpness is taken as the fine focus coordinate.
7. The automatic focusing method for digital pathological slides based on five-point heterogeneous constraints according to claim 1, characterized in that, The fine focal plane model of the current high-risk area is used as the fitting prior for the next adjacent high-risk area. The adjacency relationship between high-risk areas is determined based on the scanning direction. The fitting prior is used to constrain the fitting process of the fine focal plane model of the next adjacent high-risk area.
8. A digital pathological slide autofocusing device based on five-point heterogeneous constraints, characterized in that, include: The acquisition module is used to acquire a low-magnification preview image of the target slice with a low-magnification objective lens that is lower than the target magnification, and to segment the tissue region in the low-magnification preview image, wherein the target magnification is the rated imaging magnification that meets the clinical clarity requirements. The coarse focus module is used to select five coarse focus points on the low-magnification preview image, perform focusing at the five coarse focus points with the low-magnification objective lens to obtain the coordinates of the five coarse focus points, and fit the coordinates of the five coarse focus points to obtain the coarse focus surface model. The risk assessment module is used to divide the low-magnification preview image into multiple focus sub-regions to form a focus sub-region set. The range of each focus sub-region corresponds to the objective lens field of view at the target magnification. The focus sub-regions containing tissue areas in the focus sub-region set are used as local modeling units. In the coarse focal plane model, the local focal plane composed of multiple consecutive coarse focus points in each local modeling unit is obtained. The local modeling units corresponding to the local focal planes that meet the high-risk area screening rules are taken as high-risk areas, and the remaining local modeling units are non-high-risk areas. The fine focus module is used to select five fine focus points in each high-risk area. The objective lens at the target magnification is used to focus on the five fine focus points to obtain the coordinates of the five fine focus points. The coordinates of the five fine focus points are then fitted to obtain the fine focus plane model of the corresponding high-risk area. The autofocus path acquisition module obtains the focal plane of each non-high-risk area in the focus sub-region set as the first focal plane set in the coarse focal plane model, and obtains the focal plane of each high-risk area in the focus sub-region set as the second focal plane set in the corresponding fine focal plane model. Based on the first focal plane set and the second focal plane set, the autofocus path of the target slice is obtained.
9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the digital pathological slide autofocusing method based on five-point heterogeneous constraints as described in any one of claims 1-7.
10. A readable storage medium, characterized in that, The readable storage medium stores a computer program that, when executed by a processor, implements a digital pathological slide autofocusing method based on five-point heterogeneous constraints as described in any one of claims 1-7.