Method and device for performing offset test on multi-layer fused digital pathology slides

Abstract

The application provides a method and device for offset testing of multi-layer fusion digital pathological sections, including the following steps: obtaining multi-layer fusion digital pathological sections and performing multi-scale detail enhancement to obtain multi-layer cell nucleus segmentation images; performing feature point extraction on each focal plane in the multi-layer cell nucleus segmentation images and obtaining a feature point matching result; calculating a global offset matrix of each to-be-tested layer relative to a reference layer based on the feature point matching result; obtaining a dynamic offset threshold under a current zooming multiple, obtaining an offset amount of each to-be-tested layer based on the global offset matrix, and judging an offset condition of the to-be-tested layer based on the offset amount. The scheme realizes sub-pixel level offset detection through the technology of performing global offset matrix on the sub-pixel refined feature points of different focal planes, and sets a dynamic offset threshold based on different zooming multiples, thereby overcoming the defect that a fixed threshold cannot adapt to a multi-resolution scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of pathological detection, in particular to a method and device for testing the offset of multi-layer fusion digital pathological sections. BACKGROUND

[0002] With the development of medical technology, digital pathology technology has become the core means of modern pathological diagnosis. The slice in the medical digital imaging and communication standard format not only has a multi-file structure, but also has a multi-layer fusion feature, which makes it easy to produce pixel-level offset when integrating multi-modal image devices such as digital scanners and AI analysis software.

[0003] The prior art generally uses pixel matching or manual detection to detect the offset of pathological section images. However, this detection method has many significant defects. First, it lacks sub-pixel level offset detection capability. Traditional methods cannot effectively detect sub-pixel level offset, which is crucial for diagnosis. A 0.1 pixel level offset corresponds to a physical offset of 0.025 μm at 40X magnification, which is enough to cause cell positioning errors. Second, it lacks a batch testing scheme. In the face of large-scale digital pathological sections, existing methods lack automated testing capabilities, and manual visual inspection is extremely inefficient and highly subjective. Third, it cannot dynamically evaluate the offset at different resolutions. For example, a 1 pixel offset at 80X high resolution corresponds to a physical distance of only 0.125 μm, while at 20X resolution, the physical distance is 0.5 μm. Fourth, it does not quantify environmental factors. In the processing of multi-layer fusion digital pathological sections, the temperature of the scanning device can affect the offset. The core reason is that temperature changes can cause physical deformation of the device's mechanical structure and optical system, leading to spatial position deviation in the imaging of the section. A 5℃ temperature drift of the scanning device can increase the false detection rate of traditional methods by 18%.

[0004] The above problems directly affect the accuracy of pathological diagnosis and may lead to medical accidents. According to statistics, the offset detection rate of traditional pixel matching methods is only 78%, while manual inspection is as low as 62%, which cannot meet the clinical needs. SUMMARY

[0005] The embodiments of the present application provide a method and device for testing the offset of multi-layer fusion digital pathological sections. By refining the feature points of different focal planes to a sub-pixel level and then performing global offset matrix technology, sub-pixel level offset detection is achieved. Dynamic offset thresholds are set based on different zoom ratios, overcoming the defect that fixed thresholds cannot adapt to multi-resolution scenarios.

[0006] In a first aspect, the embodiments of the present application provide a method for testing the offset of multi-layer fusion digital pathological sections, comprising:

[0007] obtain a multi-layer cell nucleus segmentation image by performing multi-scale detail enhancement on the cell nuclei in the multi-layer fused digital pathology slice at the current zoom ratio;

[0008] extract feature points from each focal plane in the multi-layer cell nucleus segmentation image, obtain a focal plane as a reference layer, and focal planes other than the reference layer are to-be-tested layers, and match the feature points of each to-be-tested layer with the feature points of the reference layer to obtain a feature point matching result;

[0009] perform sub-pixel refinement on each feature point in the feature point matching result in the to-be-tested layers and the reference layer based on a function fitting manner, and calculate a global offset matrix of each to-be-tested layer relative to the reference layer based on the sub-pixel refinement result;

[0010] obtain a dynamic offset threshold at the current zoom ratio, wherein the dynamic offset threshold is negatively correlated with the zoom ratio, obtain an offset amount of each to-be-tested layer based on the global offset matrix, and if the offset amount is greater than the dynamic offset threshold, it is considered that the corresponding to-be-tested layer has an offset.

[0011] In a second aspect, an apparatus for testing offsets of multi-layer fused digital pathology slices is provided, and the apparatus comprises:

[0012] an obtaining module, configured to obtain a multi-layer fused digital pathology slice at a current zoom ratio, and obtain a multi-layer cell nucleus segmentation image by performing multi-scale detail enhancement on the cell nuclei in the multi-layer fused digital pathology slice at the current zoom ratio;

[0013] a feature extraction module, configured to extract feature points from each focal plane in the multi-layer cell nucleus segmentation image, take a focal plane with the highest clarity as a reference layer, take focal planes other than the reference layer as to-be-tested layers, and match the feature points of each to-be-tested layer with the feature points of the reference layer to obtain a feature point matching result;

[0014] a sub-pixel refinement module, configured to perform sub-pixel refinement on each feature point in the feature point matching result in the to-be-tested layers and the reference layer based on a function fitting manner, and calculate a global offset matrix of each to-be-tested layer relative to the reference layer based on the sub-pixel refinement result;

[0015] an offset detection module, configured to obtain a dynamic offset threshold at the current zoom ratio, wherein the dynamic offset threshold is negatively correlated with the zoom ratio, obtain an offset amount of each to-be-tested layer based on the global offset matrix, and if the offset amount is greater than the dynamic offset threshold, it is considered that the corresponding to-be-tested layer has an offset.

[0016] In a third aspect, an electronic device is provided, comprising a memory and a processor, the memory storing a computer program, and the processor is configured to run the computer program to perform a method for performing offset test on multi-layer fused digital pathology slices.

[0017] The main contributions and innovations of the present application are as follows:

[0018] The embodiments of the present application realize multi-scale detail enhancement through histogram equalization and Laplacian pyramid feature extraction, thereby improving the accuracy of cell nucleus segmentation, and based on scale-invariant feature point matching and RANSAC algorithm to remove false matches, combined with sub-pixel refinement to realize 0.1 pixel level offset detection, solving the problem of lack of sub-pixel level offset detection capability of traditional methods; by correlating the scaling factor, resolution, nuclear area ratio and temperature and other factors to set the dynamic offset threshold, thereby realizing accurate evaluation under different conditions, overcoming the defect that fixed threshold cannot adapt to multi-resolution scenes, significantly improving the offset detection rate, and effectively reducing the pathological diagnosis error caused by offset.

[0019] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0020] The drawings described herein are intended to provide further understanding of the present application, and constitute a part of the present application. The illustrative embodiments of the present application and their description serve to explain the present application, and do not constitute an improper limitation of the present application. In the drawings:

[0021] Figure 1 is a flowchart of a method for performing offset test on multi-layer fused digital pathology slices according to an embodiment of the present application;

[0022] Figure 2 is a structural block diagram of an apparatus for performing offset test on multi-layer fused digital pathology slices according to an embodiment of the present application;

[0023] Figure 3 is a hardware structure schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to represent the same elements in different drawings, unless otherwise specified. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with one or more embodiments of the present specification. Rather, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of the present specification, as detailed in the appended claims.

[0025] It should be noted that the steps of the respective methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the steps included in its method can be more or less than described in this specification. In addition, a single step described in this specification can be broken down into multiple steps for description in other embodiments; and multiple steps described in this specification can be combined into a single step for description in other embodiments.

[0026] Embodiment one

[0027] The embodiment of the present application provides a method for offset testing of a multi-layer fusion digital pathological section. The method realizes sub-pixel level offset detection through the technology of global offset matrix after sub-pixel refinement of feature points of different focal planes, and sets a dynamic offset threshold based on different scaling factors, thereby overcoming the defect that a fixed threshold cannot adapt to a multi-resolution scene. Specifically, referring to Figure 1 , the method comprises:

[0028] Obtaining a multi-layer fusion digital pathological section of a to-be-tested sample under a current scaling factor, performing multi-scale detail enhancement on cell nuclei in the multi-layer fusion digital pathological section to obtain a multi-layer cell nucleus segmentation image;

[0029] Extracting feature points from each focal plane in the multi-layer cell nucleus segmentation image, obtaining a focal plane as a reference layer, and focal planes other than the reference layer as to-be-tested layers, matching the feature points of each to-be-tested layer with the feature points of the reference layer to obtain a feature point matching result;

[0030] Sub-pixel refining each feature point in the to-be-tested layers and the reference layer in the feature point matching result based on a function fitting mode, and calculating a global offset matrix of each to-be-tested layer relative to the reference layer based on the sub-pixel refinement result;

[0031] Obtaining a dynamic offset threshold under the current scaling factor, wherein the dynamic offset threshold is negatively correlated with the scaling factor, obtaining an offset amount of each to-be-tested layer based on the global offset matrix, and if the offset amount is greater than the dynamic offset threshold, it is considered that the corresponding to-be-tested layer has an offset.

[0032] In some specific embodiments, a multi-layer fusion digital pathological section of a to-be-tested sample is obtained using a microscopic scanning device.

[0033] In some specific embodiments, before performing multi-scale detail enhancement on the cell nuclei in the multi-layer fusion digital pathological section, the multi-layer fusion digital pathological section is subjected to histogram equalization, wherein the multi-layer fusion digital pathological section is uniformly blocked, and each block is subjected to histogram equalization.

[0034] Specifically, in the step of histogram equalization on the multi-layer fused digital pathological section, the multi-layer fused digital pathological section is first divided into 8x8 local blocks, and histogram equalization is performed in each block.

[0035] Specifically, in the step of histogram equalization on each block, the contrast of each block is limited by setting a threshold value, so as to avoid excessive amplification of noise, enhance the differentiation degree of the nuclei and stroma in the low-contrast region, and improve the color difference between the alkali (blue) and acid (pink) regions in the H&E stained section. In this scheme, the threshold value is set as follows: clipLimit=3.0 is set, and the signal-to-noise ratio of the section image after histogram equalization is improved from 35.2 dB to 42.6 dB by limiting the contrast of each block.

[0036] In some embodiments, multi-scale feature extraction is performed on the multi-layer fused digital pathological section based on a Laplacian pyramid to obtain a multi-scale section image, the multi-scale section image and the multi-layer fused digital pathological section are weighted and fused to obtain a weighted fusion image, and cell nucleus segmentation is performed on the weighted fusion image to obtain a multi-layer cell nucleus segmentation image.

[0037] Further, in the step of performing multi-scale feature extraction on the multi-layer fused digital pathological section based on a Laplacian pyramid to obtain a multi-scale section image, a three-layer Gaussian pyramid is constructed to perform step-by-step downsampling on the multi-layer fused digital pathological section to obtain a first Gaussian image, a second Gaussian image, and a third Gaussian image. The third Gaussian image is upsampled and subtracted from the second Gaussian image to obtain a first Laplacian image. The first Laplacian image is upsampled and subtracted from the first Gaussian image to obtain a second Laplacian image. The second Laplacian image is upsampled and subtracted from the multi-layer fused digital pathological section to obtain a multi-scale section image.

[0038] Specifically, the Gaussian pyramid is composed of three downsampling layers. In the first downsampling layer, the multi-layer fused digital pathological section is downsampled to a first Gaussian image. In the second downsampling layer, the first Gaussian image is downsampled to a second Gaussian image. In the third downsampling layer, the second Gaussian image is upsampled to a third Gaussian image. The first Gaussian image has a resolution of 1 / 2 of the multi-layer fused digital pathological section, the second Gaussian image has a resolution of 1 / 4 of the multi-layer fused digital pathological section, and the third Gaussian image has a resolution of 1 / 8 of the multi-layer fused digital pathological section.

[0039] Specifically, the Laplacian pyramid extracts high-frequency details at each scale by reconstructing the Gaussian image through upsampling and calculating the difference with the multi-layer fused digital pathology slice. In the present scheme, the third Gaussian is upsampled and then subtracted from the second Gaussian image to separate the cell tissue area distribution, the first Laplacian image is upsampled and then subtracted from the first Gaussian image to separate the cell gland structure, and the third Gaussian is upsampled and then subtracted from the second Gaussian image to separate the cell nucleus edge details. Specifically, the line width of the cell nucleus edge details is 3-5 μm, the size of the cell gland structure is 50-100 μm, and the size of the cell tissue area distribution is greater than 200 μm. Therefore, the present scheme can obtain multi-scale slice images highlighting different cell contents by processing images of different resolutions through the Laplacian pyramid.

[0040] In the present scheme, the formula for weighted fusion of the multi-scale slice image and the multi-layer fused digital pathology slice is as follows:

[0041]

[0042] wherein, is the weighted fusion image, is the multi-layer fused digital pathology slice, is the multi-scale slice image, that is, the weight of the multi-layer fused digital pathology slice is set to 0.7 and the weight of the multi-scale slice image is set to 0.3 in the present scheme. Through the method of weighted fusion, the cell nucleus edge sharpness can be improved and the visibility of mitochondria and other subcellular structures can be improved. According to actual verification, the cell nucleus edge sharpness index of the image after weighted fusion is improved from 0.78 to 0.91, and the visibility of mitochondria and other subcellular structures is improved by 40%.

[0043] In some embodiments, the weighted fusion image is subjected to cell nucleus segmentation through the threshold segmentation method and then subjected to morphological denoising processing to obtain a multi-layer cell nucleus segmentation image.

[0044] Specifically, the present scheme performs multi-scale detail enhancement on the cell nucleus of each focal plane layer in the multi-layer fused digital pathology slice to obtain a multi-layer cell nucleus segmentation image.

[0045] In some embodiments, all focal planes of the multi-layer cell nucleus segmentation image are obtained through interface analysis, slice data of each focal plane is obtained, feature points of a reference layer and feature points of a to-be-tested layer are extracted based on the slice data of each focal plane, the feature points of each to-be-tested layer are matched with the feature points of the reference layer, and mis-matched feature points are removed to obtain a feature point matching result.

[0046] Specifically, the multi-layer fusion digital pathology section basic data focaLayerList is obtained through interface analysis, and focaLayerList is a set of all focal planes of the digital pathology section. The focaLayerList of each multi-layer fusion digital pathology section is initially fixed, for example, 【2, 1, 0, -1, -2】 represents that the multi-layer fusion digital pathology section has a total of 5 focal planes, wherein 0 is the reference layer, and 2, 1, -1, and -2 are the to-be-tested layers, that is, the reference layer is automatically obtained when the multi-layer fusion digital pathology section is obtained, and the focal plane layers other than the reference layer are to-be-tested layers.

[0047] Specifically, in the step of obtaining the section data of each focal plane, the focaLayer parameter is set to 0, so that the system automatically obtains the section data of the reference layer focal plane. Similarly, the focaLayer parameters are set to 1, 2, -1, and -2, respectively, so that the section data of the other to-be-tested layers is obtained in sequence.

[0048] Further, the feature points extracted from the reference layer and the to-be-tested layers are scale-invariant feature points, and the maximum suppression method is used to constrain the number of feature points on the reference layer and each to-be-tested layer when the feature points are extracted.

[0049] For example, the present scheme uses a FLANN matcher to match the feature points of each to-be-tested layer with the feature points of the reference layer, and uses a RANSAC algorithm to remove the mis-matched feature points to obtain a feature point matching result. The FLANN matcher and the RANSAC algorithm are prior art for matching and removing mis-matched feature points, and will not be described in detail here. The present scheme can also use other feature point matching methods and removal methods to match and remove the feature points of the reference layer and the to-be-tested layers.

[0050] Specifically, the formula for removing mis-matched feature points using the RANSAC algorithm is as follows:

[0051]

[0052] wherein k is the number of iterations, is the confidence level, and in the present scheme, is 0.99, is the mis-matching rate, The initial value of s is 0.5, and s is the minimum sample size, and in the present scheme, s = 4.

[0053] Since the present scheme matches the feature points of each to-be-tested layer with the feature points of the reference layer, the feature point matching result in the present scheme includes the matching between the feature points of each to-be-tested layer and the feature points of the reference layer.

[0054] In some embodiments, the core of sub-pixel refinement is to refine the initial corner coordinates to the sub-pixel level through local gray gradient analysis and iterative optimization. The pseudo code for sub-pixel refinement of each feature point in the to-be-tested layer and the reference layer in the feature point matching result is as follows:

[0055] criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30,0.001)

[0056] corners_refined = cv2.cornerSubPix(gray_img, matched_points, (5,5),(-1,-1), criteria)

[0057] Specifically, the matching accuracy of the feature points can reach 0.1 pixel level through sub-pixel refinement, and the use of the result after sub-pixel refinement for calculation of the global offset matrix can make the offset accuracy of the to-be-tested layer and the reference layer reach 0.1 pixel level, so as to better find the matching accuracy under different magnifications.

[0058] Further, the feature points can be sub-pixel refined in any way, and the specific technical means of sub-pixel refinement is not limited in the present scheme.

[0059] In some embodiments, the horizontal offset, the vertical offset and the angle offset of each corresponding feature point in the to-be-tested layer and the reference layer are calculated to obtain a global offset matrix, and the global offset matrix is used to represent the offset between the corresponding to-be-tested layer and the reference layer.

[0060] Specifically, the offset between the to-be-tested layer and the reference layer can be obtained through the global offset matrix, and in addition, the present scheme causes a local offset heat map based on the global offset matrix, and the offset between each feature point in the to-be-tested layer and the corresponding feature point in the reference layer is obtained based on the local offset heat map.

[0061] Exemplarily, the global offset matrix is represented as:

[0062]

[0063] wherein, is 1.2 μm, indicating that the offset between the to-be-tested layer and the reference layer in the horizontal X axis is 1.2 μm, is 0.8 μm, indicating that the offset between the to-be-tested layer and the reference layer in the horizontal Y axis is 0.8 μm, is -0.3 μm, indicating that the offset between the to-be-tested layer and the reference layer in the vertical direction is -0.3 μm, is 0.5°, indicating that the offset angle of the to-be-tested layer and the reference layer is 0.5°.

[0064] In some specific embodiments, the calculation formula of the dynamic offset threshold is as follows:

[0065]

[0066] wherein, is the dynamic offset threshold, is the preset offset threshold, is the pixel resolution of the current to-be-tested layer, is the pixel resolution of the reference layer, is the proportion of the nucleus region in the multi-layer fused digital pathology section, is the difference between the current environmental temperature and the standard temperature.

[0067] Specifically, different preset offset thresholds are set for different scaling magnifications, so that the dynamic offset threshold is negatively correlated with the scaling magnification. For example, the preset offset threshold is ±1 μm at a scaling magnification of 80X, the preset offset threshold is ±2 μm at a scaling magnification of 40X, and the preset offset threshold is ±5 μm at a scaling magnification of 20X.

[0068] Specifically, and The value range of is 0.1-1.0 μm / pixel.

[0069] Specifically, when calculating , the nucleus region in the multi-layer fused digital pathology section is segmented first, and then is calculated through and the formula.

[0070]

[0071] wherein, is the number of pixels of the nucleus region in the multi-layer fused digital pathology section, is the total number of pixels in the multi-layer fused digital pathology section.

[0072] In some specific embodiments, the offset amount of each to-be-tested layer is obtained based on the global offset matrix, and the offset amount is compared with the dynamic offset threshold. The comparison result is output in the form of a JSON report. Specifically, the content output by the JSON report also includes the offset region, the re-measurement suggestion, and the like. That is, when the offset amount is greater than the dynamic offset threshold, it is suggested to reacquire the multi-layer fused digital pathology section. If the offset amount is not greater than the dynamic offset threshold, it is indicated that the current acquired multi-layer fused digital pathology section is relatively accurate, and re-measurement is not required.

[0073] Embodiment Two

[0074] Based on the same idea, referring to Figure 2 The application also provides a method and device for performing offset testing on a multi-layer fusion digital pathological section, comprising:

[0075] An acquisition module is configured to acquire a multi-layer fusion digital pathological section at a current zoom ratio, and perform multi-scale detail enhancement on cell nuclei in the multi-layer fusion digital pathological section to obtain a multi-layer cell nucleus segmentation image.

[0076] A feature extraction module is configured to extract feature points from each focal plane in the multi-layer cell nucleus segmentation image, acquire a focal plane as a reference layer, and acquire feature point matching results by matching feature points in each test layer with feature points in the reference layer, wherein the focal planes other than the reference layer are the test layers.

[0077] A sub-pixel refinement module is configured to perform sub-pixel refinement on each feature point in the test layers and the reference layer in the feature point matching results based on a function fitting manner, and calculate a global offset matrix of each test layer relative to the reference layer based on the sub-pixel refinement results.

[0078] An offset detection module is configured to acquire a dynamic offset threshold at the current zoom ratio, wherein the dynamic offset threshold is negatively correlated with the zoom ratio, acquire an offset amount of each test layer based on the global offset matrix, and determine that an offset exists in the corresponding test layer if the offset amount is greater than the dynamic offset threshold.

[0079] Embodiment Three

[0080] The embodiment also provides an electronic device, referring to Figure 3 comprising a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to execute the computer program to perform the steps in any of the method embodiments.

[0081] Specifically, the processor 402 can include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the application.

[0082] The memory 404 can include a mass storage that stores data or instructions. For example, and without limitation, the memory 404 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 404 can be removable and / or non-removable (or fixed) as appropriate. The memory 404 can be internal or external as appropriate. In particular embodiments, the memory 404 is a Non-Volatile memory. In particular embodiments, the memory 404 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random Access Memory (FPMDRAM), an Extended Data Output Dynamic Random Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0083] The memory 404 can be used to store or cache various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 402.

[0084] The processor 402 can implement the method of any one of the above embodiments for testing the shift of the multi-layer fused digital pathology section by reading and executing the computer program instructions stored in the memory 404.

[0085] Optionally, the electronic device can further include a transmission device 406 connected to the processor 402 and an input / output device 408 connected to the processor 402.

[0086] The transmission device 406 can be used to receive or send data via a network. The network can include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0087] The input / output device 408 is used to input or output information. In the present embodiment, the input information can be the multi-layer fused digital pathology section, the magnification, etc., and the output information can be the shift result of each test layer, etc.

[0088] Optionally, in the present embodiment, the processor 402 can be configured to perform the following steps by computer program:

[0089] Obtain the multi-layer fused digital pathology section at the current magnification, and perform multi-scale detail enhancement on the cell nuclei in the multi-layer fused digital pathology section to obtain a multi-layer cell nucleus segmentation image;

[0090] Extract feature points from each focal plane in the multi-layer cell nucleus segmentation image, obtain a focal plane as a reference layer, and focal planes other than the reference layer are test layers, match the feature points of each test layer with the feature points of the reference layer to obtain a feature point matching result;

[0091] Sub-pixel refine each feature point in the feature point matching result in the test layer and the reference layer based on a function fitting manner, and calculate a global shift matrix of each test layer relative to the reference layer based on the sub-pixel refinement result;

[0092] Acquire a dynamic offset threshold under a current zoom ratio, wherein the dynamic offset threshold is negatively correlated with the zoom ratio, acquire an offset amount of each to-be-tested layer based on a global offset matrix, and if the offset amount is greater than the dynamic offset threshold, it is considered that the corresponding to-be-tested layer has an offset.

[0093] It should be noted that the specific examples in the embodiments can refer to the examples described in the above embodiments and optional implementation manners, and the embodiments will not be described here.

[0094] Generally, various embodiments can be implemented in hardware or special-purpose circuitry, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented in firmware or software to be executed by a controller, microprocessor or other computing device, although the application is not limited thereto. While various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controlers or other computing devices, or some combination thereof.

[0095] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including software routines, applets and / or macros, can be stored in any apparatus-readable data storage medium and they include program instructions to implement specific tasks. The program product can include one or more computer-executable components. The one or more computer-executable components can be at least one software code or portions thereof. Further, in this regard, it should be noted that any of the Figure 3 Any block in the logic flow of the method described herein can represent a module, segment, or portion of code which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each of the individual blocks of the application can translate to computer readable code or instructions that implement the specifi c logical function in a manner based on any available hardware or equivalents thereof.

[0096] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any combination, and in order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, it should be considered that they are within the scope of the description.

[0097] The above embodiments only express several implementation ways of the present application, and the description is specific and detailed, but it should not be understood as a limitation to the scope of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for performing offset testing on multi-layer fused digital pathology slices, characterized in that: The method comprises the following steps: obtaining a multi-layer fused digital pathological section under a current zooming ratio, performing multi-scale detail enhancement on nuclei in the multi-layer fused digital pathological section to obtain a multi-layer nucleus segmentation image; extracting feature points from each focal plane in the multi-layer nucleus segmentation image, obtaining a focal plane as a reference layer, and obtaining a focal plane other than the reference layer as a to-be-tested layer, matching the feature points of each to-be-tested layer with the feature points of the reference layer to obtain a feature point matching result; performing sub-pixel refinement on each feature point in the to-be-tested layer and the reference layer in the feature point matching result in a function fitting manner, and calculating a global offset matrix of each to-be-tested layer relative to the reference layer based on the sub-pixel refinement result, wherein the global offset matrix is used to represent the offset amount between the corresponding to-be-tested layer and the reference layer, and the global offset matrix is calculated by calculating the horizontal offset, the vertical offset and the angle offset of all corresponding feature points in each to-be-tested layer and the reference layer; obtaining a dynamic offset threshold under the current zooming ratio, and the calculation formula of the dynamic offset threshold is as follows: wherein, is a dynamic offset threshold, is a preset offset threshold, is a pixel resolution of a current layer to be tested, is a pixel resolution of a reference layer, is a proportion of a nucleus region in a multi-layer fusion digital pathology slice, is a difference between a current ambient temperature and a standard temperature, the dynamic offset threshold is negatively correlated with a scaling factor, an offset amount of each layer to be tested is obtained based on a global offset matrix, and if the offset amount is greater than the dynamic offset threshold, it is considered that the corresponding layer to be tested has an offset.

2. The method of claim 1, wherein, Before performing multi-scale detail enhancement on the nuclei in the multi-layer fused digital pathological section, performing histogram equalization on the multi-layer fused digital pathological section, wherein the multi-layer fused digital pathological section is uniformly divided into blocks, and each block is subjected to histogram equalization.

3. The method of claim 1, wherein, Performing multi-scale feature extraction on the multi-layer fused digital pathological section based on a Laplacian pyramid to obtain a multi-scale section image, performing weighted fusion on the multi-scale section image and the multi-layer fused digital pathological section to obtain a weighted fusion image, and performing nucleus segmentation on the weighted fusion image to obtain a multi-layer nucleus segmentation image.

4. The method of claim 3, wherein, Constructing a three-layer Gaussian pyramid to perform step-by-step down-sampling on the multi-layer fused digital pathological section to obtain a first Gaussian image, a second Gaussian image and a third Gaussian image, performing up-sampling on the third Gaussian to obtain a first Laplacian image by subtracting the second Gaussian image, performing up-sampling on the first Laplacian image to obtain a second Laplacian image by subtracting the first Gaussian image, and performing up-sampling on the second Laplacian image to obtain a multi-scale section image by subtracting the multi-layer fused digital pathological section.

5. The method of claim 1, wherein, Obtaining all focal planes of the multi-layer nucleus segmentation image through interface analysis, obtaining section data of each focal plane, extracting feature points of the reference layer and feature points of the to-be-tested layer based on the section data of each focal plane, matching the feature points of each to-be-tested layer with the feature points of the reference layer, and eliminating mis-matched feature points to obtain a feature point matching result.

6. The method of claim 5, wherein, The feature points extracted in the reference layer and the to-be-tested layer are scale-invariant feature points, and the maximum suppression method is used to constrain the number of feature points on the reference layer and each to-be-tested layer when the feature points are extracted.

7. A device for performing offset testing on multi-layer fused digital pathology slices, characterized in that: The method comprises the following steps: an obtaining module, configured to obtain a multi-layer fused digital pathological section under a current zooming ratio, and perform multi-scale detail enhancement on nuclei in the multi-layer fused digital pathological section to obtain a multi-layer nucleus segmentation image; The feature extraction module is configured to perform feature point extraction on each focal plane in the multi-layer cell nucleus segmentation image, take a focal plane with the highest definition as a reference layer, take focal planes other than the reference layer as to-be-tested layers, and perform matching between feature points of each to-be-tested layer and feature points of the reference layer to obtain a feature point matching result. The sub-pixel refinement module is configured to perform sub-pixel refinement on each feature point in the to-be-tested layers and the reference layer in the feature point matching result based on a function fitting manner, and calculate a global offset matrix of each to-be-tested layer relative to the reference layer based on a sub-pixel refinement result, wherein the global offset matrix is calculated by calculating horizontal offset, vertical offset and angle offset of all corresponding feature points in each to-be-tested layer and the reference layer, and the global offset matrix is used to represent an offset amount between the corresponding to-be-tested layer and the reference layer. The offset detection module is configured to obtain a dynamic offset threshold under a current zooming multiple, and a calculation formula of the dynamic offset threshold is as follows: wherein, is a dynamic offset threshold, is a preset offset threshold, is a pixel resolution of a current layer to be tested, is a pixel resolution of a reference layer, is a proportion of a nucleus region in a multi-layer fusion digital pathology slice, is a difference between a current ambient temperature and a standard temperature, the dynamic offset threshold is negatively correlated with a scaling factor, an offset amount of each layer to be tested is obtained based on a global offset matrix, and if the offset amount is greater than the dynamic offset threshold, it is considered that the corresponding layer to be tested has an offset. 8.An electronic device comprising a memory and a processor, the electronic device comprising: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for testing the offset of the multi-layer fusion digital pathological section according to any one of claims 1-6. The memory stores a computer program, and the processor is configured to run the computer program to execute the method for testing the offset of the multi-layer fusion digital pathological section according to any one of claims 1-6.

Images