A pathological section three-dimensional scanning system and method

Abstract

The application discloses a pathological section three-dimensional scanning system and method, and relates to the medical image field.The three-dimensional scanning system comprises a scanning module, an image processing module, a three-dimensional reconstruction module and a cloud uploading and pathological diagnosis module.The three-dimensional scanning method utilizes deep learning, uses a neural network to automatically learn the parameters of spatial transformation, selects a suitable spatial transformation and a similarity measurement to change the position, direction and shape attribute of an image, and after the reconstructed pathological three-dimensional structure model is uploaded and downloaded in the cloud, the pathological model can be observed and diagnostic modification opinions can be given, the structure of pathological tissues and the pathological characteristics are understood, and the ability and accuracy of pathological diagnosis are improved.

Description

Technical Field

[0001] This invention relates to the field of medical imaging, and more particularly to a three-dimensional scanning system and method for pathological slides. Background Technology

[0002] Medical imaging equipment is currently widely used in pathological diagnosis, but two-dimensional images can only display limited information. Therefore, three-dimensional pathological slide images are needed to make lesions more intuitive, which is conducive to the precision and speed of treatment.

[0003] Three-dimensional pathological slide images provide more spatial information. Compared to traditional two-dimensional slides, three-dimensional images offer more comprehensive and accurate anatomical and pathological information. Through three-dimensional images, tissue structures and lesions can be observed and analyzed on different planes. This multi-planar observation allows for a better understanding of the morphology, size, and positional relationships of lesions. Three-dimensional pathological slide images can be presented using visualization technology, allowing for a more intuitive and three-dimensional observation and analysis of lesions. Furthermore, interactive operations allow for browsing and manipulating three-dimensional images to obtain more detailed information and a more comprehensive understanding. However, pancreatic tissue exhibits significant variations in morphology and size, with unclear edges, making registration difficult. Moreover, the inability to automatically select appropriate spatial transformations necessitates manual intervention, greatly reducing reconstruction efficiency and accuracy. Therefore, it is impossible to reconstruct a three-dimensional model from pathological slides.

[0004] For example, Chinese patent CN111863202A discloses a digital pathological image scanning and analysis system. This system analyzes digital slide images obtained from an external scanning device and produces results without human intervention. Doctors can easily view the digital slide images and annotate them as needed for subsequent diagnostic reference. However, the digital pathological slide scanning device in this system processes a large number of samples simultaneously, limiting it to planar image processing of pathological slides. It cannot construct three-dimensional models for each slide individually, thus hindering more accurate and detailed pathological analysis.

[0005] Chinese patent CN114869315A discloses a method and system for three-dimensional model reconstruction and visualization of pancreatic lesions. This method includes: acquiring layered enhanced grayscale data of the abdomen from CT data of the target object and performing noise reduction, smoothing, and optimization processing to obtain the boundaries of each layer of pancreatic necrotic tissue, fluid accumulation, and pancreas with their corresponding adjacent organs; outlining the boundary contours of each layer of pancreatic necrotic tissue, fluid accumulation, and pancreas, and calculating the three-dimensional data information of the pancreatic necrotic tissue, fluid accumulation, and pancreas; repeating the above steps for organs adjacent to the region containing the pancreatic necrotic tissue, fluid accumulation, and pancreas to obtain the three-dimensional data information of the corresponding adjacent organs; reconstructing the three-dimensional model of the target object's pancreatic necrotic tissue, fluid accumulation, pancreas, and their corresponding adjacent organs, and then visualizing it. However, this method can only perform three-dimensional reconstruction based on CT images and data. Due to the significant differences in the shape and size of pathological slide images, unclear edges, and the difficulty in registration, and the inability to automatically select appropriate spatial transformations, manual intervention is required, greatly reducing reconstruction efficiency and accuracy. Therefore, it cannot reconstruct a three-dimensional model from pathological slides.

[0006] The existing technologies described above all suffer from the problems raised in this background: existing pathological slide images are difficult to register and cannot be reconstructed in three dimensions based on pathological slide images. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, this invention provides a three-dimensional scanning system and method for pathological slides. The three-dimensional scanning system includes a scanning module, an image processing module, a three-dimensional reconstruction module, and a cloud upload and pathological diagnosis module. During three-dimensional reconstruction, two-dimensional images are superimposed to construct a three-dimensional structural model. The three-dimensional scanning method utilizes deep learning, using neural networks to automatically learn the parameters of spatial transformation, and selects appropriate spatial transformations and similarity measures to change the position, orientation, and shape attributes of the images. The reconstructed three-dimensional pathological structural model is then uploaded and downloaded to the cloud.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] This invention provides a three-dimensional scanning system for pathological slides, comprising:

[0010] The scanning module scans and captures pathological slides to obtain digital pathological slide images;

[0011] The image processing module preprocesses the scanned and captured digital pathological slides, and then registers and segments the preprocessed digital pathological slide images.

[0012] The 3D reconstruction module reconstructs a 3D pathological structural model from the pre-processed, registered and cut digital pathological slide images through 2D overlay and voxel rendering.

[0013] The cloud upload and pathology diagnosis module uploads the obtained three-dimensional pathological structural model file to the cloud, allows users to download and view the three-dimensional pathological structural model through the cloud and perform pathological diagnosis, and finally uploads the pathological diagnosis results to the cloud.

[0014] Preferably, the scanning module includes a resolution setting unit, a scan range setting unit, a color mode setting unit, and an image format setting unit. The resolution setting unit is used to set the current resolution of the scanner; the scan range setting unit is used to modify the scan range of the scanner to scan the pathological slide portion; the color mode setting unit is used to modify the color mode of the scanner to color mode; and the image format setting unit is used to modify the image format of the digital pathological slide image obtained after scanning.

[0015] Preferably, the image processing module includes a preprocessing unit, a registration unit, and a cutting unit. The preprocessing unit is used to smooth and enhance the digital pathological slide images obtained after scanning; the registration unit is used to register misaligned digital pathological slide images; and the cutting unit is used to extract the contours of each tissue and generate two-dimensional segmentation labels.

[0016] Preferably, the 3D reconstruction module includes a 3D reconstruction algorithm unit and a voxel rendering unit. The 3D reconstruction unit is used to reconstruct digital pathological slide images into a 3D pathological structural model; the voxel rendering unit is used to visualize the 3D pathological structural model and present its structure and information.

[0017] Preferably, the cloud upload and pathological diagnosis module includes a creation unit, an upload unit, and a diagnosis unit; the creation unit is used to create a cloud storage bucket for uploading and downloading; the upload unit is used to upload the obtained pathological three-dimensional structural model and pathological diagnosis to the cloud storage bucket; the diagnosis unit is used to download the pathological three-dimensional structural model, perform pathological diagnosis, and generate documents.

[0018] This invention provides a method for three-dimensional scanning of pathological sections, comprising the following steps:

[0019] The pathological slides are scanned and captured to obtain digital pathological slide images;

[0020] The scanned and captured digital pathological slide images are preprocessed, and the preprocessed digital pathological slide images are registered and cut.

[0021] The preprocessed, registered and segmented digital pathological slide images are reconstructed into a three-dimensional pathological structural model after two-dimensional overlay and voxel rendering.

[0022] The obtained pathological three-dimensional structural model file is uploaded to the cloud. The pathological three-dimensional structural model is downloaded and viewed through the cloud, and a pathological diagnosis is performed. The pathological diagnosis results are then uploaded to the cloud.

[0023] The registration of the preprocessed digital pathological slide images includes the following steps:

[0024] Automatically selects spatial transformation, similarity measurement, and search strategy;

[0025] Extract features based on the patterns and structure of the original data;

[0026] Select a classification machine learning model and train it;

[0027] Using a trained classification machine learning model and evaluation metrics, and based on preset thresholds, the system makes decisions to select the best similarity measure and search strategy.

[0028] The steps for automatically selecting spatial transformation are as follows:

[0029] Select a set of input digital pathology slide images and corresponding target outputs as training data;

[0030] STN was chosen as the neural network architecture;

[0031] The mean squared error is chosen as the loss function to measure the difference between the network output and the target output. This includes calculating the difference between the predicted value and the target value, summing the squares of the calculated differences, and calculating the ratio to the total number of pixels to obtain the mean squared error.

[0032] After training, the trained network is used to perform spatial transformation on new digital pathological slide images. The new digital pathological slide images are input into the network to obtain output images with automatically learned spatial transformation parameters.

[0033] The steps for automatically selecting similarity metrics and search strategies are as follows:

[0034] Collect and prepare data for training and evaluating the model;

[0035] Define the local neighborhood, calculate the sum of the histograms, normalize each value in the histogram value list, and calculate the similarity measure;

[0036] First, SIFT feature extraction is performed. During keypoint detection, the Hessian matrix H(x,y,sigma) needs to be calculated using the following formula:

[0037]

[0038] Among them, L xx L xy and L yydenoted as the second derivative of the image after smoothing with a Gaussian filter at the current position, and sigma as the parameter controlling the smoothing degree during the calculation. After extracting SIFT features, the SIFT features in the first digital pathological slide image are matched with the SIFT features in the second digital pathological slide image. For each feature descriptor in the first digital pathological slide image, the distance between it and all feature descriptors in the second digital pathological slide image is calculated.

[0039] The registered digital pathological slide images are segmented to generate two-dimensional segmentation labels. During segmentation, isosurfaces are extracted and boundary positions are confirmed. In isosurface extraction, edge detection and curvature estimation are used in combination. Before isosurface extraction, curvature estimation is performed on the original digital pathological slide images to determine the isosurface threshold.

[0040] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0041] (1) A three-dimensional pathological structural model allows doctors to intuitively observe and understand the spatial relationship of tissues, and to observe the internal structure and characteristics of pathological tissues in detail. This helps to more accurately assess the nature, size and extent of the lesion, and can also provide accurate location of the lesion in three-dimensional space.

[0042] (2) The pathological three-dimensional structural model uploaded to the cloud can not only be accessed and interacted with remotely, but also used for medical education and pathology training. Through stereoscopic imaging, the structure and lesion characteristics of pathological tissues can be learned and understood more intuitively, thereby improving the ability and accuracy of pathological diagnosis.

[0043] (3) The pathological three-dimensional structural model can be observed more intuitively and diagnostic modification opinions can be given. The model uploaded to the cloud can also be accessed and used remotely for remote diagnosis. This is very beneficial for remote areas, emergency situations or scenarios that require expert opinions, and can speed up the diagnosis and improve patient care. Attached Figure Description

[0044] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1 This is a framework diagram of a three-dimensional scanning system for pathological slides according to the present invention;

[0046] Figure 2 This is a flowchart illustrating a three-dimensional scanning method for pathological sections according to the present invention. Detailed Implementation

[0047] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0048] Example 1:

[0049] like Figure 1 As shown, this embodiment provides a three-dimensional scanning system for pathological slides, specifically including:

[0050] The scanning module includes a resolution setting unit, a scan range setting unit, a color mode setting unit, and an image format setting unit;

[0051] The resolution setting unit allows you to select an appropriate scan resolution to balance image quality and file size. For pathological sections, a high resolution of 40x is selected to capture fine tissue details.

[0052] The scan range setting unit determines the area to be scanned; select to scan the entire slice to ensure the image includes the required anatomical structures and pathological features.

[0053] The color mode setting unit selects an appropriate color mode to meet the requirements; for pathological sections, the color mode is selected to preserve the staining information of the tissue.

[0054] The image format setting unit selects the appropriate image format based on requirements. Since high-resolution images are needed, the TIFF format, which provides lossless compression, is selected.

[0055] The image processing module includes a preprocessing unit, a registration unit, and a segmentation unit;

[0056] The preprocessing unit uses a noise reduction filter to smooth the image and reduce noise.

[0057] The registration unit extracts the contours of each tissue and automatically selects appropriate spatial transformations, similarity measures, and search strategies based on different data sources and modalities, so that different slices can be correctly aligned in spatial location.

[0058] The segmentation unit selects an adaptive threshold based on the image's grayscale value to segment the image into two parts, generating two-dimensional segmentation labels.

[0059] The 3D reconstruction module includes a 3D reconstruction algorithm unit and a voxel rendering unit;

[0060] The 3D reconstruction algorithm unit uses Unity and OpenCV libraries to overlay 2D images and reconstruct them into 3D structures. It imports pre-registered and segmented images, sets the texture mapping to appropriate positions and proportions, and adjusts it according to the thickness of the slices and the spacing between adjacent slices. It iterates through each pixel of the image, and for each pixel, it blends the corresponding pixel values ​​of the image according to a certain transparency. The higher the transparency, the greater the contribution of the image. The blended pixel values ​​are used as the corresponding pixel values ​​of the new image. In order to obtain a smoother and more continuous 3D structure, voxel interpolation is used to fill the space between slices. First, the size of the slice image is defined. Then, a 3D data is generated to store the slice image. The size of the target voxel is defined. By calculating the voxel interpolation ratio between slices, the interpolation ratio in each direction is determined. Finally, a pathological 3D structure model that fills the space between slices is generated.

[0061] The voxel rendering unit visualizes the generated 3D pathological structural model using voxel rendering technology based on isosurfaces, showcasing the three-dimensional shape and details of the tissue structure. The specific operation is as follows:

[0062] The three-dimensional data is divided into multiple voxels, each containing numerical information.

[0063] Based on the required extracted isosurface data, voxels are divided into two categories: voxels above the isosurface and voxels below the isosurface.

[0064] For each isosurface, find all intersections between voxels above and below the isosurface;

[0065] Based on the location and numerical information of the intersection points, the pixel values ​​of the two-dimensional image are mapped to the corresponding positions in the three-dimensional voxel grid.

[0066] Based on the pixel values ​​in the voxel data, the Marching Cubes isosurface extraction algorithm is used to determine the isosurfaces in the voxel mesh. The specific process is as follows:

[0067] The voxel data is divided into multiple cubic units, each containing 8 vertices and 12 edges. Based on the pixel values ​​in the voxel data, it is determined whether each vertex lies on an isosurface, i.e., whether the voxel containing the vertex crosses an isosurface.

[0068] Traverse all cube cells and determine the topology of the isosurface where the cube cell is located based on whether the voxel vertices are on the isosurface.

[0069] For each cube cell, the vertex position on the isosurface is calculated using linear interpolation based on the vertex pixel value and the threshold of the isosurface.

[0070] Suppose we have two known data points (x1, y1) and (x2, y2), and we need to perform linear interpolation at the position x between these two points to obtain the corresponding estimated value y.

[0071] The calculation steps are as follows:

[0072] Calculate the weighting factors for the interpolation position. Based on the positional relationship between the interpolation position x and the known data points, calculate weighting factors w1 and w2. Use linear weights, meaning they are inversely proportional to the distance. The weighting factors can be calculated using the following formula:

[0073]

[0074] We use weighted averages of the known data points to obtain an estimate of the interpolation location, y. The estimated interpolation location, y, is calculated using the following formula:

[0075] y = y1 × w1 + y2 × w2

[0076] Based on the topology of the isosurface and the vertex positions obtained through interpolation, a triangular mesh of the isosurface is generated. Depending on the different topologies, the connection method of the triangular vertices on each cube element can be determined.

[0077] Repeat the above steps until all cube units have been traversed, generating a complete triangular mesh.

[0078] Since pathological 3D structural models may require a large amount of computational resources, Unity's optimization techniques LOD and batch processing are used to ensure good performance during runtime. After the 3D structure is generated, Unity's camera and lighting components are used to adjust the viewpoint and lighting to better visualize and present the 3D structure.

[0079] The cloud-based upload and pathology diagnosis module includes a creation unit, an upload unit, and a diagnosis unit.

[0080] The creation unit involves creating a Google Cloud account and setting up a cloud storage bucket.

[0081] The upload unit uploads the model file (.h5 file type) and the pathological diagnosis results (.txt format) obtained after downloading and viewing to the cloud storage bucket via API;

[0082] By downloading and viewing the three-dimensional structural model of the pathology from the cloud, and using the diagnostic unit, a pathological diagnosis can be made based on the actual situation.

[0083] Example 2:

[0084] like Figure 2 As shown, this embodiment provides a three-dimensional scanning method for pathological slides, including the following steps:

[0085] The scanned and captured digital pathological slide images are preprocessed, and the preprocessed digital pathological slide images are registered and cut.

[0086] The preprocessed, registered and segmented digital pathological slide images are reconstructed into a three-dimensional pathological structural model after two-dimensional overlay and voxel rendering.

[0087] The obtained pathological three-dimensional structural model file is uploaded to the cloud. The pathological three-dimensional structural model is downloaded and viewed through the cloud and pathological diagnosis is performed. Finally, the pathological diagnosis results are uploaded to the cloud.

[0088] The registration of the preprocessed digital pathological slide images includes the following steps:

[0089] S1. The method for automatically selecting spatial transformations is as follows:

[0090] S11. Data Preparation: Prepare a set of training data containing the input image and the corresponding target output. The input image can be the original image, and the target output can be the image after spatial transformation.

[0091] S12. Construct the network structure, choosing STN as the neural network structure. STN contains three main components: localization network, mesh generator, and sampler;

[0092] Localization networks are used to learn the parameters of spatial transformations; they take an input image as input and output a vector or matrix representing the spatial transformation parameters.

[0093] The mesh generator generates a mesh representing the location of the sampling points based on the parameters output by the localization network.

[0094] The sampler performs sampling operations based on the generated grid and the input image to obtain the spatially transformed image;

[0095] S13. Define the loss function, choosing mean squared error as the loss function to measure the difference between the network output and the target output. The steps are as follows:

[0096] Mean squared error is chosen as the loss function. For each training sample, the image output by the network is compared with the target output image, and the pixel-level difference between them is calculated.

[0097] Calculate the difference between the predicted value y_pred and the target value y_target, y_pred - y_target. Then square these differences (y_pred - y_target). 2Summation involves adding the squares of all differences, then dividing the sum by the total number of pixels N (1 / N) to obtain the average value. Finally, the average value is the mean square error (MSE).

[0098] MSE calculates the mean squared difference between the predicted and target values. A larger difference results in a larger loss, while a smaller difference results in a smaller loss.

[0099] S14. Train the network using the training data. Optimize the network parameters using the backpropagation algorithm to make the network output image as close as possible to the target output. The steps are as follows:

[0100] (1) Forward propagation: The input data is passed from the input layer to the output layer through the neural network, and the output result of the network is calculated.

[0101] During forward propagation, each neuron receives the output of the neuron in the previous layer and applies an activation function for nonlinear transformation; the steps are as follows:

[0102] The activation function maps the input of a neuron to a non-linear output value to increase the expressive power of the network. The output of the activation function is used as the input of the next layer of neurons, and the weighted summation and activation function application continue in the next layer. The weighted summation and activation function application are repeated until the output layer of the network is reached. Finally, the output of the network can be represented as a combination of a series of matrix multiplications and activation functions.

[0103] (2) Calculate the loss by comparing the network output with the target output and using the mean square error to calculate the loss function value.

[0104] (3) Backpropagation: Propagate the error signal from the output layer to the input layer and calculate the gradient of each parameter with respect to the loss function.

[0105] The backpropagation algorithm uses the chain rule to calculate the gradient of each parameter. The gradient represents the rate of change of a parameter with respect to the loss function, and by calculating the gradient, the direction of parameter updates can be determined.

[0106] Backpropagation of gradients involves passing the gradient of the output layer to the previous layer and calculating the gradient for each neuron. The steps are as follows:

[0107] Multiply the gradient of the output layer by the weight matrix to obtain the gradient of the previous layer;

[0108] Based on the activation function of the previous layer, calculate the gradient dL / dz of the previous layer's neuron, where z is the weighted input of the previous layer and L represents the loss function;

[0109] Repeat these two steps to propagate the gradient from the previous layer to the layer before that, until it reaches the input layer.

[0110] (4) Update the network parameters based on the calculated parameter gradients;

[0111] (5) Repeat steps (1) to (4) to repeat the forward propagation, loss calculation, back propagation and parameter update process until the stopping condition is met.

[0112] S15. After training, the trained network can be used to perform spatial transformation on new digital pathology slide images. The new digital pathology slide image is input into the network to obtain an output image with automatically learned spatial transformation parameters.

[0113] S2. The method for automatically selecting similarity metrics and search strategies is as follows:

[0114] S21. Data preparation: Collect and prepare data for training and evaluating the model, including input data and corresponding similarity metrics and search strategies;

[0115] Methods for obtaining similarity measures include:

[0116] Because local texture information needs to be considered, Local Binary Pattern (LBP) is used as the feature descriptor here;

[0117] Define the local neighborhood by selecting a radius of 2 pixels and 8 neighboring points to define the local neighborhood of each pixel;

[0118] The LBP value is calculated by comparing the gray values ​​of neighboring pixels with the gray value of the center pixel. If the value is greater than or equal to the center pixel, it is recorded as 1, and if it is less than the center pixel, it is recorded as 0. These binary values ​​are arranged in clockwise or counterclockwise order to form a binary code. Since 8 neighboring points are used, an 8-bit binary code is obtained.

[0119] Construct an LBP histogram by traversing the entire image, calculating the LBP value for each pixel, and summing these LBP values ​​into a histogram. Each bin of the histogram represents an LBP pattern and records the frequency of that pattern in the image.

[0120] Normalization is performed to eliminate differences in brightness and contrast between different images, specifically including:

[0121] To calculate the sum of the histogram: First, calculate the sum of the values ​​of all bins in the histogram. This sum represents the number of pixels in the histogram.

[0122] Normalize the value of each bin: Iterate through each bin in the histogram, divide its value by the sum of the histogram values, normalize the value of each bin, and convert the value of each bin into a relative frequency or probability;

[0123] Perform normalization: Ensure that the sum of all bin values ​​in the normalized histogram equals 1. Rounding errors or approximations during normalization may cause the sum to be slightly over or under 1. To correct this, the normalized histogram can be readjusted so that the sum of all bin values ​​equals 1.

[0124] Calculate the similarity measure: Use Euclidean distance as a similarity measure to compare the degree of similarity between two LBP histograms; that is, the similarity measure can be obtained for the provided digital pathological slice images.

[0125] Methods for obtaining search strategies include:

[0126] Global image matching is performed using the Scale Invariant Feature Transform (SIFT) algorithm, with the following steps:

[0127] For the two images to be matched, we first need to extract SIFT features from each image, as follows:

[0128] (1) Scale space construction: The difference-of-Gaussian pyramid method is used to construct the scale space at different scales of the image. By applying Gaussian blur and sampling operations at different scales, a series of images with different scales are obtained;

[0129] Starting with the original image, an initial image is created to serve as the first layer in scale space. This initial image is typically the result of the original image undergoing a Gaussian blur.

[0130] A Gaussian pyramid is constructed by successively applying Gaussian filters and downsampling operations. Each layer is a downsampled version of the previous layer's image, and a Gaussian filter of a different scale is applied to each layer. This results in a series of images with different scales.

[0131] Difference images are calculated between adjacent layers of the Gaussian pyramid. For each scale, the image of the current layer is subtracted from its neighboring, smoother image to obtain a difference image. These difference images represent image detail information at different scales.

[0132] (2) Key point detection: In each scale space, the scale-invariant extreme value detection algorithm is used to find key points;

[0133] At each scale level, the pixel location is compared with its neighboring pixels and the corresponding pixels at adjacent scale levels. This comparison determines whether the pixel is a potential extreme point candidate.

[0134] For pixels selected as candidate extrema, sub-pixel precision localization is performed. Interpolation techniques are then used to locate the true extrema at a finer scale and position within the image.

[0135] Because edges are continuous, they generate a large number of feature points and are unstable. Therefore, the eigenvalues ​​of the Hessian matrix can be used to determine whether extreme points are located on edges and to remove key points of these edge responses.

[0136] For each keypoint location (x, y) and the selected scale level sigma, calculate the Hessian matrix H(x, y, sigma):

[0137]

[0138] Among them, L xx L xy and L yy represents the second derivative of the image smoothed by the Gaussian filter at the current position, and sigma represents the parameter that controls the degree of smoothing during the calculation process;

[0139] For each calculated Hessian matrix H(x,y,sigma), its eigenvalues ​​λ1 and λ2 are calculated. The ratio of eigenvalues ​​λ1 and λ2 is used to determine whether the image region around the keypoint is an edge.

[0140]

[0141] If R exceeds the preset threshold, it indicates that the region may be located at the edge;

[0142] (3) Orientation Assignment: For each keypoint, calculate the gradient orientation histogram of its local image patch. Determine the main orientation based on the peak value of the gradient orientation histogram, and assign the orientation attribute of the keypoint to this main orientation;

[0143] (4) Feature Description: For each keypoint, a 128-dimensional feature vector is generated based on the gradient information of its surrounding region. This feature vector considers the gradient magnitude and direction near the keypoint and has scale invariance and rotation invariance;

[0144] (5) Feature point screening: Based on the contrast criterion, the extracted features are screened to remove low-quality or redundant feature points.

[0145] After extracting SIFT features, feature matching is performed, matching the SIFT features in the first digital pathology slide image with the SIFT features in the second digital pathology slide image. Nearest neighbor matching is used as the feature matching method; by calculating the distance between features, corresponding feature point pairs in the two images are found.

[0146] For each feature descriptor in the first digital pathology slide image, calculate its distance to all feature descriptors in the second digital pathology slide image. Since there may be false matches, set a threshold and use threshold-based elimination to filter out inaccurate matches.

[0147] Based on the selected matching feature point pairs, the least squares method can be used to calculate the transformation matrix between digital pathology slide images. Using the calculated transformation matrix, one digital pathology slide image can be transformed to align with another digital pathology slide image, thus enabling the two digital pathology slide images to be matched globally.

[0148] S22. Feature Engineering: Using automated feature extraction algorithms, representative features are automatically extracted from the original data based on its patterns and structure.

[0149] S23. Model Selection and Training: Select a classification machine learning model and train it using the prepared data;

[0150] S24. Automated Selection: Using a trained classification machine learning model and evaluation metrics, a decision-making system is established based on preset thresholds to automatically select the best similarity metric and search strategy.

[0151] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A three-dimensional scanning system for pathological slides, characterized in that, include: The scanning module scans and captures pathological slides to obtain digital pathological slide images; The image processing module preprocesses the scanned and captured digital pathological slide images, and then registers and segments the preprocessed digital pathological slide images. The 3D reconstruction module reconstructs a 3D pathological structural model from the pre-processed, registered and cut digital pathological slide images through 2D overlay and voxel rendering. The cloud upload and pathology diagnosis module uploads the obtained three-dimensional pathological structural model file to the cloud, downloads and views the three-dimensional pathological structural model through the cloud, performs pathological diagnosis, and uploads the pathological diagnosis results to the cloud. The registration of the preprocessed digital pathological slide images includes the following steps: Automatically select spatial transformation, similarity measurement, and search strategy; extract features based on the patterns and structure of the original data; select a classification machine learning model and train it. Using a trained classification machine learning model and evaluation metrics, and based on preset thresholds, the system makes decisions to select the best similarity measure and search strategy. The steps for automatically selecting spatial transformation are as follows: Select a set of input digital pathology slide images and corresponding target outputs as training data; STN is chosen as the neural network structure; mean squared error is chosen as the loss function to measure the difference between the network output and the target output, including calculating the difference between the predicted value and the target value, summing the squares of the calculated differences, and calculating the ratio to the total number of pixels to obtain the mean squared error. After training, the trained network is used to perform spatial transformation on new digital pathological slide images. The new digital pathological slide images are input into the network to obtain output images with automatically learned spatial transformation parameters.

2. The pathological slide three-dimensional scanning system according to claim 1, characterized in that: The scanning module includes a resolution setting unit, a scanning range setting unit, a color mode setting unit, and an image format setting unit; The resolution setting unit is used to set the current resolution of the scanner; the scan range setting unit is used to set the current scan range of the scanner, which is the pathological slide portion; the color mode setting unit is used to convert the colors output by the scanner; and the image format setting unit is used to set the image format of the digital pathological slide image obtained after scanning.

3. The pathological slide three-dimensional scanning system according to claim 2, characterized in that: The image processing module includes a preprocessing unit, a registration unit, and a cutting unit; The preprocessing unit is used to perform desmoothing and image enhancement operations on the scanned digital pathology slide images; the registration unit is used to register misaligned digital pathology slide images; and the cutting unit is used to extract the contours of each tissue and generate two-dimensional segmentation labels.

4. A three-dimensional scanning system for pathological slides according to claim 3, characterized in that: The 3D reconstruction module includes a 3D reconstruction algorithm unit and a voxel rendering unit; The three-dimensional reconstruction unit is used to reconstruct digital pathological slide images into three-dimensional pathological structural models; the voxel rendering unit is used to visualize the three-dimensional pathological structural models and present their structure and information.

5. A three-dimensional scanning system for pathological slides according to claim 4, characterized in that: The cloud upload and pathological diagnosis module includes a creation unit, an upload unit, and a diagnosis unit; The creation unit is used to create cloud storage buckets for uploading and downloading; the upload unit is used to upload the obtained pathological three-dimensional structural model and pathological diagnosis to the cloud storage bucket; and the diagnosis unit is used to download the pathological three-dimensional structural model, perform pathological diagnosis, and generate documents.

6. A method for three-dimensional scanning of pathological slides, implemented based on a three-dimensional scanning system for pathological slides according to any one of claims 1 to 5, characterized in that, Includes the following steps: The pathological slides are scanned and captured to obtain digital pathological slide images; The scanned and captured digital pathological slide images are preprocessed, the preprocessed digital pathological slide images are registered, the registered digital pathological slide images are cut, and two-dimensional segmentation labels are generated. The preprocessed, registered and segmented digital pathological slide images are reconstructed into a three-dimensional pathological structural model after two-dimensional overlay and voxel rendering. The obtained pathological three-dimensional structural model file is uploaded to the cloud. The pathological three-dimensional structural model is downloaded and viewed through the cloud, and a pathological diagnosis is performed. The pathological diagnosis results are then uploaded to the cloud.

7. A method for three-dimensional scanning of pathological sections according to claim 6, characterized in that, Automatically select similarity metrics and search strategies, including the following steps: Collect and prepare data for training and evaluating the model; Define the local neighborhood, calculate the sum of the histograms, normalize each value in the histogram value list, and calculate the similarity measure; First, SIFT feature extraction is performed. Then, the Hessian matrix needs to be calculated during keypoint detection. The calculation formula is: ； in, , and Let represent the second derivative of the image at the current position after smoothing with a Gaussian filter, respectively. The parameter represents the smoothness during the calculation process; after extracting SIFT features, the SIFT features in the first digital pathological slide image are matched with the SIFT features in the second digital pathological slide image. For each feature descriptor in the first digital pathological slide image, the distance between it and all feature descriptors in the second digital pathological slide image is calculated.

8. A method for three-dimensional scanning of pathological sections according to claim 6, characterized in that: The registered digital pathological slide images are segmented to generate two-dimensional segmentation labels. During segmentation, isosurfaces are extracted and boundary positions are confirmed. In isosurface extraction, edge detection and curvature estimation are used in combination. Before isosurface extraction, curvature estimation is performed on the original digital pathological slide images to determine the isosurface threshold.

Images