A data processing method for a specimen digital management platform

Abstract

The application relates to the technical field of electric data processing, in particular to a data processing method for a specimen digital management platform; the method comprises the following steps: performing multi-scale Tucker decomposition on a three-order data tensor to obtain a low-rank component representing a tissue structure, and taking the low-rank component as a correction image; extracting a semantic vector of unstructured clinical text by using a preset bidirectional long short-term memory network; inputting a graph embedding vector, a topological feature vector and the semantic vector into a multi-modal fusion network, and calculating a unified multi-modal index code by using a hash function; and encapsulating an identifier of the correction image, the graph embedding vector, the semantic vector, the multi-modal index code and specimen metadata into a structured data entity. The application has the effect of facilitating accurate analysis of data.

Description

Technical Field

[0001] This application relates to the field of electrical data processing technology, and in particular to a data processing method for a specimen digitization management platform. Background Technology

[0002] With the popularization of digital pathology technology, whole-slide images (WSI) have become the main data form for pathological diagnosis, teaching, and research. These high-resolution images are massive in volume, posing significant challenges to the digital management, storage, and analysis of specimens. During digital scanning, due to factors such as slide preparation, staining, or scanning equipment, WSI often contains various imaging artifacts, including color deviation, blurring, and tissue folding. These artifacts not only affect the subjective interpretation of pathologists but also severely interfere with the performance of subsequent computer-aided analysis algorithms, leading to a decrease in the accuracy of tasks such as cell segmentation and feature extraction. Existing image preprocessing methods mostly employ global color normalization or simple filtering techniques, which are insufficient to effectively distinguish and remove complex or sparse artifacts mixed with tissue structures, thus limiting the potential for extracting high-fidelity biological information from images.

[0003] Furthermore, pathological diagnosis relies not only on image information but also on the patient's clinical textual records (such as pathology reports and medical history). However, these clinical texts are often unstructured, making them difficult to use directly for quantitative analysis and efficient retrieval. Currently, most specimen management platforms manage image and text data as independent information sources. This fragmented data processing ignores the inherent connection between pathological morphological features and clinical semantic information, failing to construct a comprehensive and unified case profile. This impacts the accurate analysis and judgment of the patient's condition, or the precise analysis of cases during teaching. Summary of the Invention

[0004] To facilitate more accurate analysis of case conditions during teaching, this application provides a data processing method for a specimen digitization management platform.

[0005] This application provides a data processing method for a specimen digitization management platform, employing the following technical solution:

[0006] A data processing method for a specimen digitization management platform includes: acquiring the original digital image of the specimen and associated unstructured clinical text; dividing the original digital image into multiple overlapping image blocks and constructing a third-order data tensor; performing multi-scale Tucker decomposition on the third-order data tensor to obtain low-rank components representing tissue structure, and using the low-rank components as the correction image; acquiring the cell nucleus positions and types in the correction image, and constructing a K-nearest neighbor graph based on the Euclidean distance between cell nuclei; performing convolution processing on the K-nearest neighbor graph using a graph attention network to output a graph embedding vector representing the spatial relationships of the pathological microenvironment; simultaneously calculating the persistent homology of simple complexes composed of specific cell types in the K-nearest neighbor graph to generate a topological feature vector that quantifies the global topological structure; extracting the semantic vector of the unstructured clinical text using a pre-built bidirectional long short-term memory network; inputting the graph embedding vector, topological feature vector, and semantic vector into a multimodal fusion network, and calculating a unified multimodal index code using a hash function; and encapsulating the identifier of the correction image, the graph embedding vector, the semantic vector, the multimodal index code, and the specimen metadata into a structured data entity.

[0007] Through multi-scale Tucker decomposition, tissue structures and imaging artifacts are effectively separated, enabling image correction and improving structural fidelity. A K-nearest neighbor graph based on cell nucleus location and type is constructed, and microscopic spatial relationships are learned by combining graph attention networks. Furthermore, continuous homology quantization of global topological morphology is introduced to form a multi-level image structure description. Combined with text semantic vectors extracted by Bi-LSTM, a multi-modal fusion network is used to uniformly encode them into hash index codes. Finally, all key information is encapsulated into structured data entities, supporting efficient, cross-modal intelligent retrieval and analysis, promoting the upgrade of specimen management to structured and intelligent methods, and facilitating precise teaching and learning.

[0008] Optionally, the original digital image is divided into multiple overlapping image patches and a third-order data tensor is constructed, including:

[0009] The original digital image is segmented using a sliding window method, so that there is a preset proportion of overlap between any two adjacent overlapping image blocks in the horizontal or vertical direction; the RGB three channels of each overlapping image block form a three-dimensional array, and all overlapping image blocks are stacked to form a third-order data tensor.

[0010] The sliding window method divides the original digital image into multiple overlapping image blocks, effectively enhancing the continuity between these blocks and ensuring that microstructures such as cell nuclei are not truncated or cropped at the edges. This improves the integrity and stability of subsequent image tensor construction and analysis. The design of the overlapping regions helps preserve image contextual information, making image details and edge consistency more coherent in tensor decomposition, cell nucleus identification, and graph modeling, thus improving the overall processing quality.

[0011] Optional, the default ratio is 25%.

[0012] Optionally, in the process of performing multi-scale Tucker decomposition on the third-order data tensor to obtain the low-rank components representing the organizational structure: the third-order data tensor is decomposed at at least two different resolution scales, and an iterative optimization algorithm is used to solve the core tensor and factor matrix until the preset convergence condition is met.

[0013] A Tucker decomposition of the third-order image tensor is performed at multiple resolution scales, and an iterative optimization algorithm is used to solve for the core tensor and factor matrix, achieving simultaneous modeling of the macroscopic and microscopic structures of the image. The multi-scale decomposition mechanism can extract image features at different levels, improving the accuracy of artifact removal and tissue representation; while the iterative solution ensures the robustness and accuracy of the model under complex image structures, laying a solid foundation for generating artifact-free corrected images.

[0014] Optionally, the core tensor can be multiplied by the factor matrix to reconstruct the low-rank component, which can then be used as the corrected image.

[0015] By reconstructing the low-rank components by multiplying the core tensor with the factor matrix, and using this as a correction image, imaging artifacts are effectively removed while preserving the true tissue structure information.

[0016] Optionally, set the nearest neighbor number K, and for each cell nucleus node, determine the K nearest neighbor nodes based on its spatial coordinates and establish connections; encode the cell nucleus type as the initial node feature vector.

[0017] A K-nearest neighbor graph is constructed based on the spatial coordinates of the cell nucleus and a nearest neighbor number K is set. Initial features are added to the nodes through type encoding, effectively capturing the spatial adjacency relationships and biological attributes between cell nuclei. This graph structure can reflect the distribution pattern of cells in tissues and the differences in their functional roles.

[0018] Optionally, a graph attention network is used to convolve the K-nearest neighbor graph to output a graph embedding vector representing the spatial relationships of the pathological microenvironment, including:

[0019] A network including at least one graph attention layer is used to aggregate and update the node features in the K-nearest neighbor graph, and pooling is performed on the updated node features to generate graph embedding vectors.

[0020] Optionally, the persistent cohomology of simple complexes composed of specific cell types in the K-nearest neighbor graph is calculated to generate a topological feature vector that quantifies the global topological structure, including:

[0021] Select one or more preset types of cell nucleus nodes from the K-nearest neighbor graph, construct a simple complex based on the spatial distance between these nodes, calculate the continuous homology of the simple complex in at least one dimension, and extract the statistics from it as topological feature vectors.

[0022] Optionally, the bidirectional long short-term memory network is Bi-LSTM.

[0023] Optionally, the HoVer-Net model can be used to obtain the location and type of cell nuclei in the corrected image.

[0024] This application has the following technical effects:

[0025] Multi-scale Tucker decomposition accurately separates and removes imaging artifacts while preserving key tissue structural details to the greatest extent, significantly improving the effective information content of the image. This application not only constructs a graph to represent microscopic cellular spatial relationships but also introduces continuous cohomology to quantify macroscopic tissue topology. Combined with graph attention networks for deep feature extraction, it achieves a multi-dimensional and in-depth characterization of the pathological microenvironment from local to global perspectives. By efficiently fusing comprehensive image pathological features with deep semantic information extracted from clinical text, a unified multimodal index code is generated, thereby realizing the intrinsic correlation between image and text information. This greatly improves the efficiency and accuracy of cross-modal retrieval and ultimately lays a solid foundation for subsequent precise analysis and management through structured data entities. Attached Figure Description

[0026] Figure 1 This is a flowchart of a data processing method for a specimen digitization management platform according to an embodiment of this application. Detailed Implementation

[0027] This application discloses a data processing method for a specimen digitization management platform, referring to... Figure 1 A data processing method for a specimen digitization management platform, comprising:

[0028] S1: Obtain the raw digital images of the specimen and the associated unstructured clinical text.

[0029] Through the interface of the specimen digitization management platform, the original digital images, such as SVS or TIF format whole slide images at 20x magnification, are obtained from the digital slide scanner to obtain the original data images. At the same time, the pathology report text corresponding to the specimen is retrieved from the hospital information system or electronic medical record system to obtain the clinical text.

[0030] S2: Divide the original digital image into multiple overlapping image blocks and construct a third-order data tensor. Perform multi-scale Tucker decomposition on the third-order data tensor to obtain low-rank components that represent the tissue structure. Use the low-rank components as the corrected image.

[0031] A sliding window method is used to segment the original digital image, resulting in multiple overlapping image blocks. This ensures that any two adjacent overlapping image blocks have a predetermined overlap ratio in the horizontal or vertical directions. The RGB channels of each overlapping image block form a three-dimensional array, and all overlapping image blocks are stacked to form a third-order space-space-channel data tensor. For example, for an original digital image with a resolution of 1000 x 1000 pixels, a 256 x 256 pixel sliding window can be set. If the preset overlap ratio is 25%, the window moves in steps of 192 pixels horizontally and vertically. In this way, adjacent overlapping image blocks will share an overlap band of 64 pixels wide or high, which helps maintain the continuity of microstructures such as cell nuclei at the block edges.

[0032] An accelerated proximal gradient method is employed to solve the multi-scale Tucker decomposition model, decomposing the third-order data tensor into a core tensor representing the core organizational pattern, a factor matrix, and an error tensor representing sparse artifacts. Finally, the core tensor is multiplied by the factor matrix to reconstruct the low-rank component, which is the artifact-free corrected image.

[0033] During the multi-scale Tucker decomposition of a third-order data tensor:

[0034] The third-order data tensor is decomposed at at least two different resolution scales, and an iterative optimization algorithm is used to solve for the core tensor and factor matrix until a preset convergence condition is met. Specifically, assuming the data tensor consists of overlapping image patch features, it can first be decomposed at the original 256x256 pixel scale, and then downsampled to a 128x128 pixel scale for further decomposition. At each scale, the core tensor and factor matrix are updated using iterative algorithms such as alternating least squares. The algorithm terminates when the change in reconstruction error calculated in two consecutive iterations is less than one ten-thousandth or the number of iterations reaches the upper limit of 100. This multi-scale strategy can simultaneously capture both macroscopic and microscopic features of the image.

[0035] S3: Obtain the location and type of cell nuclei in the corrected image, and construct a K-nearest neighbor graph based on the Euclidean distance between cell nuclei.

[0036] Specifically, the HoVer-Net model is used to process the corrected image. This model can simultaneously detect and classify cell nuclei, outputting the location (center coordinates) and type of each cell nucleus, such as tumor cell, lymphocyte, or stromal cell. Then, the Euclidean distance between all identified cell nuclei is calculated. For each cell nucleus, the K nearest neighbors are selected. For example, setting the nearest neighbor number K to 8, for a tumor cell nucleus node located at coordinates (150, 200), its Euclidean distance to all other cell nucleus nodes is calculated, and the 8 nearest neighbors are established. Simultaneously, the tumor cell type of each node is converted into a three-dimensional feature vector using one-hot encoding, such as vector 1, 0, 0, where the first dimension represents tumor cells, and the last two dimensions represent immune cells and stromal cells, respectively, thus completing the construction of the K-nearest neighbor graph.

[0037] S4: Convolutional processing of the K-nearest neighbor graph is performed using a graph attention network to output a graph embedding vector representing the spatial relationships of the pathological microenvironment; at the same time, the continuous cohomology of simple complexes composed of specific cell types in the K-nearest neighbor graph is calculated to generate a topological feature vector that quantifies the global topological structure.

[0038] Specifically, the K-nearest neighbor graph with initial node features is input into a graph attention network containing three graph attention layers. By aggregating neighbor node information and assigning different attention weights, the graph-level representation output by the last layer of the network is the graph embedding vector. Simultaneously, all tumor cell nodes in the graph are selected, and their Vietoris-Rips complexes are constructed. Then, the persistent cohomology of this complex is calculated using the GUDHI library to obtain a persistent graph describing the lifecycle of zero-dimensional connected components and one-dimensional ring structures. This persistent graph is then transformed into a fixed-length topological feature vector.

[0039] The network with at least one graph attention layer aggregates and updates the node features in the K-nearest neighbor graph, and then performs pooling operations on the updated node features to generate graph embedding vectors.

[0040] For example, in this embodiment, a network with two graph attention layers is used to process the K-nearest neighbor graph. In the first layer, the initial 3D feature vector of each node is weighted and aggregated according to the attention weights between it and its neighboring nodes to generate a 64-dimensional intermediate feature. The second layer further updates these 64-dimensional features by performing global average pooling on the final feature vectors of all nodes in the graph to obtain a 128-dimensional graph embedding vector representing the entire graph structure.

[0041] Select one or more preset types of cell nucleus nodes from the K-nearest neighbor graph, construct a simple complex based on the spatial distance between these nodes, calculate the continuous homology of the simple complex in at least one dimension, and extract the statistics from it as topological feature vectors.

[0042] In this embodiment, all tumor cell nuclear nodes are screened out, and a Vietoris-Rips complex is constructed based on their spatial coordinates. As the distance threshold increases from 0 pixels to 50 pixels, the generation and disappearance processes of zero-dimensional and one-dimensional homology groups, i.e., connected components and loop structures, are observed in the complex. Then, the number of loop structures with a duration of more than 10 pixels and the average length of the lifespan of all loop structures are counted using the GUDHI library. These two values, for example, a number of 5 and an average length of 25.3, are combined into a two-dimensional topological feature vector.

[0043] S5: Extract semantic vectors from unstructured clinical texts using a pre-built bidirectional long short-term memory network.

[0044] Specifically, the clinical text undergoes preprocessing such as word segmentation and stop word removal. Then, the words are converted into pre-trained word vectors, for example, word vectors generated using the Word2Vec model. The word vector sequence is then input into a bidirectional long short-term memory network (Bi-LSTM), which captures contextual information from both forward and backward directions. Finally, the hidden state vectors from both directions are concatenated or averaged to obtain a semantic vector representing the core semantics of the entire text.

[0045] S6: Input the graph embedding vector, topological feature vector, and semantic vector into the multimodal fusion network, and calculate a unified multimodal index code through a hash function.

[0046] Specifically, the graph embedding vector, topological feature vector, and semantic vector are concatenated to form a long vector. This long vector is then input into a multimodal fusion network consisting of multiple fully connected layers and activation functions. The network learns to map features from different modalities to a common latent space. The feature vector output by the fusion network is then binarized into a set of binary codes, such as 64 bits, through a learning-based hash function, such as a deep hash network; this is the multimodal index code.

[0047] Specifically, the previously generated 128-dimensional graph embedding vector, 2-dimensional topological feature vector, and a 512-dimensional semantic vector extracted from the high-level model are concatenated to form a 642-dimensional fused feature vector. This fused vector is input into a fully connected layer, which linearly transforms it into a 64-dimensional vector. Finally, a sign function is applied to each element of this vector, mapping positive numbers to 1 and negative numbers or zero to 0, thereby generating a 64-bit multimodal index code.

[0048] S7: Encapsulate the identifier, graph embedding vector, semantic vector, multimodal index code, and specimen metadata of the corrected image into a structured data entity.

[0049] Specifically, a JSON object or database record is created as a structured data entity [1][2]. The entity contains multiple key-value pairs, with keys being image unique ID, graph embedding vector, semantic vector, multimodal index code, patient ID, specimen number, sampling site, etc., and corresponding values ​​being the specific data obtained from the aforementioned steps and the metadata extracted from the original specimen information.

[0050] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A data processing method for a specimen digitization management platform, characterized in that, include: Obtain raw digital images of specimens and associated unstructured clinical text; The original digital image is divided into multiple overlapping image patches and a third-order data tensor is constructed. Multi-scale Tucker decomposition is performed on the third-order data tensor to obtain low-rank components representing tissue structure, and the low-rank components are used as the correction image. The cell nucleus positions and types in the correction image are obtained, and a K-nearest neighbor graph is constructed based on the Euclidean distance between cell nuclei. The K-nearest neighbor graph is convolved using a graph attention network to output a graph embedding vector representing the spatial relationship of the pathological microenvironment. At the same time, the persistent cohomology of simple complexes composed of specific cell types in the K-nearest neighbor graph is calculated to generate a topological feature vector that quantifies the global topological structure, including: selecting one or more preset types of cell nucleus nodes from the K-nearest neighbor graph and constructing a simple complex based on the spatial distance between these nodes. Calculate the persistent homology of the simple complex in at least one dimension and extract the statistics from it as topological feature vectors. The semantic vectors of unstructured clinical texts are extracted using a pre-built bidirectional long short-term memory network; the graph embedding vectors, topological feature vectors, and semantic vectors are input into a multimodal fusion network, and a unified multimodal index code is calculated using a hash function; the identifier of the corrected image, the graph embedding vector, the semantic vector, the multimodal index code, and the specimen metadata are encapsulated into a structured data entity.

2. A data processing method for a specimen digitization management platform according to claim 1, characterized in that, The original digital image is divided into multiple overlapping image patches, and a third-order data tensor is constructed, including: The original digital image is segmented using a sliding window method, so that there is a preset proportion of overlap between any two adjacent overlapping image blocks in the horizontal or vertical direction; the RGB three channels of each overlapping image block form a three-dimensional array, and all overlapping image blocks are stacked to form a third-order data tensor.

3. A data processing method for a specimen digitization management platform according to claim 2, characterized in that, The preset ratio is 25%.

4. A data processing method for a specimen digitization management platform according to claim 1, characterized in that, In the process of performing multi-scale Tucker decomposition on a third-order data tensor to obtain low-rank components representing organizational structure: the third-order data tensor is decomposed at at least two different resolution scales, and an iterative optimization algorithm is used to solve for the core tensor and factor matrix until the preset convergence condition is met.

5. A data processing method for a specimen digitization management platform according to claim 4, characterized in that, The low-rank component is reconstructed by multiplying the core tensor with the factor matrix, and then used as the corrected image.

6. A data processing method for a specimen digitization management platform according to claim 1, characterized in that, Set the nearest neighbor number K. For each cell nucleus node, determine the K nearest neighbor nodes based on its spatial coordinates and establish connections; encode the cell nucleus type as the initial node feature vector.

7. A data processing method for a specimen digitization management platform according to claim 1, characterized in that, A graph attention network is used to convolve the K-nearest neighbor graph, outputting graph embedding vectors that represent the spatial relationships of the pathological microenvironment, including: The network with at least one graph attention layer is used to aggregate and update the node features in the K-nearest neighbor graph, and the updated node features are pooled to generate graph embedding vectors.

8. A data processing method for a specimen digitization management platform according to claim 1, characterized in that, The bidirectional long short-term memory network is Bi-LSTM.

9. A data processing method for a specimen digitization management platform according to claim 1, characterized in that, The HoVer-Net model was used to obtain the location and type of cell nuclei in the corrected image.

Images