Tumor cell accurate identification and analysis system based on digital pathological image

Abstract

The invention relates to the technical field of medical image processing, and discloses a tumor cell accurate recognition and analysis system based on a digital pathological image, which effectively overcomes the problem of global context deficiency caused by traditional pathological image blocking processing by constructing a microcosmic and macroscopic parallel multi-scale feature extraction mechanism. A cell topological graph is constructed by utilizing spatial semantic double constraints to simulate a biological spatial distribution rule of tumor cells, and precise navigation and weighted enhancement of microscopic cell characteristics by macroscopic organization structure information are realized through a cross-scale attention aggregation technology. Therefore, the model can fully refer to the surrounding microenvironment when identifying the heterotypic cells, and the misjudgment risk caused by background noise or local form similarity is remarkably reduced; in addition, a structured decision-making mechanism based on manifold consistency eliminates isolated prediction noisy points and ensures the continuity and rationality of a diagnosis result on a biological structure.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a tumor cell precise identification and analysis system based on a digital pathology image. BACKGROUND

[0002] With the popularization of digital pathology scanning technology, pathological diagnosis is undergoing a transformation from traditional microscope observation to digital intelligent diagnosis. In the gold standard process of clinical pathological diagnosis, the determination of tumor by pathologists relies on a multi-scale comprehensive cognitive mode, which not only needs to observe the microscopic morphological details such as nuclear atypia, mitotic figures and chromatin distribution under high magnification, but also needs to examine the arrangement of cells, the integrity of glandular structure and the infiltration of tumor to the surrounding basement membrane or interstitium under low magnification. Digital pathology whole slide images have become the core carrier of this process because they can completely retain all the information of the tissue section. Such images usually have extremely high resolution, and the number of pixels of a single image often reaches the level of one billion, containing a large amount of biological characteristics and pathological semantic information, which can reflect the complex interaction between tumor cells and their microenvironment, and provide a rich data basis for precision medicine.

[0003] However, in the existing computer-aided diagnosis technology system based on deep learning, due to the physical bottleneck of the graphics processing unit memory capacity and computing resources, it is not possible to directly extract and analyze the features of the whole one billion pixel level whole slide image as a single input in an end-to-end manner. The current mainstream processing scheme usually adopts a block processing strategy, that is, the huge whole slide image is first cropped into thousands of small size local image blocks, and then the features are extracted and integrated. The existing multiple instance learning framework is the main technical means to solve this problem, which usually assumes that each image block is independent of each other, and uses simple statistical aggregation strategies such as maximum pooling or average pooling to integrate local features to obtain the diagnosis result of the whole slide. Although this processing method solves the problem of limited computing resources, it essentially breaks the inherent spatial continuity and topological structure relationship in the pathology image, resulting in a serious disconnection between local features and global pathological context during the processing of the algorithm. Specifically, the existing aggregation algorithm cannot establish a long-distance feature dependency relationship, so that the model cannot judge the properties of local cells in combination with the surrounding tissue environment like pathologists, for example, without the assistance of macroscopic structure information, highly active inflammatory reaction areas or benign hyperplasia tissues are easily misjudged as malignant tumors due to the similarity of local cell morphology, or the judgment of tumor stage is wrong due to the inability to effectively identify the infiltration mode of tumor cells to the periphery. This neglect of spatial topological semantic information limits the accuracy and reliability of the existing system in complex and difficult pathological diagnosis, and cannot meet the urgent needs of clinical tumor cell precise identification and analysis. SUMMARY

[0004] This application proposes a precise tumor cell identification and analysis system based on digital pathological images to address the problems mentioned in the background art.

[0005] To achieve the above objectives, this application adopts the following technical solution: a tumor cell precision identification and analysis system based on digital pathological images, comprising: a multi-scale feature extraction module, a topology map construction module, a cross-scale feature fusion module, and an identification decision module, wherein;

[0006] The multi-scale feature extraction module is used to respond to the input of digital pathology whole slice images. It extracts the set of local morphological feature vectors under high resolution view and the global semantic feature map under low resolution view through parallel processing channels, and establishes a data foundation that combines micro-details and macro-structure.

[0007] The topology graph construction module receives the set of local morphological feature vectors output by the multi-scale feature extraction module, establishes connection edges based on the spatial proximity and semantic similarity between each vector in the set of local morphological feature vectors, and constructs a cell topology graph representing the cell distribution structure, wherein the set of local morphological feature vectors is instantiated as graph nodes of the cell topology graph.

[0008] The cross-scale feature fusion module receives the cell topology map output by the topology graph construction module and the global semantic feature map output by the multi-scale feature extraction module. It uses an attention mechanism to calculate the correlation weight between the graph nodes and the global semantic feature map, and weights the macro-contextual information in the global semantic feature map to the graph nodes to generate a fusion node feature vector that integrates macro- and micro-information.

[0009] The identification decision module receives the fusion node feature vectors output by the cross-scale feature fusion module, parses the fusion node feature vectors through a classifier, and outputs tumor cell identification results with spatial structure constraints.

[0010] Furthermore, the specific operation of the multi-scale feature extraction module in extracting the set of local morphological feature vectors is as follows:

[0011] First, a sliding window cropping operation is performed on the high-resolution view to obtain a series of local image patches, and a global spatial index matrix is ​​built to record the physical center position coordinates of each local image patch.

[0012] Subsequently, dual-path parallel processing is performed for each local image patch. On the one hand, a deep residual network is used to extract basic texture features, and on the other hand, the determinant of the second-order Hessian matrix of the local image patch is calculated to capture the speckled structure of the cell nucleus.

[0013] Finally, the response value of the determinant graph of the second-order Hessian matrix is ​​mapped to a nonlinear shape attention weight using the hyperbolic tangent function. The shape attention weight is then multiplied element-wise with the basic texture features to generate a set of local morphological feature vectors that incorporate second-order geometric curvature information.

[0014] Furthermore, the specific operations of the multi-scale feature extraction module in extracting the global semantic feature map are as follows:

[0015] A gated context convolution mechanism is used to process low-resolution views. This gated context convolution mechanism is configured with a content flow path for extracting basic semantics of organizational structure and a gated flow path for generating background suppression masks.

[0016] The multi-scale feature extraction module uses a non-linear activation function with an output value range of zero to one to process the output data of the gated flow path to generate a background suppression mask. The background suppression mask is then multiplied element-wise with the output data of the content flow path to filter background noise in non-organic regions and generate a global semantic feature map that encodes macro-historical semantics.

[0017] Furthermore, the specific operations for constructing a cell topology graph using the topology graph construction module are as follows:

[0018] The topology graph construction module first instantiates each local morphological feature vector in the set of local morphological feature vectors into a graph node, and then calls the physical location coordinates corresponding to the graph node from the data output by the multi-scale feature extraction module.

[0019] Subsequently, the topology graph construction module constructs a multi-dimensional spatial index tree to manage the spatial distribution of all graph nodes. For any target graph node, it uses the physical location coordinates of the target graph node as the center and the preset micro-environment perception radius as the search limit to filter out a set of candidate neighbor nodes that are spatially close in the multi-dimensional spatial index tree.

[0020] Furthermore, the specific operations for establishing connecting edges in the topology graph construction module are as follows:

[0021] The morphological space coupling affinity operator is used to calculate the coupling affinity weight between a node in the target graph and any candidate node in the set of candidate neighbor nodes;

[0022] The topology construction module first calculates the squared Euclidean distance between the physical location coordinates of the two, and then calculates the spatial attenuation term that decays exponentially in combination with the preset spatial bandwidth parameter.

[0023] Meanwhile, the topology graph construction module calculates the cosine similarity between the local morphological feature vectors of the two, uses the learnable semantic gain coefficient and similarity truncation bias to perform a linear transformation on the cosine similarity, and inputs the transformed result into a smoothing rectifier function to obtain the semantic gating term.

[0024] Finally, the topology graph construction module determines the product of the spatial decay term and the semantic gating term as the coupling affinity weight, and retains only the connecting edges whose weights meet the preset conditions to generate a sparse adjacency matrix.

[0025] Furthermore, the specific operation of the cross-scale feature fusion module in calculating the relevance weights using the attention mechanism is as follows:

[0026] First, projection and alignment across the modal feature space are performed. The cross-scale feature fusion module calls the local morphological feature vector carried by each graph node in the cell topology graph as the data source for the query vector.

[0027] Simultaneously, spatial flattening is performed on the global semantic feature map to form a feature sequence, and the feature vector at each pixel in the feature sequence is used as the data source for the key vector and value vector;

[0028] Subsequently, the cross-scale feature fusion module uses a learnable linear projection matrix to map the query vector, key vector, and value vector to a unified semantic vector space.

[0029] Furthermore, the specific operation of the cross-scale feature fusion module in performing spatially constrained cross-scale attention aggregation is as follows:

[0030] The spatially biased cross-attention mechanism is used to calculate the relevance weights;

[0031] The cross-scale feature fusion module first calculates the dot product between the query vector of a specific graph node and the key vector of a specific global pixel to measure semantic consistency.

[0032] At the same time, the cross-scale feature fusion module calls the physical location coordinates of specific graph nodes and determines the mapping coordinates of specific global pixels in the digital pathology whole slice image coordinate system, and calculates the square of the Euclidean distance between them.

[0033] The product of the square of the Euclidean distance and the learnable geometric constraint coefficients is then processed using the natural logarithm function to generate a spatial penalty.

[0034] Finally, the cross-scale feature fusion module subtracts the spatial penalty from the semantic consistency calculation result and inputs the subtraction result into the normalized exponential function to generate the relevance weight.

[0035] Furthermore, the specific operation of the cross-scale feature fusion module in generating the fused node feature vector is as follows:

[0036] The cross-scale feature fusion module performs weighted aggregation of macro-contextual information by using relevance weights to perform weighted summation on the value vectors corresponding to all global pixels in the feature sequence, thereby aggregating macro-contextual information for a specific graph node.

[0037] Subsequently, the cross-scale feature fusion module uses the gated residual fusion unit to calculate the balance coefficient, and then uses the balance coefficient to weight and fuse the macro-contextual information with the original local morphological feature vector to generate a fused node feature vector that combines micro-texture details and macro-organizational semantics.

[0038] Furthermore, the specific operation of the identification decision module in predicting the probability distribution based on the fused node feature vectors is as follows:

[0039] The identification decision module receives the fusion node feature vector output by the cross-scale feature fusion module, and uses a classification projection head composed of multiple fully connected layers to perform feature space mapping on the fusion node feature vector, thereby outputting an initial prediction probability vector for each graph node. This initial prediction probability vector represents the confidence of the cell to which the corresponding graph node belongs for each preset pathological category.

[0040] Simultaneously, the decision-making module calculates the information entropy functional of the initial prediction probability vector and uses this information entropy functional as a numerical indicator to quantify the uncertainty of the prediction distribution.

[0041] Furthermore, the specific operation of the identification decision module in constructing the manifold consistency energy function is as follows:

[0042] First, the initial prediction probability vectors of all graph nodes are stacked to construct the full graph prediction matrix, and then the sparse adjacency matrix generated by the topology graph construction module is called.

[0043] Then, a degree matrix is ​​constructed in which the diagonal elements are equal to the sum of the elements of the corresponding rows of the sparse adjacency matrix. The degree matrix is ​​then used to normalize the sparse adjacency matrix to generate a symmetric normalized graph Laplace matrix.

[0044] The identification decision module calculates the matrix product of the transpose of the full-map prediction matrix, the symmetric normalized graph Laplacian matrix, and the full-map prediction matrix, and performs a matrix trace operation on the matrix product to obtain the Laplacian smoothing term.

[0045] Finally, the information entropy functional is weighted and accumulated using a learnable uncertainty penalty coefficient to obtain the maximum entropy regularization term. The Laplace smoothing term is then added to the maximum entropy regularization term to obtain the structural risk penalty term used to constrain the tumor cell identification results.

[0046] The beneficial effects of this invention are as follows:

[0047] This invention effectively overcomes the problem of missing global context caused by traditional block processing of pathological images by constructing a multi-scale feature extraction mechanism that combines microscopic and macroscopic features. It uses spatial semantic dual constraints to construct a cell topology map to simulate the biological spatial distribution of tumor cells, and achieves precise navigation and weighted enhancement of microscopic cell features by cross-scale attention aggregation technology. This allows the model to fully refer to the surrounding microenvironment when identifying atypical cells and significantly reduces the risk of misjudgment caused by background noise or local morphological similarity. In addition, the structured decision-making mechanism based on manifold consistency eliminates isolated prediction noise and ensures the continuity and rationality of diagnostic results in biological structure. This greatly improves the accuracy and robustness of tumor cell identification and provides panoramic intelligent auxiliary support for clinical pathological diagnosis from pixel-level detection to tissue-level analysis. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort:

[0049] Figure 1 This is a schematic diagram of the system framework of the present invention;

[0050] Figure 2 This is a logic diagram of the multi-scale feature extraction module of the present invention;

[0051] Figure 3 This is the logic diagram of the topology graph construction module of the present invention;

[0052] Figure 4 This is a logic diagram of the cross-scale feature fusion module of the present invention;

[0053] Figure 5 This is a logic diagram of the identification decision module of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Example 1, as Figure 1As shown, the tumor cell precision identification and analysis system based on digital pathological images includes: a multi-scale feature extraction module, a topology graph construction module, a cross-scale feature fusion module, and an identification decision module, wherein;

[0056] The multi-scale feature extraction module is used to respond to the input of digital pathology whole slice images. It extracts the set of local morphological feature vectors under high resolution view and the global semantic feature map under low resolution view through parallel processing channels, and establishes a data foundation that combines micro-details and macro-structure.

[0057] The topology graph construction module receives the set of local morphological feature vectors output by the multi-scale feature extraction module, establishes connection edges based on the spatial proximity and semantic similarity between each vector in the set of local morphological feature vectors, and constructs a cell topology graph representing the cell distribution structure, wherein the set of local morphological feature vectors is instantiated as graph nodes of the cell topology graph.

[0058] The cross-scale feature fusion module receives the cell topology map output by the topology graph construction module and the global semantic feature map output by the multi-scale feature extraction module. It uses an attention mechanism to calculate the correlation weight between the graph nodes and the global semantic feature map, and weights the macro-contextual information in the global semantic feature map to the graph nodes to generate a fusion node feature vector that integrates macro- and micro-information.

[0059] The identification decision module receives the fusion node feature vectors output by the cross-scale feature fusion module, parses the fusion node feature vectors through a classifier, and outputs tumor cell identification results with spatial structure constraints.

[0060] Example 2, as Figure 2 As shown, in one embodiment of the present invention, the workflow of the multi-scale feature extraction module is described in detail. The multi-scale feature extraction module extracts a set of local morphological feature vectors. The specific operation is as follows:

[0061] First, in the digital pathology whole slide image Perform a sliding window cropping operation on a high-resolution view to obtain a series of local image patches. During this process, the system sets the high-resolution view to typically correspond to 40x optical magnification, and the sliding window moves in preset steps. Move upwards to extract each local image patch. All of them contain high-precision cell texture information. At the same time, a system is established to record each local image patch. Global spatial index matrix of physical center location coordinates The global space index matrix As a lookup table, it stores the first... Local image patch exist Absolute coordinates in a coordinate system This ensures that microscopic data can be accurately located in subsequent steps.

[0062] Then, for each local image patch Perform dual-path parallel processing, utilizing a deep residual network on one hand. Extracting basic texture features, this path mainly captures the color, texture, and edge gradient information of the cytoplasm; on the other hand, it calculates local image patches. Second-order Hessian matrix Determinant diagrams are used to capture the speckled structure of the cell nucleus; in this step, a second-order Hessian matrix is ​​used. It can describe the change of the second derivative of the image pixel intensity, and its determinant response value The image exhibits extreme values ​​in the blobular region (i.e., the cell nucleus region), thereby identifying potential regions of interest at the geometric level.

[0063] Finally, the hyperbolic tangent function is used. The second-order Hessian matrix The response values ​​of the determinant graph are mapped to nonlinear shape attention weights, and these shape attention weights are then multiplied element-wise with the underlying texture features to generate a set of local morphological feature vectors that incorporate second-order geometric curvature information. The specific mathematical calculation model for this step is shown in the following formula:

[0064] ;

[0065] In this formula:

[0066] Represents the input number A local image patch; This represents the basic texture feature vector extracted by the pre-trained deep convolutional neural network; This represents the element-wise product operator (Hadamard Product), used to apply geometric attention weights to texture features.

[0067] The parentheses on the right side of the formula constitute a morphology-based attention gating mechanism. Among them, The determinant of the second-order Hessian matrix of a local image patch is... The numerical response can significantly characterize the speckled structures (i.e., the tumor cell nucleus region) in the image. This means flattening the two-dimensional determinant response graph into a vector.

[0068] It is a learnable shape projection matrix that linearly maps high-dimensional geometric response vectors to the same dimensional space as the underlying texture feature vectors, achieving dimensional alignment. During model initialization, the parameters of this matrix follow a normal distribution, and it is optimized during training using the backpropagation algorithm.

[0069] It is a preset scale normalization factor (hyperparameter) used to adjust the sensitivity of the attention mechanism. In this embodiment, Let be a positive real number, preferably ranging from 1.0 to 5.0. The purpose of introducing this factor is to scale the projected eigenvalues, preventing excessively large values ​​from affecting the hyperbolic tangent function. Entering the saturation region can lead to the gradient vanishing problem.

[0070] pass The composite operation maps geometric features to dynamic weights ranging from (0,2). When a significant nuclear structure is detected, the weight value is greater than 1, thereby enhancing the corresponding texture feature response; otherwise, the feature is preserved or suppressed. This allows the neural network to focus on key regions of tumor cells using explicit prior knowledge of geometric morphology.

[0071] The specific problem addressed by this formula in this implementation is that, through the nonlinear mapping of second-order derivative information, it endows traditional deep learning features with geometric perception capabilities of "spot-like" cell nuclear structures, thereby enabling the final generated set of local morphological feature vectors. Higher activation values ​​were observed in regions with clear nuclear outlines and significant atypia, thus significantly enhancing the ability to characterize tumor cell morphology.

[0072] Furthermore, the multi-scale feature extraction module extracts the global semantic feature map. The specific operation is as follows:

[0073] Using gated contextual convolution to process low-resolution views The low-resolution view Digital pathology whole slide images An overview image obtained after downsampling (e.g., at 5x or 10x magnification). This gated contextual convolution mechanism is configured with a content flow path for extracting the underlying semantics of the organizational structure, and a background suppression mask for generating the mask. The gated flow path focuses on extracting diagnostically significant pathological structures such as glands and stroma, while the content flow path focuses on identifying blank backgrounds, bubbles, or scanning artifacts.

[0074] The multi-scale feature extraction module utilizes a non-linear activation function with an output value range of zero to one. Process the output data of the gated flow path to generate a background suppression mask. and use the background suppression mask Element-wise multiplication is performed with the output data of the content flow path to filter out background noise in non-organic regions, generating a global semantic feature map that encodes macroscopic histological semantics. The specific mathematical calculation model for this step is shown in the following formula:

[0075] ;

[0076] In this formula:

[0077] The global semantic feature map representing the output; A low-resolution macro view representing the input; Represents a two-dimensional convolution operation; This represents the element-wise product operator (Hadamard Product).

[0078] The right side of the formula is divided into two parts, corresponding to the two paths of the gated context convolution mechanism:

[0079] 1. Part One This is the content flow path. Among them, The content-flow convolution kernel weight matrix is ​​a learnable parameter. In the model initialization stage, it is preferred to use the He normal distribution for initialization to extract the basic texture and structural features in the image. As a non-linear activation function, a linear rectified unit (ReLU) is selected in this embodiment to increase the non-linear expressive power of feature extraction.

[0080] 2. Part Two This is the gated flow path. The gated flow convolution kernel weight matrix is ​​also a learnable parameter used to identify foreground tissues and background regions in an image. It is the Sigmoid activation function, whose output value range is strictly limited to (0,1).

[0081] The calculation using this formula generates a continuous soft mask for the gated flow path. When When the local area in the image corresponds to blank areas or artifact noise on the slide, The output approaches 0, suppressing the feature response at the corresponding position in the content flow path through element-wise multiplication; when the local region corresponds to effective biological tissue... The output approaches 1, thus preserving and activating the corresponding semantic features. This mechanism ensures that the generated... It can automatically focus on pathological structures with diagnostic significance, providing high signal-to-noise ratio macroscopic contextual information for subsequent feature fusion.

[0082] Example 3, as Figure 3 As shown, the topology graph construction module constructs a cell topology graph. The specific operation is as follows:

[0083] The topology graph construction module first sets the local morphological feature vectors. Each local morphological feature vector in Instantiate as a graph node It also retrieves the physical location coordinates corresponding to each graph node from the data output by the multi-scale feature extraction module. In this step, the graph structure initialized by the system contains a set of vertices. Each top graph node It not only carries microscopic morphological feature data, but also binds its geometric coordinates in the whole slice. This has achieved a preliminary binding of biometrics and spatial information.

[0084] Subsequently, the topology graph construction module builds a multidimensional spatial index tree. To manage all graph nodes The spatial distribution, in this embodiment, is a multidimensional spatial index tree. Preferred This reduces the time complexity of subsequent neighborhood searches. And for any target graph node... With the target graph node physical location coordinates Centered on a circle with a preset micro-environment sensing radius To define search boundaries, in a multidimensional index tree Select a set of candidate neighbor nodes that are spatially close to each other. The microenvironment sensing radius The maximum physical limit of intercellular interactions is defined (e.g., set to 50 micrometers). This step serves to perform rapid spatial screening, avoiding pairwise calculations for all nodes in the entire image, thereby significantly improving the efficiency of large-scale pathological image processing.

[0085] Furthermore, the specific operations for establishing connecting edges in the topology graph construction module are as follows:

[0086] Target graph nodes are calculated using the morphological space coupling affinity operator. With the set of candidate neighbor nodes Any candidate node Coupling affinity weights between This operator aims to address the problem that traditional distance metrics cannot distinguish between physically adjacent but biologically different cells (e.g., tumor cells are adjacent to vascular endothelial cells).

[0087] The topology graph construction module first calculates the physical location coordinates of the two. The square of the Euclidean distance between them And combined with preset spatial bandwidth parameters Calculate the spatial decay term that exhibits an exponential decay.

[0088] Simultaneously, the topology graph construction module calculates the local morphological feature vectors of both. , The cosine similarity between the two pairs of data is used to leverage learnable semantic gain coefficients. Similarity truncation bias Perform a linear transformation on the cosine similarity and input the transformed result into a smoothing rectifier function. To obtain semantic gating terms;

[0089] Finally, the topology graph construction module determines the coupling affinity weight by multiplying the spatial decay term and the semantic gating term. Only edges whose weights meet preset conditions are retained to generate a sparse adjacency matrix. .

[0090] The above calculation of coupling affinity weights The mathematical model is shown in the following formula:

[0091] ;

[0092] In this formula:

[0093] The nodes that are calculated represent With nodes The coupling affinity weight between them; Represents the physical location coordinates of two graph nodes in the full slice coordinate system; This represents the Euclidean norm, used to calculate physical distances.

[0094] The exponent term on the left side of the formula This is the spatial attenuation term. Wherein, This is a preset spatial bandwidth parameter (hyperparameter) used to control the rate at which the interaction strength between nodes decays with increasing physical distance. In this embodiment, it is based on the average diameter of common tumor cells at 40x optical magnification (approximately 20 to 40 pixels). The preferred value range is set to 20 to 100 (pixels) to ensure that connections only occur within the effective micro-environment neighborhood.

[0095] The right side of the formula It is a semantic gating term, which contains parameters for the feature vector. , Calculate the cosine similarity. The semantic gain coefficient is a learnable parameter in the network. Its initial value is preferably between 1.0 and 5.0. It is used to scale the cosine similarity to enhance the model's sensitivity to feature differences. The similarity truncation bias is also a learnable parameter, and its initial value is preferably between 0.0 and 0.5. The function of this bias is to subtract a baseline value from the similarity value, so that the weight of node pairs with low morphological similarity approaches zero after being mapped by the Softplus function, thereby achieving automatic filtering of noisy connections.

[0096] Using the Softplus function This aims to ensure the smoothness and non-negativity of semantic weights, avoid gradient truncation during training, and ensure the stability of backpropagation. Ultimately, the product of the spatial term and the semantic term forms... It implements the topology construction logic of "establishing strong connections only when cells are physically close and morphologically similar".

[0097] The specific problem addressed by this formula in this implementation is that it achieves dual constraints on connection edges through the product mechanism of "spatial decay" and "semantic gating." A strong connection is only established between two cells when they are physically close enough (large exponential term) and morphologically highly similar (large Softplus term). This effectively prevents nodes that are physically close but belong to different tissue types (such as epithelium and mesenchyme) from being incorrectly connected, thus ensuring the generation of a sparse adjacency matrix. It can accurately reflect the "homogeneous cluster" distribution structure of tumor cells, providing a pure topological path for subsequent graph neural network inference.

[0098] Example 4, as Figure 4 As shown, the specific operation of the cross-scale feature fusion module in calculating the relevance weights using the attention mechanism is as follows:

[0099] First, projection and alignment across the modal feature space are performed, and then the cross-scale feature fusion module calls the cell topology map. Each graph node Carrying local morphological feature vectors As the data source for the query vector;

[0100] Simultaneously, the global semantic feature map Perform spatial flattening to form a feature sequence and the feature sequence Each pixel The feature vector at a given location serves as the data source for the key vector (Key) and the value vector (Value).

[0101] Subsequently, the cross-scale feature fusion module utilizes a learnable linear projection matrix. The query vector, key vector, and value vector are mapped to a unified semantic vector space. In this step, the system eliminates the dimensionality differences between different modalities of data through linear transformation; the calculation formula is: Query Vector key vector value vector .

[0102] Furthermore, the specific operation of the cross-scale feature fusion module in performing spatially constrained cross-scale attention aggregation is as follows:

[0103] The spatially biased cross-attention mechanism is used to calculate the relevance weights;

[0104] The cross-scale feature fusion module first calculates the query vector for a specific graph node. Key vector of a specific global pixel The dot product between them is used to measure semantic consistency; the dot product value reflects the degree of matching between microscopic cell morphology and macroscopic tissue regions in pathological semantics (e.g., "atypical cells" and "infiltrated area" have a high degree of matching).

[0105] Simultaneously, the cross-scale feature fusion module calls the physical location coordinates of specific graph nodes. And determine the mapping coordinates of a specific global pixel in the digital pathology whole slice image coordinate system. Calculate the square of the Euclidean distance between the two. ;

[0106] Then, the natural logarithm function was used. Processing the square of the Euclidean distance with the learnable geometric constraint coefficients The product of these terms generates a space penalty.

[0107] Finally, the cross-scale feature fusion module subtracts the spatial penalty from the semantic consistency calculation result and inputs the result of the subtraction into the normalized exponential function. To generate relevance weights. The core mathematical model involved in this calculation process—the cross-scale context aggregation formula—is shown below:

[0108] ;

[0109] This formula is used to calculate the macro-contextual information vector for a specific graph node. By embedding geometric distance constraints in the semantic attention mechanism, it achieves precise interaction between local and global features, where:

[0110] The calculated value for the first The macro-contextual information vector of each graph node; Represents the index of all pixel positions after the global semantic feature map is flattened. Perform a traversal and summation; This is a normalized exponential function used to convert the attention score calculated within the parentheses into a probability weight distribution.

[0111] The first term inside the parentheses in the formula This is a semantic similarity term. Among them, Both are learnable linear projection matrices used to map features from different modalities to a unified semantic space. The feature dimension after projection is preferably 256 or 512 in this embodiment. The purpose of scaling the denominator is to prevent the dot product value from becoming too large due to excessive dimensionality, thereby avoiding the Softmax function from entering the saturation region where the gradient is minimal, and ensuring the stability of model training.

[0112] The second term within the parentheses of the formula This is a spatial penalty. Among them, Representing the The physical coordinates of the first graph node and the second graph node The squared Euclidean distance between the mapped coordinates of each global pixel. The geometric constraint coefficient is a learnable positive real-valued hyperparameter. In this embodiment, its initial value is preferably between 0.1 and 1.0. This coefficient is used to dynamically adjust the penalty strength of physical distance on the correlation weight: when When the size is large, the model tends to favor macroscopic features with very close physical distances to the polymer; when... When the size is small, the model allows for the aggregation of macro-contexts that are slightly further apart.

[0113] Using the natural logarithm function The distance transformation aims to smooth the penalty gradient generated by long distances and avoid excessive truncation of long-distance features.

[0114] In summary, this formula, by subtracting a spatial penalty from semantic similarity, achieves an attention allocation strategy that prioritizes semantic matching while also considering spatial proximity. This effectively addresses the background noise interference introduced by excessive physical distance in billion-pixel full-slice images. This allows each cell node to adaptively aggregate macroscopic features with specific semantic associations around it, effectively shielding against noise interference from physically distant (e.g., irrelevant regions) areas in the full-slice image, thus achieving precise local-global information interaction.

[0115] Finally, the cross-scale feature fusion module generates the feature vector of the fused node. The specific operation is as follows:

[0116] The cross-scale feature fusion module performs weighted aggregation of macro-contextual information and utilizes relevance weights to optimize the feature sequences. The value vector corresponding to all global pixels in A weighted summation operation is performed to aggregate the macro-contextual information for a specific graph node (i.e., the information calculated by the above formula). );

[0117] Subsequently, the cross-scale feature fusion module uses gated residual fusion units to calculate the balance coefficient, and then uses this balance coefficient to integrate macroscopic contextual information. Compared with the original local morphological feature vector Weighted fusion is performed to generate fused node feature vectors that combine microscopic texture details with macroscopic organizational semantics. This step ensures that the final feature vector used for classification retains both the morphological fingerprint of the cell itself and embeds the pathological microenvironment context in which it exists, providing panoramic information support for subsequent accurate identification.

[0118] Example 5, as Figure 5 As shown, the identification decision module is based on the fusion node feature vectors. The specific steps for performing probability distribution prediction are as follows:

[0119] The identification decision module receives the fused node feature vectors output by the cross-scale feature fusion module. The classification projection head, composed of multiple fully connected layers, is used to fuse the feature vectors of the nodes. In this embodiment, the classification projection head performs feature space mapping, mapping the high-dimensional feature vectors to a preset category space (such as tumor, stromal, lymphocytes, etc., category C), thereby outputting an initial prediction probability vector for each graph node. The initial prediction probability vector It represents the confidence level of the cell to which the corresponding graph node belongs for each preset pathological category, and the sum of all elements in the vector is 1;

[0120] Simultaneously, the decision-making module calculates the initial prediction probability vector. Information entropy functional And the information entropy functional As a numerical indicator for quantifying the uncertainty of predicted distribution, higher information entropy indicates that the model is more hesitant in classifying the cell (it may be at the tumor boundary), while lower entropy indicates that the prediction is very confident.

[0121] The specific operations for constructing the manifold consistency energy function in the identification and decision-making module are as follows:

[0122] First, initialize the prediction probability vectors of all graph nodes. Stacking is performed to construct a full-map prediction matrix. ,in It is The matrix contains the classification prediction status of all cells in the entire pathological slide; and calls the sparse adjacency matrix generated by the topology graph construction module. This matrix defines the strength of the topological connections between cells.

[0123] Then, construct a diagonal element matrix whose values ​​are equal to those of a sparse adjacency matrix. Degree matrix of the sum of the elements of the corresponding rows ,

[0124] The sparse adjacency matrix is ​​normalized using the degree matrix to generate a symmetric normalized graph Laplace matrix. And using the degree matrix For sparse adjacency matrix Normalization is performed to generate a symmetric normalized graph Laplacian matrix. The calculation logic for this step is as follows: The aim is to construct operators that describe the smoothness of graph signals.

[0125] The identification decision module calculates the transpose of the full-map prediction matrix. Matrix, symmetric normalized graph Laplace matrix and the full-map prediction matrix The matrix product of the three elements is then applied, and the trace operation is performed on the matrix product. This yields the Laplace smoothing term; the physical meaning of this term is to penalize those node pairs that are strongly connected on the topological graph but have vastly different prediction results, forcing the realization that "adjacent and similar cells should have the same prediction label".

[0126] Finally, the learnable uncertainty penalty coefficient is utilized. Information entropy functional Weighted summation is performed to obtain the maximum entropy regularization term, and the Laplace smoothing term is added to the maximum entropy regularization term to obtain the structural risk penalty term used to constrain the tumor cell identification results. The above-mentioned calculation structure risk penalty item The mathematical model—the manifold uniform energy function—is shown below:

[0127] ;

[0128] This formula is used to calculate the structural risk penalty term that constrains tumor cell identification results. By introducing spectral theory and information theory into the deep learning framework, it achieves structured prediction based on manifold consistency. Wherein:

[0129] This represents the calculated structural risk penalty item; Represents the total number of graph nodes (cells) within the currently input digital pathology image region; It is a normalization factor used to eliminate the magnitude difference in loss values ​​caused by fluctuations in cell number.

[0130] The first term of the formula This is a smoothing constraint term based on the Laplace matrix. Wherein, Represents the trace operation of a matrix; The full-map prediction matrix is ​​composed of the stacked initial prediction probability vectors of all nodes; the internal composite matrix... Here, the symmetric normalized graph Laplace matrix is... The sparse adjacency matrix output by the topology graph construction module. For the corresponding degree matrix, this term is mathematically equivalent to the sum of the weighted squared differences of the predicted distributions of adjacent nodes on the graph. By minimizing this term, the model is forced to output consistent classification results for cells that are topologically strongly connected (i.e., physically adjacent and morphologically similar), thereby effectively suppressing isolated misclassification noise (i.e., "salt and pepper noise").

[0131] The second term of the formula This is the maximum entropy regularization term. Wherein, Representing the The information entropy functional of the predicted probability distribution of each graph node is used to quantify the uncertainty of the model's classification of that node. The uncertainty penalty coefficient is a learnable positive real-valued hyperparameter. In this embodiment, its preferred initial value range is 0.01 to 0.1. This coefficient is introduced to balance structural smoothness with the uncertainty of individual predictions, preventing the model from making overconfident hard classification predictions when dealing with tumor-infiltrating boundary cells with ambiguous features. By retaining an appropriate probability margin, it improves the system's generalization ability and robustness in complex pathological scenarios.

[0132] The specific problem addressed by this formula in this implementation is that by unifying "classification accuracy" and "structural continuity" within a mathematical framework, the Laplace smoothing term is used to eliminate the "salt and pepper noise" (i.e. isolated misclassification points appearing in large areas of tumor) commonly found in pathological image recognition, and the maximum entropy regularization term is used to process high-risk boundary regions, ensuring that the final output tumor cell recognition results are not only statistically accurate, but also highly consistent and reasonable in terms of biological topology.

[0133] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A precise tumor cell identification and analysis system based on digital pathological images, characterized in that, include: The system includes a multi-scale feature extraction module, a topology graph construction module, a cross-scale feature fusion module, and a recognition decision module. The multi-scale feature extraction module is used to respond to the input of digital pathology whole slice images. It extracts the set of local morphological feature vectors under high resolution view and the global semantic feature map under low resolution view through parallel processing channels, and establishes a data foundation that combines micro-details and macro-structure. The topology graph construction module receives the set of local morphological feature vectors output by the multi-scale feature extraction module, establishes connection edges based on the spatial proximity and semantic similarity between each vector in the set of local morphological feature vectors, and constructs a cell topology graph representing the cell distribution structure, wherein the set of local morphological feature vectors is instantiated as graph nodes of the cell topology graph. The cross-scale feature fusion module receives the cell topology map output by the topology graph construction module and the global semantic feature map output by the multi-scale feature extraction module. It uses an attention mechanism to calculate the correlation weight between the graph nodes and the global semantic feature map, and weights the macro-contextual information in the global semantic feature map to the graph nodes to generate a fusion node feature vector that integrates macro- and micro-information. The identification decision module receives the fusion node feature vectors output by the cross-scale feature fusion module, parses the fusion node feature vectors through a classifier, and outputs tumor cell identification results with spatial structure constraints.

2. The tumor cell precision identification and analysis system based on digital pathological images according to claim 1, characterized in that, The specific operations of the multi-scale feature extraction module in extracting the set of local morphological feature vectors are as follows: First, a sliding window cropping operation is performed on the high-resolution view to obtain a series of local image patches, and a global spatial index matrix is ​​built to record the physical center position coordinates of each local image patch. Subsequently, dual-path parallel processing is performed for each local image patch. On the one hand, a deep residual network is used to extract basic texture features, and on the other hand, the determinant of the second-order Hessian matrix of the local image patch is calculated to capture the speckled structure of the cell nucleus. Finally, the response value of the determinant graph of the second-order Hessian matrix is ​​mapped to a nonlinear shape attention weight using the hyperbolic tangent function. The shape attention weight is then multiplied element-wise with the basic texture features to generate a set of local morphological feature vectors that incorporate second-order geometric curvature information.

3. The tumor cell precision identification and analysis system based on digital pathological images according to claim 2, characterized in that, The specific operations of the multi-scale feature extraction module to extract the global semantic feature map are as follows: A gated context convolution mechanism is used to process low-resolution views. This gated context convolution mechanism is configured with a content flow path for extracting basic semantics of organizational structure and a gated flow path for generating background suppression masks. The multi-scale feature extraction module uses a non-linear activation function with an output value range of zero to one to process the output data of the gated flow path to generate a background suppression mask. The background suppression mask is then multiplied element-wise with the output data of the content flow path to filter background noise in non-organic regions and generate a global semantic feature map that encodes macro-historical semantics.

4. The tumor cell precision identification and analysis system based on digital pathological images according to claim 3, characterized in that, The specific operations for constructing a cell topology graph using the topology graph construction module are as follows: The topology graph construction module first instantiates each local morphological feature vector in the set of local morphological feature vectors into a graph node, and then calls the physical location coordinates corresponding to the graph node from the data output by the multi-scale feature extraction module. Subsequently, the topology graph construction module constructs a multi-dimensional spatial index tree to manage the spatial distribution of all graph nodes. For any target graph node, it uses the physical location coordinates of the target graph node as the center and the preset micro-environment perception radius as the search limit to filter out a set of candidate neighbor nodes that are spatially close in the multi-dimensional spatial index tree.

5. The tumor cell precision identification and analysis system based on digital pathological images according to claim 4, characterized in that, The specific operations for establishing connecting edges in the topology graph construction module are as follows: The morphological space coupling affinity operator is used to calculate the coupling affinity weight between a node in the target graph and any candidate node in the set of candidate neighbor nodes; The topology construction module first calculates the squared Euclidean distance between the physical location coordinates of the two, and then calculates the spatial attenuation term that decays exponentially in combination with the preset spatial bandwidth parameter. Meanwhile, the topology graph construction module calculates the cosine similarity between the local morphological feature vectors of the two, uses the learnable semantic gain coefficient and similarity truncation bias to perform a linear transformation on the cosine similarity, and inputs the transformed result into a smoothing rectifier function to obtain the semantic gating term. Finally, the topology graph construction module determines the product of the spatial decay term and the semantic gating term as the coupling affinity weight, and retains only the connecting edges whose weights meet the preset conditions to generate a sparse adjacency matrix.

6. The tumor cell precision identification and analysis system based on digital pathological images according to claim 5, characterized in that, The cross-scale feature fusion module calculates relevance weights using an attention mechanism as follows: First, projection and alignment across the modal feature space are performed. The cross-scale feature fusion module calls the local morphological feature vector carried by each graph node in the cell topology graph as the data source for the query vector. Simultaneously, spatial flattening is performed on the global semantic feature map to form a feature sequence, and the feature vector at each pixel in the feature sequence is used as the data source for the key vector and value vector; Subsequently, the cross-scale feature fusion module uses a learnable linear projection matrix to map the query vector, key vector, and value vector to a unified semantic vector space.

7. The tumor cell precision identification and analysis system based on digital pathological images according to claim 6, characterized in that, The specific operation of the cross-scale feature fusion module in performing spatially constrained cross-scale attention aggregation is as follows: The spatially biased cross-attention mechanism is used to calculate the relevance weights; The cross-scale feature fusion module first calculates the dot product between the query vector of a specific graph node and the key vector of a specific global pixel to measure semantic consistency. At the same time, the cross-scale feature fusion module calls the physical location coordinates of specific graph nodes and determines the mapping coordinates of specific global pixels in the digital pathology whole slice image coordinate system, and calculates the square of the Euclidean distance between them. The product of the square of the Euclidean distance and the learnable geometric constraint coefficients is then processed using the natural logarithm function to generate a spatial penalty. Finally, the cross-scale feature fusion module subtracts the spatial penalty from the semantic consistency calculation result and inputs the subtraction result into the normalized exponential function to generate the relevance weight.

8. The tumor cell precision identification and analysis system based on digital pathological images according to claim 7, characterized in that, The specific operations of the cross-scale feature fusion module in generating the feature vector of the fused node are as follows: The cross-scale feature fusion module performs weighted aggregation of macro-contextual information by using relevance weights to perform weighted summation on the value vectors corresponding to all global pixels in the feature sequence, thereby aggregating macro-contextual information for a specific graph node. Subsequently, the cross-scale feature fusion module uses the gated residual fusion unit to calculate the balance coefficient, and then uses the balance coefficient to weight and fuse the macro-contextual information with the original local morphological feature vector to generate a fused node feature vector that combines micro-texture details and macro-organizational semantics.

9. The tumor cell precision identification and analysis system based on digital pathological images according to claim 8, characterized in that, The specific operation of the identification decision module in predicting the probability distribution based on the feature vectors of the fused nodes is as follows: The identification decision module receives the fusion node feature vector output by the cross-scale feature fusion module, and uses a classification projection head composed of multiple fully connected layers to perform feature space mapping on the fusion node feature vector, thereby outputting an initial prediction probability vector for each graph node. This initial prediction probability vector represents the confidence of the cell to which the corresponding graph node belongs for each preset pathological category. Simultaneously, the decision-making module calculates the information entropy functional of the initial prediction probability vector and uses this information entropy functional as a numerical indicator to quantify the uncertainty of the prediction distribution.

10. The tumor cell precision identification and analysis system based on digital pathological images according to claim 9, characterized in that, The specific operations for constructing the manifold consistency energy function in the identification and decision-making module are as follows: First, the initial prediction probability vectors of all graph nodes are stacked to construct the full graph prediction matrix, and then the sparse adjacency matrix generated by the topology graph construction module is called. Then, a degree matrix is ​​constructed in which the diagonal elements are equal to the sum of the elements of the corresponding rows of the sparse adjacency matrix. The degree matrix is ​​then used to normalize the sparse adjacency matrix to generate a symmetric normalized graph Laplace matrix. The identification decision module calculates the matrix product of the transpose of the full-map prediction matrix, the symmetric normalized graph Laplacian matrix, and the full-map prediction matrix, and performs a matrix trace operation on the matrix product to obtain the Laplacian smoothing term. Finally, the information entropy functional is weighted and accumulated using a learnable uncertainty penalty coefficient to obtain the maximum entropy regularization term. The Laplace smoothing term is then added to the maximum entropy regularization term to obtain the structural risk penalty term used to constrain the tumor cell identification results.

Images