Pathology detection device and method based on image recognition analysis

Abstract

The invention relates to the technical field of image data processing, and discloses a pathological detection device and method based on image recognition and analysis. According to the method, cell nucleus and gland structures are identified by acquiring a full pathological image; constructing a cross-scale heterogeneous graph containing a cell nucleus node, a gland node, a same-scale edge and a cross-scale edge based on an identification result; inputting the heterogeneous graph into a graph neural network for coding to obtain node representation fused with cross-scale context information; in the training stage, part of gland node features are randomly shielded, and self-supervised learning is carried out by predicting shielded features; in a detection stage, a prediction error of each gland node is calculated as a structural anomaly index, and a structural anomaly thermodynamic diagram is generated. According to the method, accurate quantification of tissue structure heterogeneity is achieved, the technical problems that an existing method neglects tissue structure semantics and depends on expensive labels are solved, and the method has the advantages of being high in detection accuracy, high in interpretability and good in clinical applicability.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a pathological detection device and method based on image recognition and analysis. Background Technology

[0002] Existing deep learning-based pathological image analysis methods typically divide the full pathological image into several image blocks and extract visual features such as texture and morphology of the image blocks through convolutional neural networks to classify or segment tumor regions. However, these methods have a fundamental methodological flaw: they treat pathological images as natural images, and the model learns pixel patterns rather than structural semantics.

[0003] The essence of pathological diagnosis is a structural analysis problem. Cancer cells are identified not because of abnormal individual cell morphology, but because they disrupt normal tissue structure. When doctors determine whether a gland is malignant, they rely on semantic information at the structural level, such as whether the gland outline is intact, whether the cell arrangement is disordered, and whether the glandular lumen morphology is regular. However, existing models do not truly understand the structural semantics of "how cells organize into glands and how glands organize into tissues," but instead learn the correlation between the pixel statistical distribution of local image patches and lesion labels. This makes the models extremely sensitive to "false structural perturbations" caused by film artifacts, but they struggle to identify early lesions where structural damage is not yet significant, and they are also prone to misjudgment because benign lesions happen to have texture features similar to malignant lesions.

[0004] Therefore, there is an urgent need for a pathological detection method that enables models to understand the semantics of tissue structure and to make diagnostic inferences based on the spatial organizational relationship between cells and glands. Summary of the Invention

[0005] The technical problem to be solved by this invention is that existing deep learning-based pathological image analysis and diagnosis systems are difficult to accurately identify lesions. To address this, we propose a pathological detection device and method based on image recognition and analysis.

[0006] To achieve the above objectives, this application adopts the following technical solution: a pathological detection method based on image recognition analysis, comprising the following steps: Step S1. Obtain a full pathological image of the tissue to be tested, and identify the cell nuclei and glandular structures in the image at at least two magnifications; Step S2. Construct a cross-scale heterogeneous graph based on the identified cell nuclei and glands. The cross-scale heterogeneous graph contains two types of nodes: the first type of node corresponds to each cell nucleus and is associated with a first feature vector, and the second type of node corresponds to each gland and is associated with a second feature vector. The cross-scale heterogeneous graph also includes a first type of edge constructed based on the spatial proximity relationship between cell nuclei, a second type of edge constructed based on the spatial proximity relationship between glands, and cross-scale edges established between corresponding nodes when the cell nucleus is located inside the gland. Step S3. Encode the cross-scale heterogeneous graph into the graph neural network, so that each node aggregates the information of its neighboring nodes through an attention mechanism to obtain a node representation vector that integrates cross-scale context information; wherein, the neighboring nodes include nodes of the same scale and nodes of different scales connected by cross-scale edges; Step S4. During the training phase, the second feature vectors of some gland nodes are randomly obscured. The unobscured nodes are encoded by a graph neural network and then input into the prediction network to predict the second feature vectors of the obscured nodes. The graph neural network and the prediction network are trained with the goal of minimizing the error between the predicted value and the true value. Step S5. In the detection phase, the cross-scale heterogeneous graph of the sample to be tested is input into the trained graph neural network and prediction network, and the prediction error of each gland node is calculated as the structural abnormality index of the gland. Step S6. Generate a structural abnormality heatmap of the pathological image based on the structural abnormality index, and output the detection results.

[0007] Preferably, the identification of cell nuclei includes: segmenting the outline of each cell nucleus on a high-magnification pathological image using a deep learning segmentation model, and extracting nuclear morphological features and texture features to form a first feature vector; the identification of glands includes: segmenting the outline of each gland on a medium-magnification pathological image using a deep learning segmentation model, and extracting gland morphological features and cell distribution features to form a second feature vector.

[0008] Preferably, the construction of the cross-scale heterogeneous graph specifically includes: using all cell nuclei as first-class nodes, with node features being a first feature vector containing nuclear area, nuclear perimeter, nuclear roundness, and chromatin texture; using all glands as second-class nodes, with node features being a second feature vector containing glandular lumen area, glandular morphological regularity, number of cell layers, and cell density within the gland; for each pair of cell nucleus nodes, establishing a first-class edge when their spatial Euclidean distance is less than a first preset threshold; for each pair of gland nodes, establishing a second-class edge when their spatial Euclidean distance is less than a second preset threshold; for each cell nucleus node, if its coordinates are located within the segmentation region of any gland node, then establishing a cross-scale edge between the two.

[0009] Preferably, the graph neural network is a graph attention network, and its encoding process includes: for each node, calculating its attention coefficient with all neighboring nodes, the attention coefficient being obtained based on the similarity between the node's own features and the features of its neighboring nodes; using the attention coefficient to perform weighted aggregation of the features of neighboring nodes, and updating the representation vector of the current node through nonlinear transformation; after multiple iterations, making the final representation vector of each node contain the contextual information of its local structure, the contextual information including both the distribution of cells or glands at the same scale, and the individual features of the cells under its jurisdiction or the overall features of the glands to which it belongs.

[0010] Preferably, the second feature vector of randomly occluding some gland nodes specifically includes: in each training iteration, randomly selecting some gland nodes according to a preset occlusion ratio, and replacing their second feature vectors with preset occlusion label vectors; the prediction network is a multilayer perceptron, which maps the final representation vector obtained after encoding the occluded nodes through a graph neural network to the predicted feature vector; the error is minimized by calculating the Euclidean distance between the predicted feature vectors of all occluded nodes and their true second feature vectors and averaging it as the loss function.

[0011] Preferably, the structural abnormality index is calculated as follows: for each gland node in the test sample, the final representation vector obtained after encoding by the graph neural network is input into the trained prediction network to obtain the predicted feature vector of the gland node. The Euclidean distance between the predicted feature vector and the original second feature vector of the gland node is calculated as the structural abnormality index of the gland. The index reflects the degree of coordination between the gland and the context environment composed of its surrounding cells and other glands. The larger the index, the higher the probability of structural abnormality.

[0012] Preferably, generating a structural abnormality heatmap includes: dividing the whole pathological image into a regular grid; for each grid cell, calculating the structural abnormality index of all glandular nodes within that grid, and calculating the statistic as the abnormality score of that grid cell, including the mean, maximum, or median; converting the abnormality scores of each grid cell into pixel values ​​through color mapping, and generating a heatmap aligned with the original image space, wherein darker areas in the heatmap indicate more significant structural abnormalities.

[0013] Preferably, the method also includes a diagnostic classification step: extracting global features based on the structural abnormality heatmap, the global features including the number of abnormal regions, total area, average abnormality index, and abnormal region distribution density, inputting the global features into a pre-trained classifier, and outputting the diagnostic category of the tissue to be tested, the diagnostic category including normal, benign lesion and malignant lesion.

[0014] Preferably, during the training phase, only pathological images of normal tissues are used for training, without the need for labeling of lesion areas; through self-supervised learning, the model learns the spatial organizational rules between cells and glands in normal tissue structures, thereby identifying structurally abnormal regions that deviate from the normal rules through prediction errors during the detection phase.

[0015] This invention proposes another technical solution: a pathological detection device based on image recognition analysis, comprising: The image acquisition module is used to acquire full-section pathological images of the tissue to be tested. The structure recognition module is used to identify cell nuclei and glandular structures from whole-slide pathological images at at least two magnifications, and to extract the first feature vector of each cell nucleus and the second feature vector of each gland. The graph construction module is used to construct a cross-scale heterogeneous graph based on the identified cell nuclei and glands. The cross-scale heterogeneous graph includes cell nucleus nodes, gland nodes, same-scale edges based on spatial proximity, and cross-scale edges based on inclusion. The graph encoding module is used to encode cross-scale heterogeneous graphs through graph neural networks to obtain a representation vector of each node that incorporates cross-scale context information. The prediction module is used to predict the second feature vector of the gland node and output the predicted feature vector. The anomaly scoring module is used to calculate the error between the predicted feature vector and the true second feature vector of each gland node, which serves as the structural anomaly index of the gland. The visualization output module is used to generate and output structural anomaly heatmaps based on the structural anomaly index. The graph encoding module and the prediction module are obtained through pre-training.

[0016] The technical effects and advantages of this invention are as follows: This invention transforms pathological images from pixel space into a structural semantic space by constructing a heterogeneous graph containing cell nucleus nodes, gland nodes, and cross-scale edges. This enables the model to perform diagnostic reasoning based on the spatial organizational relationship between cells and glands, thus overcoming the fundamental deficiency of existing methods that only learn local pixel patterns and cannot understand the semantics of tissue structure. At the same time, this invention adopts a self-supervised occlusion prediction strategy, which only requires normal tissue images to train the model to master normal structural patterns. By calculating the prediction error of each gland node as a structural abnormality index, it achieves accurate quantification of tissue structural aberrations, significantly reducing the dependence on expensive lesion annotation data.

[0017] Furthermore, this invention automatically focuses on key structural units through the attention mechanism of graph attention networks to generate a structural anomaly heatmap aligned with the original image. This not only improves the accuracy and robustness of anomaly detection but also provides pathologists with intuitive and traceable diagnostic evidence. It can sensitively capture subtle structural arrangement disorders in early lesions and has significant clinical practical value. Attached Figure Description

[0018] The disclosure of this invention is illustrated with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings, the same reference numerals are used to refer to the same parts: Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system architecture diagram of the present invention; Figure 3 This is a cross-scale heterogeneous diagram of the present invention. Detailed Implementation

[0019] It is readily understood that, based on the technical solution of this invention, those skilled in the art can propose various interchangeable structural methods and implementations without altering the essential spirit of the invention. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this invention and should not be considered as the entirety of the invention or as limitations or restrictions on the technical solution of this invention.

[0020] Example 1, refer to Figures 1-3 As shown, this embodiment provides a pathological detection method based on image recognition analysis. This method utilizes graph neural network technology to automatically detect tissue structural abnormalities by analyzing the cross-scale structural relationships between cell nuclei and glands in pathological images. Specifically, it includes the following steps: Step S1. Obtain a full-section pathological image of the tissue to be tested and identify cell nuclei and glandular structures: First, digital pathological whole-section images of the tissue to be tested (such as colon tissue, prostate tissue, etc.) are acquired using a whole-section scanner; in order to capture histological features at different scales, this step analyzes the images at at least two magnifications.

[0021] Specifically, on high-magnification pathological images (e.g., 40x magnification), a pre-trained deep learning segmentation model (e.g., U-Net, MaskR-CNN, etc.) is used to segment each cell nucleus in the image; after segmentation, the first feature vector is extracted for each cell nucleus. The first feature vector includes, but is not limited to: kernel area, kernel perimeter, kernel roundness, kernel length-to-minor axis ratio, and texture features reflecting chromatin distribution (such as energy, contrast, correlation, etc. extracted based on the gray-level co-occurrence matrix GLCM).

[0022] Simultaneously, on medium-magnification pathological images (e.g., 10x or 20x magnification), a deep learning segmentation model is used to segment glandular structures in the images; glands typically consist of a lumen surrounded by epithelial cells and a surrounding cell layer. For each segmented gland, its second feature vector is extracted. The second feature vector includes, but is not limited to: glandular lumen area; The perimeter of the gland's outline; Regularity of gland morphology, for example, by fitting the eccentricity of an ellipse or Fourier descriptor of its contour; Cell layer number was determined by line scanning perpendicular to the glandular lumen boundary; And the cell density within the gland, and the ratio of the number of cell nuclei within the glandular region to the glandular area.

[0023] Through the above dual-scale recognition, basic data at both the microscopic (cell nucleus) and mesoscopic (glandular) levels in pathological images were obtained.

[0024] Step S2. Construct a cross-scale heterogeneous graph: Based on the cell nuclei and glands identified in step S1, this step constructs a cross-scale heterogeneous map that can characterize the spatial topological relationships of tissues. The graph structure contains two types of nodes and three types of edges.

[0025] Node construction: Define a collection of nodes .

[0026] Type I nodes (nuclear nodes) Each cell nucleus identified in the image corresponds to a node in the diagram. Each cell nuclear node The first feature vector extracted in step S1 is associated with the first feature vector. .

[0027] Second type of node (gland node) Each gland identified in the image corresponds to a node in the graph. Each gland node The second feature vector extracted in step S1 is associated with this feature vector. .

[0028] Edge construction: Define the set of edges .

[0029] Type I edge (cell-to-cell edge) Used to characterize the local proximity relationship between cell nuclei; for any two nucleus nodes and Calculate the spatial Euclidean distance between their centroids; when this distance is less than a first preset threshold... Time (e.g.) Set to twice the average diameter of the cell nucleus, and establish an undirected edge between them; this allows the model to perceive the crowding and local distribution patterns of the cell nuclei.

[0030] Type II edge (gland-gland edge) : Used to characterize the spatial arrangement relationship between glands; for any two gland nodes and Calculate the spatial Euclidean distance between their centroids; when this distance is less than a second preset threshold... Time (e.g.) Set to twice the average diameter of the glands, and establish an undirected edge between them; this allows the model to capture macroscopic structural features such as whether the glands are arranged regularly, whether they are crowded or sparse.

[0031] Cross-scale edge : Used to characterize the subordinate and containment relationship between the cell nucleus and glands; for each cell nucleus node Determine whether its coordinates are located at any gland node. Within the segmented polygon region; if so, then within and A directed edge is established between them (usually from the cell nucleus to the gland, or a bidirectional connection); this cross-scale connection allows microscopic nuclear features to be transmitted upward to the gland, while also allowing the overall features of the gland to modulate the understanding of individual cells downward.

[0032] Through the above construction process, the pathological image is transformed into graph data rich in structured information, laying the foundation for subsequent deep learning modeling.

[0033] Step S3. Graph Neural Network Encoding: The constructed cross-scale heterogeneous graph is input into the graph neural network for encoding; in this embodiment, a graph attention network is preferably used as the encoder.

[0034] First, define the initial feature vectors of the nodes; the initial feature vectors of all nodes are denoted as... For nuclear nodes Its initial eigenvector That is, the first feature vector extracted in step S1; for gland nodes Its initial eigenvector That is, the second feature vector extracted in step S1.

[0035] Graph attention networks iteratively update the feature representation of each node through multiple layers of graph convolution operations; in the... layer( ),node The feature update formula is as follows: ; in: It is a node The set of neighboring nodes; in the heterogeneous graph of this invention, neighbors include: neighbors of the same scale (such as those obtained through...). Other cell nuclei connected, or through Other glands connected), and neighbors across scales (such as through Connected glands or cell nuclei), and the node itself (achieved by adding self-loop edges).

[0036] It is the first The learnable linear transformation matrix of the layer.

[0037] It is a non-linear activation function, using ELU or ReLU. It is the attention coefficient, representing the neighboring nodes. For nodes Importance; through nodes and nodes Feature similarity calculation yields: ; in It is the first The learnable attention vector of the layer has a dimension of 512×1, and... The splicing dimension matching uses random normal initialization (mean 0, variance 0.01). Attention vector The transpose of the matrix has a dimension of 1×512. The purpose of the transpose operation is to perform the dot product calculation with the concatenated feature vectors to obtain the scalar attention score. This indicates the feature vector column concatenation operation, which concatenates the feature vector columns. (256-dimensional) and (256-dimensional) concatenated along the column direction to form a 512-dimensional feature vector, providing dimension matching for dot product calculation. All layers are concatenated in column to ensure computational consistency. LeakyReLU is a non-linear activation function used to perform a non-linear transformation on attention scores to avoid linear correlation of features; its negative slope parameter is preferably 0.2, with a value range of 0.1~0.3. This parameter can ensure that the gradient in the negative interval is not 0, thus avoiding neuron death.

[0038] This is an exponential function used to convert the attention score into a positive value, thus avoiding negative values ​​during subsequent normalization.

[0039] denominator For nodes The attention scores of all neighboring nodes k are summed to normalize the attention coefficients, ensuring that the sum of the attention coefficients of all neighboring nodes is 1, thus avoiding feature distortion caused by excessively large coefficients; the summation range includes the self-loop edges of the node itself.

[0040] go through After iterative propagation through the layers, each node (whether it's a cell nucleus or a gland) obtains a final high-dimensional representation vector. The representation vector not only contains the original features of the node itself, but also aggregates the information of its multi-hop neighbors (preferably 3 hops). Specifically, for gland nodes, the representation vector integrates the morphological features of the internal cell nucleus (through cross-scale edges) and the arrangement information of the surrounding glands (through gland-gland edges), thereby achieving deep fusion of cross-scale contextual information, enabling the model to understand the structural semantics of how cells constitute glands and how glands are organized into tissues.

[0041] Step S4. Training Phase—Self-Supervised Learning: A self-supervised learning strategy is used to train the graph neural network, which solves the technical problem of scarce lesion annotation data in pathological images. In this stage, only pathological images of normal tissues are used for training, without any lesion annotations. The normal tissue samples in the training set need to cover multiple tissue types (colon, prostate, breast, etc.) with a sample size of ≥200 cases.

[0042] The core idea of ​​training is occlusion and prediction: randomly occlude some features of gland nodes in the image, forcing the model to use the remaining contextual information (surrounding cell nuclei, other glands) to recover the occluded features; if the model can make accurate predictions, it means that it has mastered the inherent laws of normal tissue structure.

[0043] The specific process is as follows: 1. Random Occlusion: In each training iteration, random occlusion is performed according to a preset occlusion ratio. From the set of gland nodes Randomly select a subset of nodes as the occlusion set , The value range is 15% to 30%; for sets Each gland node in Input its features (i.e., the second feature vector) Replace it with a preset masking marker vector. The masking marker vector can be a zero vector with the same dimension as the second feature vector, or it can be set as a learnable embedding vector.

[0044] 2. Graph Encoding: Input the heterogeneous graph with occlusion markers into the graph attention network described in step S3; when processing occluded nodes, the network's initial feature vectors... It is an all-zero vector, and its representation vector can only be updated by relying on information from its neighboring nodes (unmasked cell nuclei and glands).

[0045] 3. Feature prediction: For each masked gland node The final representation vector after being encoded by a graph attention network. Input a prediction network; specifically, this prediction network is a multilayer perceptron with the following structure: 256 dimensions (input) → 64 dimensions (hidden layers) → 256 dimensions (output). The hidden layer activation function is ReLU, and the output layer has no activation function. The core function of the prediction network is to map the high-dimensional graph representation vector back to the original feature space, and it is responsible for mapping the high-dimensional graph representation back to the original feature space, outputting a predicted feature vector. Dimension and the true second feature vector Consistent (256 dimensions).

[0046] 4. Loss Calculation and Optimization: Calculate the predicted feature vector. The true second feature vector of that node The error between them; this embodiment uses Euclidean distance as the error metric and calculates the average error of all shading nodes as the loss function. : ; For occlusion set The number of gland nodes in the image is a positive integer, determined by the occlusion ratio. Total number of gland nodes Decide, ; The square of the Euclidean distance between the predicted feature vector and the true feature vector is chosen because: it amplifies the feature differences, improving the model's training sensitivity; and it eliminates the square root, simplifying gradient calculation and avoiding gradient oscillations during training. The formula for calculating the square of the Euclidean distance is: , The dimension index of the feature vector; The total error is obtained by summing the squares of the Euclidean distances of all shading nodes; The total error is averaged to obtain the average loss, which eliminates the influence of the number of occluded nodes on the loss value and ensures the comparability of loss values ​​in different training iterations.

[0047] Model optimization method: Minimize the loss function using stochastic gradient descent algorithm, with a preferred learning rate of 0.001, a preferred weight decay coefficient of 0.0001 (to prevent overfitting), a preferred batch size of 8, and a preferred number of training epochs of 100; simultaneously optimize all learnable parameters of the graph attention network and the prediction network using backpropagation algorithm. , The weight matrix of the MLP is used. When the loss function converges (the change in loss value is <0.0001 for 10 consecutive rounds), training is stopped and the model parameters are saved.

[0048] Through this self-supervised training, the model learns prior knowledge about "how cell nuclei form glands" and "how glands are arranged" in normal tissues.

[0049] Step S5. Detection phase: Calculate the structural anomaly index: After the model training is completed, the detection phase is entered to process the test samples (which may contain lesions). The core is to quantify the degree of abnormality of glandular structure through prediction error and use the prior knowledge of normal tissue from self-supervised training to identify glandular structures that deviate from the normal pattern.

[0050] For the pathological whole-slice image of the tissue to be tested, first construct its cross-scale heterogeneous map according to steps S1 and S2; at this time, no feature masking is performed, and the true features of all nodes are preserved.

[0051] The constructed graph is input into the trained graph attention network and prediction network; for each gland node in the graph... Perform the following operations in sequence: 1. Encode the heterogeneous graph using a trained graph attention network and calculate gland nodes. The final representation vector .

[0052] 2. Input the trained multilayer perceptron prediction network to obtain the predicted feature vector. .

[0053] 3. Calculate the structural abnormality index of this glandular node. The formula is as follows: ; To predict the Euclidean distance (without squares) between the feature vector and the true feature vector, the difference from the loss function is that the square is discarded. The reasons are: to restore the physical meaning of the anomaly index, so that the index value matches the actual magnitude of the feature difference; and to avoid the index value being too large, so as to facilitate subsequent threshold setting and visualization.

[0054] The structural abnormality index of glandular nodes is a non-negative real number. Its physical meaning is the degree of deviation of the glandular structure from the normal tissue pattern, and it also reflects the degree of coordination between the gland and the contextual environment formed by its surrounding cells and other glands. The physical meaning of this index is that if the glandular structure in the tested tissue conforms to the spatial distribution pattern of normal tissue, then the features predicted by the model based on its context (surrounding cells and glands) are... Will with its true characteristics Highly consistent, at this time The predicted values ​​are relatively small; conversely, if the glands become diseased (e.g., structurally disordered, or morphologically abnormal), their characteristics will deviate from the patterns learned by the model on normal data, leading to a significant difference between the predicted and actual values. Larger; therefore, This directly reflects the possibility of glandular structural abnormalities.

[0055] Step S6. Generate and output the structural anomaly thermal map: To visually represent the lesion area, this step generates a heatmap based on the calculated structural anomaly index.

[0056] 1. Grid division: The original whole-slice pathological image is divided into a regular spatial grid, for example, a grid unit of 256×256 pixels (based on the resolution of 0.25μm / pixel of the whole-slice scanner, corresponding to an actual physical size of 64×64μm). This grid size is the preferred value and can be adaptively adjusted according to the image magnification: if the high magnification is set to 100x, the image is adjusted to 128×128 pixels; if the low magnification is set to 50x, the image is adjusted to 512×512 pixels. The core principle is to ensure that each grid unit contains at least 1 to 2 gland nodes to avoid the abnormal area being blurred due to the grid being too large.

[0057] 2. Grid scoring: For each grid cell Count the set of all gland nodes located within this grid range. Calculate the anomaly score for this grid. The calculation method uses statistics of glandular abnormality indices within the set, such as the mean, maximum, or median; preferably, the mean is used to reflect the overall degree of abnormality, or the maximum is used to capture the most significant lesions. ; Represents grid cells The number of gland nodes within the system is a non-negative integer; If there are no glands in the grid, the score is set to 0.

[0058] 3. Color Mapping: Resolve anomaly scores for all grid cells. Normalization is performed (e.g., normalized to the 0-1 range), and a color map (such as a Jet color map or a Hot color map) is applied to convert it to RGB pixel values; the darker the color (e.g., red), the higher the abnormality index, and the lighter the color (e.g., blue), the more normal the tissue structure.

[0059] 4. Output Results: The generated heatmap is spatially aligned with the original pathological image and can be directly overlaid or displayed side-by-side for pathologists to review. Doctors can focus on highlighted areas, thereby improving diagnostic efficiency and accuracy.

[0060] Furthermore, as an extension of this embodiment, a diagnostic classification step is also included: Based on the generated structural anomaly heatmap, global features can be further extracted, such as the number of connected components in the anomalous region, the total area of ​​the anomalous region, the average anomalous index of the entire area, and the distribution density of the anomalous region. These global features are then input into a pre-trained classifier (such as a support vector machine, random forest, or a simple fully connected neural network) to output the final diagnostic category of the tissue under test, such as "normal," "benign lesion," or "malignant lesion." This achieves a closed loop from anomaly localization to qualitative diagnosis.

[0061] Example 2: This example provides a pathological examination device for implementing the above method; the device includes: an image acquisition module, used to acquire a pathological whole-section image of the tissue to be tested through a digital slide scanner interface or a database interface.

[0062] The structure recognition module is equipped with a deep learning segmentation model to identify cell nuclei and glandular structures from whole-slide pathological images at at least two magnifications. This module is also responsible for extracting the first feature vector (morphological and texture features) for each cell nucleus and the second feature vector (morphological and cell distribution features) for each gland.

[0063] The graph construction module is used to construct cross-scale heterogeneous graphs based on the identified cell nuclei and glands. This module is responsible for instantiating nodes and edges, specifically including: mapping cell nuclei and glands as nodes, establishing same-scale edges (cell-cell edges, gland-gland edges) based on spatial distance thresholds, and establishing cross-scale edges (cell-gland edges) based on coordinate inclusion relationships.

[0064] The graph encoding module, which incorporates a graph neural network (preferably a graph attention network), is used for multi-layer encoding of heterogeneous graphs across scales. This module aggregates neighbor node information through an attention mechanism and outputs a representation vector for each node that incorporates cross-scale contextual information.

[0065] The prediction module contains a prediction network (such as a multilayer perceptron); during the training phase, it is used to predict the features of occluded nodes based on the output of the graph coding module; during the detection phase, it is used to predict the features of all gland nodes based on the output of the graph coding module.

[0066] The anomaly scoring module is used to calculate the structural anomaly index of each gland node during the detection phase. Specifically, it compares the predicted feature vector output by the prediction module with the original second feature vector of the gland node and calculates the Euclidean distance as the anomaly index.

[0067] The visualization output module is used to generate structural anomaly heatmaps based on the structural anomaly index. It grids the image, calculates grid scores, performs color mapping, and outputs visualized detection results; optionally, this module may also include a classification unit for outputting diagnostic classification results based on heatmap features.

[0068] Each module in the above-mentioned device can be implemented by hardware (such as DSP, FPGA, ASIC) or by software (such as computer program code) running on a general-purpose processor; through the coordinated work of each module, the device can efficiently and accurately complete the task of detecting structural abnormalities in pathological images.

[0069] Example 3: Based on the above examples, this example further supplements and optimizes some technical details.

[0070] Regarding the construction of cross-scale heterogeneous graphs: When constructing the first type of edge (cell-to-cell edge), the first preset threshold is... The choice of threshold is crucial; too small a threshold will result in an overly sparse graph, preventing effective information transmission; too large a threshold will introduce noise and increase computational complexity; in this embodiment, The preferred setting is 1.5 to 2.5 times the average diameter of the cell nucleus; similarly, for the second type of edge (gland-gland edge), the second preset threshold is... The preferred setting is 1.5 to 3 times the average diameter of the gland; this dynamic threshold setting based on statistical characteristics can adapt to images with different scanning magnifications and tissue types.

[0071] Regarding the attention mechanism of graph neural networks: To prevent model overfitting, a multi-head attention mechanism can be introduced when calculating the attention coefficients; that is, the attention is run in parallel. Each head has an independent attention head, learning different feature concerns. The outputs of these heads are then concatenated or averaged. This enhances the model's ability to capture complex structural patterns. Furthermore, to avoid gradient vanishing, the activation function... The ELU function should be used preferentially.

[0072] Regarding occlusion strategies in self-supervised training: In addition to random occlusion, structured occlusion strategies can be combined; for example, randomly selecting a gland node and occluding not only the node itself, but also other gland nodes in its spatial neighborhood. This strategy forces the model to use not only local cell information, but also information about the arrangement of glands at a greater distance for inference, thereby enhancing the model's ability to understand the global context.

[0073] Applications of the structural anomaly index: In practical applications, tissue staining may differ among different patients, resulting in different baselines for the feature vectors. To eliminate such non-pathological differences, the feature vectors can be standardized (e.g., Z-score standardization) before calculating the structural abnormality index, or the calculated abnormality index can be normalized based on the statistical distribution of the whole slice to ensure that the abnormality indices of different samples are comparable.

[0074] Regarding the generation of heatmaps: To improve the smoothness and visual appeal of heatmaps, a Gaussian filter can be applied to smooth out the abnormal score map after grid scoring. In addition, the color mapping scheme can adopt the "medical heatmap" color scheme that doctors are accustomed to, that is, low-scoring areas are displayed as transparent or light blue, and high-scoring areas are displayed as green, yellow or even red, in order to conform to the intuition of clinical image reading.

[0075] The technical scope of this invention is not limited to the content described above. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the protection scope of this invention.

Claims

1. A pathological detection method based on image recognition analysis, characterized in that, Includes the following steps: Step S1. Obtain a full pathological image of the tissue to be tested, and identify the cell nuclei and glandular structures in the image at at least two magnifications; Step S2. Construct a cross-scale heterogeneous graph based on the identified cell nuclei and glands. The cross-scale heterogeneous graph contains two types of nodes: the first type of nodes corresponds to each cell nucleus and is associated with a first feature vector, and the second type of nodes corresponds to each gland and is associated with a second feature vector. The cross-scale heterogeneous graph also includes a first type of edge constructed based on the spatial proximity relationship between cell nuclei, a second type of edge constructed based on the spatial proximity relationship between glands, and cross-scale edges established between corresponding nodes when the cell nucleus is located inside the gland. Step S3. Encode the cross-scale heterogeneous graph input graph neural network so that each node aggregates the information of its neighboring nodes through an attention mechanism to obtain a node representation vector that integrates cross-scale context information; wherein, the neighboring nodes include nodes of the same scale and nodes of different scales connected by cross-scale edges; Step S4. During the training phase, the second feature vectors of some gland nodes are randomly obscured. The unobscured nodes are encoded by the graph neural network and then input into the prediction network to predict the second feature vectors of the obscured nodes. The graph neural network and the prediction network are trained with the goal of minimizing the error between the predicted value and the true value. Step S5. In the detection phase, the cross-scale heterogeneous graph of the sample to be tested is input into the trained graph neural network and prediction network, and the prediction error of each gland node is calculated as the structural abnormality index of the gland. Step S6. Generate a structural abnormality heatmap of the pathological image based on the structural abnormality index, and output the detection results.

2. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, The identification of the cell nucleus includes: segmenting the outline of each cell nucleus on a high-magnification pathological image using a deep learning segmentation model, and extracting nuclear morphological features and texture features to form a first feature vector; the identification of the gland includes: segmenting the outline of each gland on a medium-magnification pathological image using a deep learning segmentation model, and extracting gland morphological features and cell distribution features to form a second feature vector.

3. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, The construction of the cross-scale heterogeneous graph specifically includes: using all cell nuclei as first-class nodes, with node features being a first feature vector containing nuclear area, nuclear perimeter, nuclear roundness, and chromatin texture; using all glands as second-class nodes, with node features being a second feature vector containing glandular lumen area, glandular morphological regularity, number of cell layers, and cell density within the gland; for each pair of cell nucleus nodes, establishing a first-class edge when their spatial Euclidean distance is less than a first preset threshold; for each pair of gland nodes, establishing a second-class edge when their spatial Euclidean distance is less than a second preset threshold; for each cell nucleus node, if its coordinates are located within the segmentation region of any gland node, then establishing a cross-scale edge between the two.

4. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, The graph neural network is a graph attention network, and its encoding process includes: for each node, calculating its attention coefficient with all neighboring nodes, the attention coefficient being obtained based on the similarity between the node's own features and the features of its neighboring nodes; using the attention coefficient to perform weighted aggregation of the features of neighboring nodes, and updating the representation vector of the current node through nonlinear transformation; after multiple iterations, making the final representation vector of each node contain the contextual information of its local structure, the contextual information including both the distribution of cells or glands at the same scale, and the individual features of the cells under its jurisdiction or the overall features of the glands to which it belongs.

5. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, The second feature vector of the randomly occluded gland nodes specifically includes: in each training iteration, randomly selecting some gland nodes according to a preset occlusion ratio, and replacing their second feature vectors with preset occlusion label vectors; the prediction network is a multilayer perceptron, which maps the final representation vector obtained after encoding the occluded nodes through a graph neural network to the predicted feature vector; the error minimization is achieved by calculating the Euclidean distance between the predicted feature vectors of all occluded nodes and their true second feature vectors and averaging it as the loss function.

6. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, The structural anomaly index is calculated as follows: For each gland node in the test sample, the final representation vector obtained after encoding by the graph neural network is input into the trained prediction network to obtain the predicted feature vector of the gland node. The Euclidean distance between the predicted feature vector and the original second feature vector of the gland node is calculated as the structural anomaly index of the gland. This index reflects the degree of coordination between the gland and the context environment formed by its surrounding cells and other glands. The larger the index, the higher the probability of structural anomaly.

7. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, The process of generating a structural anomaly heatmap includes: dividing the full pathological image into a regular grid; for each grid cell, calculating the structural anomaly index of all glandular nodes within that grid, and calculating a statistic as the anomaly score for that grid cell, wherein the statistic includes the mean, maximum, or median; converting the anomaly score of each grid cell into pixel values ​​through color mapping, and generating a heatmap aligned with the original image space, wherein darker areas in the heatmap indicate more significant structural anomalies.

8. The pathological detection method based on image recognition analysis according to claim 1, characterized in that, It also includes a diagnostic classification step: extracting global features based on the structural abnormality heatmap, the global features including the number of abnormal regions, total area, average abnormality index, and abnormal region distribution density, inputting the global features into a pre-trained classifier, and outputting the diagnostic category of the tissue to be tested, the diagnostic category including normal, benign lesion and malignant lesion.

9. A pathological detection method based on image recognition analysis according to any one of claims 1-8, characterized in that, The training phase uses only pathological images of normal tissues for training, without the need for labeling of lesion areas; through self-supervised learning, the model learns the spatial organizational patterns between cells and glands in normal tissue structures, thereby identifying structurally abnormal regions that deviate from the normal patterns through prediction errors in the detection phase.

10. A pathological examination device based on image recognition analysis, used to implement the pathological examination method based on image recognition analysis as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to acquire full-section pathological images of the tissue to be tested. The structure recognition module is used to identify cell nuclei and glandular structures from whole-slide pathological images at at least two magnifications, and to extract the first feature vector of each cell nucleus and the second feature vector of each gland. The graph construction module is used to construct a cross-scale heterogeneous graph based on the identified cell nuclei and glands. The cross-scale heterogeneous graph includes cell nucleus nodes, gland nodes, same-scale edges based on spatial proximity, and cross-scale edges based on inclusion. The graph encoding module is used to encode the cross-scale heterogeneous graph through a graph neural network to obtain a representation vector of each node that incorporates cross-scale context information. The prediction module is used to predict the second feature vector of the gland node and output the predicted feature vector. The anomaly scoring module is used to calculate the error between the predicted feature vector and the true second feature vector of each gland node, which serves as the structural anomaly index of the gland. A visualization output module is used to generate and output a structural anomaly heatmap based on the structural anomaly index. The graph encoding module and prediction module are obtained through pre-training.

Images