Pathological image auxiliary diagnosis model construction and image classification method and system oriented to multi-source heterogeneous data fusion

Abstract

The invention provides a pathological image auxiliary diagnosis model construction and image classification method and system for multi-source heterogeneous data fusion, and relates to the technical field of image processing. The method comprises the core steps of constructing an instantiated local tissue atlas through pixel-level registration of a panoramic form image and a molecular identity image, analyzing a wrapping state of each epithelial cell nest and a context position in the atlas based on the atlas, and performing contradiction analysis according to the wrapping state and the context position so as to distinguish in-situ cancer and invasive cancer components. The core advantage of the method is that morphology, molecule and anatomy knowledge are deeply fused in a unified space framework, and the information island problem in the prior art is solved. And through mapping spatial relationship quantification, the identification accuracy of the micro-infiltrating kitchen range is remarkably improved, and the interpretability of the model is enhanced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for constructing a pathological image-assisted diagnostic model and classifying images for multi-source heterogeneous data fusion. Background Technology

[0002] Accurate pathological diagnosis of breast diseases, especially the differentiation between ductal carcinoma in situ and invasive ductal carcinoma, and the identification of microinvasive lesions, are the core basis for clinical treatment planning and prognosis assessment. This diagnostic process highly relies on professionals' comprehensive interpretation of multi-dimensional information from tissue sections. Traditionally, pathological diagnosis is mainly based on panoramic morphological images obtained from routine hematoxylin and eosin (H&E) staining, judging by observing the morphology of epithelial cell nests, mitotic figures, and their relationship with the surrounding stroma. However, H&E morphology alone is sometimes insufficient for decision-making, for example, when in situ cancer cells breach the basement membrane and cause early, minute invasion, or when strong stroma reactions lead to ductal structural distortion. Therefore, immunohistochemical (IHC) staining has become an indispensable auxiliary tool, and its visualization of myoepithelial integrity is one of the gold standards for distinguishing between intraductal (in situ) and extraductal (invasive) lesions. Currently, the development of digital pathology and artificial intelligence technologies has spurred many automated analysis algorithms based on single data sources (such as H&E only or IHC only) that attempt to detect and classify cell nuclei or segment regions.

[0003] Despite advancements in existing technologies, they still exhibit significant limitations when addressing the aforementioned complex diagnostic tasks, essentially stemming from information silos and cognitive fragmentation. Most methods analyze only a single type of pathological image (morphological or molecular), lacking a mechanism for deep integration based on precise spatial alignment of the rich morphological information provided by Hematologic & Epithelial Processing (H&E) with the precise molecular identity information provided by Intracellular Pathology (IHC). Simply using the analysis results of the two images in parallel, while ignoring their precise correspondence at the cellular and tissue structure level, can lead to feature association errors, such as incorrectly associating a morphologically questionable nest of epithelial cells with a neighboring, rather than the myoepithelial, area of ​​absence. Furthermore, existing methods typically focus on extracting low-level visual features from images, failing to formally encode and integrate prior histological knowledge into the analytical model. This results in algorithms lacking similar anatomical common sense, making it difficult to understand the abnormality and potential risks of an epithelial cell cluster isolated in adipose tissue, detached from its normal ductal location.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for constructing a pathological image-assisted diagnostic model and classifying images for multi-source heterogeneous data fusion, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for constructing a pathological image-assisted diagnostic model and classifying images for multi-source heterogeneous data fusion, comprising the following steps:

[0008] Step 1: Obtain panoramic morphological images, molecular identity images, and tissue structure knowledge graphs of the same breast tissue specimen, and perform pixel-level spatial registration on the panoramic morphological images and molecular identity images;

[0009] Step 2: Identify epithelial cell nests in the panoramic morphology image and extract the first feature set; identify and segment regions expressing myoepithelial cell markers from the molecular identity image, and generate a binary myoepithelial distribution semantic layer;

[0010] Step 3: Segment and identify key structural components on the panoramic morphological image; map the identified structural components to the corresponding nodes of the tissue knowledge graph to construct an instantiated local tissue graph for the panoramic morphological image;

[0011] Step 4: For each identified epithelial cell nest, perform feature analysis and contradiction resolution; the feature analysis refers to determining the spatial relationship of the epithelial cell nest in the myoepithelial distribution semantic layer and quantifying its encapsulation state, while determining its contextual position in the instantiated local tissue atlas; the contradiction resolution, based on the feature analysis results, determines whether the epithelial cell nest is an in situ carcinoma component or an invasive carcinoma component.

[0012] Step 5: Based on the results of feature analysis and contradiction analysis, perform a comprehensive diagnostic classification of the entire tissue region represented by the panoramic morphological image and generate a visualization result map; the comprehensive diagnostic classification includes at least: ductal carcinoma in situ, ductal carcinoma in situ with significant stromal reaction, invasive ductal carcinoma, and ductal carcinoma in situ with microinvasiveness.

[0013] Furthermore, the panoramic morphological image is a hematoxylin-eosin stained section image; the molecular identity image refers to an immunohistochemical staining digital image of consecutive sections from the same tissue block; the immunohistochemical staining includes at least a marker for myoepithelial cells; the marker includes at least a p63 protein marker or a Calponin protein marker;

[0014] The tissue structure knowledge graph is a graph model based on the predefined histological anatomy of the breast, used to describe the topological relationships of the normal ductal lobule system of the breast; the nodes of the graph model include at least terminal ducts, lobular alveoli, large ducts, fibrous stroma, and adipose tissue; the edges of the graph model are used to define the inherent spatial connection relationships between different types of nodes;

[0015] Pixel-level spatial registration includes: selecting several sets of corresponding feature points distributed in different tissue regions in the panoramic morphological image and the molecular identity image; calculating the spatial transformation model based on the set of corresponding feature points using affine transformation or elastic transformation algorithms; and resampling the molecular identity image using the spatial transformation model to achieve precise alignment with the panoramic morphological image at the cellular and tissue structure level.

[0016] Furthermore, the epithelial cell nests are obtained by processing the registered panoramic morphological image through a pre-trained segmentation model; the segmentation model is a deep learning-based instance segmentation model, which is trained on a panoramic morphological image with pre-annotated epithelial cell nest regions.

[0017] Furthermore, the first feature set includes at least the morphological contour features of epithelial cell nests, nuclear polymorphism features, and reactivity features of the surrounding stroma;

[0018] The morphological contour features of epithelial cell nests include the irregularity of the outer contour of the epithelial cell nest, the face-to-circumference ratio, and the depth of the convex hull defect;

[0019] Nuclear polymorphism features are obtained by image segmentation and measurement of the nuclei in the epithelial cell nest region, specifically including the average area of ​​all segmented nuclei, the ratio of the major and minor axes of each nucleus, and the uniformity of chromatin texture.

[0020] The reactivity characteristics of the surrounding stroma were obtained by analyzing images of the stroma region immediately adjacent to the boundary of the epithelial cell nest, specifically including the density of fibroblasts and the infiltration density of inflammatory cells;

[0021] Using a threshold-based segmentation method or a pre-trained deep learning semantic segmentation model, positive signal regions that are colored by immunohistochemical staining are identified in the molecular identity image; the positive signal regions correspond to regions expressing myoepithelial cell markers; the pixel values ​​of the positive signal regions in the molecular identity image are set as the first value, and the pixel values ​​of the non-positive signal regions are set as the second value, thereby generating a binarized myoepithelial distribution semantic layer.

[0022] When the molecular identity image is an immunohistochemical staining image targeting the p63 protein, the deep learning semantic segmentation model is specifically trained to recognize positive signals in the nuclear staining pattern; when the molecular identity image is an immunohistochemical staining digital image targeting the Calponin protein, the deep learning semantic segmentation model is specifically trained to recognize positive signals in the cytoplasmic staining pattern.

[0023] Furthermore, key structural components are segmented and identified on the panoramic morphological image, specifically including:

[0024] Segmentation and identification of structural components are performed using a pre-trained deep convolutional neural network semantic segmentation model. The input of the deep convolutional neural network semantic segmentation model is a panoramic morphological image, and its output is a pixel-level classification label map of the same size as the input image. Each pixel of the pixel-level classification label map is classified into one of the following categories: epithelial region, fibrous stroma region, fat region, ductal structure outline region, and background region.

[0025] The epithelial regions identified by the deep convolutional neural network semantic segmentation model are screened, and the epithelial regions that meet the following conditions are determined as ductal structure contour regions: the proportion of the number of outer contour pixels of the epithelial region belonging to the positive signal region in the myoepithelial distribution semantic layer is counted. If the proportion of the number exceeds the preset first proportion threshold, the epithelial region is determined to be wrapped by myoepithelial tissue and the epithelial region is classified as ductal structure contour region.

[0026] The identified organizational structure components are mapped to corresponding nodes in the organizational structure knowledge graph, specifically as follows:

[0027] The segmented and identified ductal structure contour regions are mapped to terminal ducts or large duct nodes in the tissue structure knowledge graph; the fibrous stroma regions are mapped to fibrous stroma nodes; and the adipose regions are mapped to adipose tissue nodes. At the same time, based on the geometric center and spatial distribution of the ductal structure contour regions, their spatial connection relationships as ductal structure nodes are reconstructed, thereby generating an instantiated local tissue map.

[0028] In the instantiated local tissue knowledge graph, by calculating the spatial distance and connectivity between nodes, the adjacency relationship between each ductal structure contour node and the nearest fibrous interstitial node and adipose tissue node is defined, and this adjacency relationship is quantified as the edge attribute of the graph to represent the normal spatial adjacency relationship.

[0029] Furthermore, quantifying the encapsulation state of the epithelial cell nests specifically includes:

[0030] The encapsulation status of epithelial cell nests includes fully encapsulated, partially encapsulated, and not encapsulated. The specific judgment logic is as follows:

[0031] The algorithm calculates the percentage of continuous length by which the contour boundary of the epithelial cell nest is covered by the positive signal region in the myoepithelial distribution semantic layer. If the percentage of continuous length is higher than a first preset threshold, the epithelial cell nest is considered to be completely wrapped. If the percentage of continuous length is lower than a second preset threshold, the epithelial cell nest is considered to be unwrapped. If the percentage of continuous length is between the first and second preset thresholds, the epithelial cell nest is considered to be incompletely wrapped. The calculation logic for the percentage of continuous length is as follows: extract the pixel-level contour of the epithelial cell nest in the panoramic morphological image; sample along the pixel-level contour in the myoepithelial distribution semantic layer and record whether each sampling point is a positive signal; calculate the total length of the continuous line segment formed by all sampling points that are positive signals, and calculate its percentage of the perimeter of the entire pixel-level contour. This percentage is defined as the percentage of continuous length.

[0032] The contextual position of the epithelial cell nest in the instantiated local tissue atlas is determined by judging the spatial relationship between the epithelial cell nest and the nodes in the instantiated local tissue atlas. The specific method for determining this is as follows:

[0033] Spatial inclusion relationship is determined between the geometric contour of the epithelial cell nest and the polygonal region defined by all ductal structure contour nodes in the instantiated local tissue map. If all pixels of the epithelial cell nest are located within a certain polygonal region, it is determined that it is completely within the ductal structure contour node corresponding to the polygonal region, and then a contextual position label within the duct is assigned to it.

[0034] Conversely, an isolation determination is performed: the Euclidean distance from the geometric center of the epithelial cell nest to the nearest fibrous mesenchymal node and adipose tissue node is calculated; if the Euclidean distance to the nearest adipose tissue node is less than the Euclidean distance to the nearest fibrous mesenchymal node and less than the preset isolation distance threshold, then an intra-adipose isolated context location label is assigned to it; otherwise, an intra-messenchymal suspected context location label is assigned to it.

[0035] The aforementioned contradiction analysis refers to determining whether an epithelial cell nest is a component of carcinoma in situ or invasive carcinoma based on the results of feature analysis through a hierarchical decision process. The specific judgment rules are as follows:

[0036] An epithelial cell nest is considered an invasive carcinoma component if it meets all of the following criteria:

[0037] The context location label is isolated within fat or suspected within the interstitium, and the wrapping status is unwrapped;

[0038] If the epithelial cell nest does not meet the criteria for invasive carcinoma, it shall be re-evaluated according to the following sub-rules:

[0039] Sub-rule 1: The context location label is "inside the duct", the encapsulation status is "completely encapsulated", and the morphological outline features show high irregularity and / or significant nuclear polymorphism features;

[0040] Sub-rule 2: The context location label is "inside the catheter," the wrapping status is "not wrapped or incompletely wrapped," and the reactive characteristics of the surrounding stroma show significant pro-fibrotic connective tissue proliferation or inflammatory infiltration.

[0041] If the epithelial cell nest satisfies either sub-rule one or sub-rule two above, it is determined to be a carcinoma in situ component.

[0042] The high irregularity refers to the irregularity of the epithelial cell nest being greater than a preset first morphological threshold, and its face-to-circumference ratio being less than a preset first proportional threshold.

[0043] The significant nuclear polymorphism characteristic refers to the fact that the standard deviation of the area of ​​all nuclei in the epithelial cell nest is greater than the preset first nuclear state threshold, and the average of the long and short axis ratios of all nuclei is greater than the preset first axis ratio threshold.

[0044] Based on the pixel-level outline of the epithelial cell nest, a band-shaped region with a preset width is generated by morphological dilation operation, which only includes the outer side of the original outline. When the fibroblast density calculated in this band-shaped region is greater than the preset first cell density threshold, it is judged that there is significant fibrotic connective tissue proliferation. When the inflammatory cell density calculated in this band-shaped region is greater than the preset second cell density threshold, it is judged that there is significant inflammatory infiltration.

[0045] Furthermore, a comprehensive diagnostic classification of the entire organizational region is conducted, and decision-making integration is carried out through the following rules:

[0046] The average stromal reactivity intensity of the entire image is calculated based on the first feature set. If all epithelial cell nests are identified as carcinoma in situ components, and the average stromal reactivity intensity of the entire image calculated based on the first feature set exceeds the preset stromal reactivity threshold, then the comprehensive diagnosis classification is ductal carcinoma in situ with significant stromal reaction; if only all epithelial cell nests are identified as carcinoma in situ components, then the comprehensive diagnosis classification is ductal carcinoma in situ.

[0047] The calculation logic for the overall average interstitial reactive feature intensity is as follows: For all identified epithelial cell nests in the panoramic morphological image, extract two sub-feature values ​​from the reactive features of the surrounding interstitium: fibroblast density and inflammatory cell infiltration density. Calculate the arithmetic mean of the two sub-feature values ​​in all epithelial cell nests, which are then used as the global fibroblast reactive intensity and inflammatory response intensity. The fibroblast reactive intensity and inflammatory response intensity are then weighted and summed using a preset importance coefficient to obtain the overall average interstitial reactive feature intensity.

[0048] If at least one epithelial cell nest is identified as an invasive carcinoma component, a microinvasive assessment is initiated. This involves: calculating the equivalent diameter of all epithelial cell nests identified as invasive carcinoma components; if the largest equivalent diameter does not exceed 1 mm, the overall diagnosis is classified as ductal carcinoma in situ with microinvasiveness; if the equivalent diameter of any invasive carcinoma component exceeds 1 mm, the overall diagnosis is classified as invasive ductal carcinoma. The equivalent diameter is defined as the diameter of a circle with the same pixel area as the epithelial cell nest in the panoramic morphological image.

[0049] The visualization result is specifically a transparent overlay layer with the same size as the panoramic image. In this layer: all epithelial cell nest areas identified as carcinoma in situ are filled with a first preset color in a semi-transparent way; all epithelial cell nest areas identified as invasive carcinoma components are filled with a second preset color in a semi-transparent way; and at the same time, in the instantiated local tissue atlas, the boundary lines of the ductal structure outline nodes are outlined and overlaid with a third preset color.

[0050] This invention also provides a pathological image-assisted diagnostic model construction and image classification system for multi-source heterogeneous data fusion. This system is used to implement the aforementioned pathological image-assisted diagnostic model construction and image classification method for multi-source heterogeneous data fusion, and includes:

[0051] The multi-source data acquisition and registration module is used to acquire panoramic morphological images, molecular identity images, and tissue structure knowledge graphs of the same breast tissue specimen, and to perform pixel-level spatial registration of the panoramic morphological images and molecular identity images.

[0052] The multimodal feature and semantic layer extraction module is used to identify epithelial cell nests in panoramic morphological images and extract the first feature set; it identifies and segments regions expressing myoepithelial cell markers from molecular identity images and generates a binary myoepithelial distribution semantic layer.

[0053] The structural parsing and atlas instantiation module is used to segment and identify key structural components on panoramic morphological images; it maps the identified structural components to corresponding nodes in the tissue structure knowledge graph, thus constructing an instantiated local tissue atlas for the panoramic morphological image.

[0054] The encapsulation state and context analysis module is used to perform feature analysis and contradiction resolution for each identified epithelial cell nest. The feature analysis refers to determining the spatial relationship of the epithelial cell nest in the myoepithelial distribution semantic layer and quantifying its encapsulation state, while determining its context position in the instantiated local tissue atlas. The contradiction resolution determines whether the epithelial cell nest is an in situ carcinoma component or an invasive carcinoma component based on the feature analysis results.

[0055] The comprehensive diagnosis and visualization output module is used to perform comprehensive diagnosis and classification of the entire tissue region represented by the panoramic morphological image based on the results of feature analysis and contradiction resolution, and generate a visualization result map; the comprehensive diagnosis and classification includes at least: ductal carcinoma in situ, ductal carcinoma in situ with significant stromal reaction, invasive ductal carcinoma, and ductal carcinoma in situ with microinvasiveness.

[0056] The technical effects and advantages provided by the present invention in the above technical solution are as follows:

[0057] The core of this invention lies in solving the problems of information silos and cognitive fragmentation by deeply integrating morphological, molecular identity, and anatomical prior knowledge within a unified spatial framework for interpretable reasoning. Specifically, this invention, through pixel-level spatial registration, instantiated local tissue atlas construction, and rule-based feature analysis, enables the model to precisely quantify the myoepithelial encapsulation integrity of each epithelial cell nest, and understand its spatial relationship with surrounding fat, stroma, and other normal anatomical structures, much like a seasoned pathologist. This deep integration mechanism significantly improves the objectivity and accuracy of differential diagnosis, particularly in capturing minute infiltrates that are easily missed by morphology alone, or clarifying complex cases caused by strong interstitial reactions. Furthermore, the model's final output not only provides classification results but also transparent decision-making criteria (such as visualized encapsulation status and spatial location labels), which significantly enhances the credibility of the assisted diagnostic results, achieving a crucial leap from black-box prediction to white-box decision support. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0059] Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0061] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0062] Example:

[0063] Please see Figure 1 The present invention provides a technical solution:

[0064] A method for constructing a pathological image-assisted diagnostic model and classifying images for multi-source heterogeneous data fusion, comprising the following steps:

[0065] Step 1: Obtain panoramic morphological images, molecular identity images, and tissue structure knowledge graphs of the same breast tissue specimen, and perform pixel-level spatial registration on the panoramic morphological images and molecular identity images.

[0066] In this embodiment, the panoramic morphological image is a digital image of a whole slide obtained from conventional hematoxylin and eosin (H&E) staining. This image is obtained by scanning the stained slide using a whole-slide scanner (e.g., Aperio AT2, Hamamatsu NanoZoomer, etc.) to capture the overall morphology, structure, and staining characteristics of the tissue and cells at high resolution (e.g., each pixel corresponds to a 0.25 μm × 0.25 μm tissue area). H&E staining is the gold standard for pathological diagnosis, clearly displaying morphological details such as the cell nucleus (blue), cytoplasm, and stroma (red).

[0067] Considering that the original scanned images may have issues such as uneven brightness and color shifts due to the optical characteristics of the scanner and differences in dyeing batches, the first acquired panoramic digital image needs to be preprocessed. Preprocessing operations include, but are not limited to:

[0068] Color normalization: Using statistical methods (such as the Reinhard method) or deep learning models (such as CycleGAN), the color distribution of the image is mapped to the color space of a standard reference image to reduce color differences caused by different staining batches or scanners, ensuring the stability of subsequent feature extraction. For example, an H&E slide image with suitable color, confirmed by a senior pathologist, can be selected as a reference template, and the color statistical characteristics (mean, standard deviation) of all input images can be adjusted to be consistent with the template.

[0069] Image denoising and enhancement: Non-local mean denoising or wavelet threshold denoising algorithms can be applied to suppress Gaussian noise or speckle noise introduced during the scanning process. At the same time, contrast-limited adaptive histogram equalization technology is used to enhance the contrast of tissue structures in local areas, making cell boundaries and interstitial texture clearer, which is convenient for subsequent segmentation operations.

[0070] The molecular identification image is a digital image obtained from a continuous or adjacent section (typically 4-5 micrometers thick) of the H&E-stained section using immunohistochemical staining. This immunohistochemical staining includes at least markers targeting myoepithelial cells, such as p63 protein (nuclear marker) or Calponin protein (cytoplasmic marker). These markers are specifically stained (typically with brown DAB staining) to precisely locate the presence and distribution of myoepithelial cells in the tissue space. The image is also acquired using a whole-section scanner.

[0071] Preprocessing of molecular identity images is equally crucial, and specifically includes:

[0072] Positive signal separation: Immunohistochemical staining includes not only specific positive signals (DAB staining) but also the hematoxylin-stained background of cell nuclei. Preprocessing requires a color deconvolution algorithm (e.g., the Ruifrok & Johnston method) to separate the digital image into independent DAB and hematoxylin channels. Specifically, this method uses a pre-defined or estimated staining vector matrix from the image to decompose the RGB color space of the image into optical density spaces corresponding to each staining agent, thereby obtaining a grayscale image containing only brown positive signals for subsequent quantitative analysis.

[0073] Background correction: For areas with uneven staining or non-specific staining, background correction can be performed using morphological opening operations or thresholding methods to more accurately extract the true positive signal areas.

[0074] The tissue structure knowledge graph is not extracted directly from images, but is a predefined graphical model based on the histological anatomy of the breast, used to describe the topological relationships of the normal ductal lobule system of the breast. This graph constitutes a priori, standardized spatial relationship framework.

[0075] Node Definition: Nodes in the atlas represent different types of histological structures, including at least terminal ducts, lobular acini, large ducts, fibrous stroma, and adipose tissue. Each node type is associated with a series of morphological and textural features.

[0076] Edge definition: Edges in a graph are used to define the inherent, biologically consistent spatial connections between different types of nodes. For example, there is a connection between terminal duct nodes and lobular acinar nodes; ductal nodes (terminal ducts, large ducts) are usually surrounded or adjacent to fibrous stroma. These relationships can be undirected or directed and can be weighted to represent the strength or probability of the connection.

[0077] To interpret morphological images at the molecular level, precise alignment of molecular identity images with panoramic morphological images at the subcellular level is essential. Since both images originate from continuous sections and exhibit highly similar but not identical tissue morphologies, a non-rigid spatial transformation is required. Pixel-level spatial registration involves selecting several sets of corresponding, at least 10 to 20 pairs of syngeneic feature points distributed across different tissue regions in both the panoramic morphological and molecular identity images. These syngeneic feature points are selected from clearly identifiable and stably positioned anatomical or morphological structures on both slices, such as: intersections or contour inflections of blood vessels, corners or centers of specific glandular structures, edge features of large fat vacuoles, and intersections of prominent mesenchymal fiber bundles. To ensure accuracy, a coarse selection can be performed first on low-resolution previews of both images, followed by precise localization in a high-resolution view of the selected areas. Automatic selection can be achieved using scale-invariant feature transformations, accelerated robust features, or deep learning-based feature matching networks.

[0078] Based on this set of syntagmatic feature points, a spatial transformation model is calculated to map the coordinate system of the molecular identity image to the coordinate system of the panoramic morphological image. Different models can be selected depending on the complexity of the tissue deformation. Specifically:

[0079] Affine transformation: Suitable for cases where there are only overall deformations between two slices, such as translation, rotation, scaling, and shearing. Its transformation matrix has 6 degrees of freedom and can be solved using the least squares method to minimize the mean square error between corresponding points after the transformation. For example, for a pair of matching points... and The affine transformation relationship is:

[0080]

[0081] Where a, b, c, d, e, f are the parameters to be determined.

[0082] Elastic transformation / non-rigid transformation: Since tissue sectioning, fixation, and staining processes may introduce local nonlinear deformations (such as wrinkling and stretching), elastic transformation models are preferred. Commonly used algorithms include:

[0083] A B-spline-based free deformation model is proposed: a locally smooth deformation field is generated by overlaying a uniform grid of control points on an image and moving these control points. The transformation is typically solved by maximizing the mutual information between two images or minimizing a penalty term for the distance between feature points.

[0084] Thin-plate spline transformation (TPS): This is an interpolation method based on radial basis functions that accurately matches all feature point pairs and produces the smoothest deformation field globally. Its transformation formula can be expressed as an affine transformation with the addition of a curvature term based on feature point distances. For pathological image registration with local nonlinear deformations, TPS is one of the preferred implementation methods.

[0085] After obtaining the spatial transformation model, it is applied to the molecular identity image (as a transformed floating image). Using resampling algorithms such as bilinear interpolation or cubic spline interpolation, the new position and pixel value of each pixel in the molecular identity image in the panoramic morphological image coordinate system are calculated, generating a resampled molecular identity image spatially aligned with the panoramic morphological image. To verify the registration accuracy, the average Euclidean distance (bullseye error) of the registered corresponding feature point pairs can be calculated. Typically, for high-resolution pathological images, this error should be less than 10-20 pixels (approximately 2.5-5 micrometers) to achieve cell-level alignment accuracy. If the error is too large, it is necessary to re-examine the accuracy of the feature point selection or try a more complex transformation model.

[0086] Step 2: Identify epithelial cell nests in the panoramic morphology image and extract the first feature set; identify and segment the regions expressing myoepithelial cell markers from the molecular identity image to generate a binary myoepithelial distribution semantic layer.

[0087] Input the panoramic morphological image and molecular identity image after preprocessing and spatial registration in step 1. Based on this, to perform efficient and accurate feature extraction, the following task-oriented preprocessing steps are executed:

[0088] To improve computational efficiency, a lightweight tissue region semantic segmentation model based on the Otsu global thresholding method or a pre-trained model is used to quickly identify and generate binary masks of tissue regions on panoramic morphological images. Subsequent identification and feature extraction of all epithelial cell nests will be limited to this masked area, significantly reducing computation on blank background regions.

[0089] Positive signal channel extraction from the molecular identity image (a crucial preliminary step in generating the myoepithelial distribution semantic layer). To accurately quantify the expression of myoepithelial markers, color deconvolution is first applied to the registered molecular identity image. Specifically, using the Ruifrok & Johnston method, pre-defined staining vectors corresponding to DAB (brown) and Hematoxylin (blue) are used to decompose the RGB image into independent DAB channel grayscale images and Hematoxylin channel grayscale images. The grayscale intensity of the DAB channel image directly reflects the density and distribution of positive signals and will serve as the main input for generating the myoepithelial distribution semantic layer. This step effectively removes the nuclear counterstain background, making the positive signals more prominent.

[0090] In this embodiment, the epithelial cell nests are obtained by processing the registered panoramic morphological image using a pre-trained segmentation model. This segmentation model is a deep learning-based instance segmentation model employing the Mask R-CNN architecture. Its input is a normalized RGB panoramic morphological image with a fixed input size of 1024×1024 pixels, corresponding to a tissue region of 0.5mm×0.5mm (resolution 0.5). / pixel). The model structure consists of the following parts: First, a feature extraction backbone network, using a ResNet-50 pre-trained on ImageNet, containing five stages of convolutional layers and residual connections, outputting multi-scale feature maps; next, a Region Proposal Network (RPN), which slides across the feature maps output by the backbone network to generate candidate regions and outputs the class score and bounding box offset for each candidate region; then, a region alignment layer, mapping each candidate region to a fixed-size (14×14) feature network; finally, two parallel output heads: a classification and bounding box regression head, consisting of two fully connected layers, outputting the class and precise bounding boxes; and a mask segmentation head, using a small fully convolutional network (FCN), outputting a 28×28 binary mask for each candidate region. During training, the model optimizes the multi-task loss function end-to-end, including the classification and regression loss of the RPN, the classification and regression loss of the detection head, and the binary cross-entropy loss of the mask head. The entire model is trained using a large number of panoramic morphological images labeled with epithelial cell nests, ultimately outputting a pixel-level mask and confidence score for each epithelial cell nest. Besides the preferred Mask R-CNN architecture, advanced deep learning architectures such as Cascade Mask R-CNN or Hybrid Task Cascade can also be used. During training, data augmentation techniques (such as random rotation, flipping, and color jitter) are employed to improve the model's robustness.

[0091] The preprocessed panoramic morphology image is input into the model, which outputs multiple candidate epithelial cell nest regions. Each region is accompanied by a binary mask (identifying the precise pixel location of the cell nest) and a confidence score. Low-confidence false detection regions can be filtered out by setting a confidence threshold (e.g., 0.7), ultimately yielding a series of reliable epithelial cell nest segmentation results.

[0092] The confidence threshold here is determined by evaluating model performance on an independent validation set. A precision-recall curve is plotted, and a balance point on the curve is chosen based on the needs of the diagnostic task. For example, if the goal is to minimize the loss of true epithelial cell nests (high recall), the threshold can be appropriately lowered (e.g., 0.5); if very precise identification is required (high precision), the threshold should be increased (e.g., 0.8). Typically, a threshold that maximizes the F1 score (the harmonic mean of precision and recall) is chosen as the preset value.

[0093] In this embodiment, the first feature set includes at least the morphological contour features of epithelial cell nests, nuclear polymorphism features, and reactivity features of the surrounding stroma.

[0094] The morphological contour features of epithelial cell nests include the irregularity of the outer contour, the face-to-circumference ratio, and the depth of the convex hull defect. Irregularity: This is usually calculated using the compactness of the shape or its variations, and the formula is: Irregularity = ( (4π * area) / (4π * area). A perfect circle has an irregularity value of 1; the larger the value, the more irregular the outline. For example, an epithelial cell nest with an area of ​​1000 pixels and a perimeter of 200 pixels has an irregularity of approximately 3.18, reflecting its relatively irregular outline.

[0095] The area-to-perimeter ratio is the ratio of area to perimeter. The smaller this value, the smaller the area enclosed by a unit perimeter, and the more elongated or complex the shape or edges.

[0096] Convex hull defect depth: Calculates the difference between the epithelial cell nest contour and its convex hull (the smallest convex polygon containing the contour). The depth of all recessed regions is statistically analyzed, and the maximum or average value is calculated to quantify the degree of contour indentation. A larger depth value indicates a deeper gap or lobulation in the contour.

[0097] Nuclear polymorphism features are obtained by image segmentation and measurement of nuclei within the epithelial cell nest region. Specifically, these features include the average area of ​​all segmented nuclei, the aspect ratio of each nucleus, and the uniformity of chromatin texture. Within the epithelial cell nest region, deep learning-based nuclear segmentation models or traditional algorithms based on color deconvolution and watersheds can be used to segment hematoxylin-stained nuclei individually.

[0098] The average area is the average area of ​​all the divided cell nuclei.

[0099] The major-minor axis ratio is calculated by fitting an ellipse to each cell nucleus, taking the ratio of its major axis to its minor axis, and then averaging the major-minor axis ratios of all cell nuclei. A value greater than 1 and the greater the deviation from 1, the more elongated the cell nucleus.

[0100] The uniformity of chromatin texture is determined by calculating the average of texture features such as contrast or energy in the gray-level co-occurrence matrix within the cell nucleus region. For example, a low energy value indicates a coarse and uneven texture, which may suggest abnormal chromatin distribution. For nuclear texture analysis, fixed, empirical parameter settings are typically used. For instance, when calculating the gray-level co-occurrence matrix, the pixel-to-pixel distance is often set to 1 (to analyze the relationship between adjacent pixels), and the orientation is averaged across four directions: 0°, 45°, 90°, and 135°, to obtain a rotation-invariant texture description.

[0101] The reactivity characteristics of the surrounding stroma are obtained by analyzing images of the stroma region adjacent to the boundary of the epithelial cell nest, specifically including the density of fibroblasts and the infiltration density of inflammatory cells.

[0102] The mesenchymal region refers to the area extending outward from the outline of the epithelial cell nests using a morphological dilation operation (e.g., using a circular structuring element with a radius of 50 pixels), creating a ring-shaped peripheral mesenchymal region, which is then subtracted from the original epithelial cell nest region itself. If morphological dilation is used, its radius should be determined based on the histological spatial scale, typically set by analyzing the typical extension distance of reactive mesenchyme in a large number of training samples. For example, when evaluating mesenchymal response, the region within 50-100 micrometers immediately adjacent to the epithelial boundary is usually observed. Depending on the image's pixel resolution (e.g., 0.5 micrometers / pixel), this physical distance can be converted to a pixel distance (e.g., 100 micrometers corresponds to 200 pixels). A reasonable range is 100 to 300 pixels.

[0103] In the peripheral stroma region, fibroblasts are identified by utilizing their unique elongated spindle-shaped morphology and staining characteristics, either by training a lightweight semantic segmentation model or by using morphological filtering combined with directional analysis. Fibroblast density = total number of identified fibroblast nuclei / area of ​​the peripheral stroma region.

[0104] Inflammatory cells (such as lymphocytes) typically appear as small, round, deeply stained nuclei. They can be counted using a pre-trained cell detector (such as Faster R-CNN) or after thresholding in a color space. The density of inflammatory cell infiltration is calculated similarly to that of fibroblasts.

[0105] The morphology (nuclear staining p63 vs. cytoplasmic / membrane staining Calponin) and intensity differences of positive immunohistochemical staining signals determine the choice of method. For example, threshold-based segmentation methods are suitable for situations with high staining contrast and a clean background. For instance, for Calponin (cytoplasmic staining), the image can be converted to HSV or Lab color space, and a fixed threshold or Otsu adaptive threshold can be applied to the channels representing brown (such as the b channel) for segmentation.

[0106] Pre-trained deep learning semantic segmentation models are more robust and preferred methods, better able to distinguish between specific coloring, non-specific background staining, and tissue artifacts. For example, a deep learning semantic segmentation model for identifying positive signals in molecular identity images uses the U-Net++ architecture. The input is a single-channel DAB grayscale image obtained after color deconvolution, with an input size of 512×512 pixels. The model structure is an encoder-decoder form. The encoder part contains four downsampling stages, each consisting of two 3×3 convolutional layers, a ReLU activation function, and a 2×2 max-pooling layer, progressively extracting features and reducing spatial resolution. The decoder part contains four upsampling stages. Each stage upsamples through bilinear interpolation and then performs a skip connection with the feature map of the corresponding layer in the encoder, followed by feature fusion through two 3×3 convolutional layers. Finally, the output layer uses a 1×1 convolution and a sigmoid activation function to output a single-channel probability map of the same size as the input, where each pixel value represents the probability of a positive signal at that location. The model is trained using a weighted sum of Dice loss and binary cross-entropy loss, with Adam as the optimizer and an initial learning rate of 1e-4. Training data consists of immunohistochemical images and their pixel-level positive region annotations. Data augmentation includes random rotation, flipping, and brightness / contrast adjustment. During training, the training set is divided into training, validation, and test sets. After initial training, the model's evaluation metrics on the independent test set are: Dice coefficient 0.86, precision 0.88, recall 0.87, and F1 score 0.86. At this point, all evaluation metrics are above 0.85, indicating successful training.

[0107] For p63: The model is trained to recognize small, round, strongly stained cell nuclei. Training data must be labeled with positive signals at the nuclear level.

[0108] For Calponin: The model is trained to recognize continuous or discontinuous brownish cytoplasmic / membrane staining surrounding epithelial cells. Training data must be labeled with positive regions (accuracy to the individual cell level is not required).

[0109] The above method identifies positive signal regions in molecular identity images that are colored by immunohistochemical staining. The pixel values ​​of the positive signal regions in the molecular identity image are set to a first value (e.g., 255, representing white), and the pixel values ​​of the non-positive signal regions are set to a second value (e.g., 0, representing black). This generates a binary myoepithelial distribution semantic layer aligned with the panoramic morphological image space. In this layer, white pixels indicate the location of myoepithelial cells.

[0110] Step 3: Segment and identify key structural components on the panoramic morphological image; map the identified structural components to the corresponding nodes of the organizational knowledge graph to construct an instantiated local organizational graph for the panoramic morphological image.

[0111] In this embodiment, key structural components are segmented and identified on the panoramic morphological image, specifically including:

[0112] A pre-trained deep convolutional neural network (DCNN) semantic segmentation model is used to perform pixel-level classification of panoramic morphological images. This model employs the DeepLabv3+ architecture, taking a 512×512 pixel panoramic morphological image as input after registration and preprocessing in previous steps. The backbone network uses Xception-65 pre-trained on ImageNet, outputting multi-scale features. The ASPP module uses four dilated convolutions with different dilation rates in parallel to extract multi-scale contextual information, which is then fused using 1×1 convolutions. The decoder concatenates the ASPP output features with the shallow features from the backbone network, followed by a series of 3×3 convolutions and upsampling operations to gradually restore spatial details. The final output layer is a 5-channel convolutional layer corresponding to five categories (epithelial region, fibrous stroma region, fat region, ductal structure contour region, and background region), outputting the category probability for each pixel using a Softmax activation function. The model is trained using a combination of weighted cross-entropy loss and Dice loss, with the weights dynamically adjusted based on category frequency. The training data consists of pixel-level annotated panoramic images, with the annotation categories conforming to the five tissue components mentioned above. During training, enhancement strategies such as random cropping, color dithering, and elastic deformation are employed to improve model robustness. In this embodiment, the annotation of the panoramic images was completed by three professionals, with a category annotation consistency of 0.9. During training, the SGD optimizer is used with a momentum of 0.9, an initial learning rate of 0.01, a multinomial learning decay strategy, a batch size of 6, and a training period of 80 epochs. The loss function is a combination of weighted cross-entropy loss (category weights set according to the reciprocal of frequency) and Dice loss (weight ratio 2:1).

[0113] Accurate determination of ductal structural contours based on molecular evidence is one of the key points of this step. Because relying solely on morphological features sometimes makes it difficult to distinguish between true ductal intraepithelium (which should be enveloped by myoepithelial cells) and invasive cancer nests that resemble ducts but have lost their myoepithelial cells. Therefore, this invention creatively introduces a semantic layer of myoepithelial distribution generated in step 2 as a basis for biological verification.

[0114] Epithelial regions identified by the deep convolutional neural network semantic segmentation model are screened, and those meeting the following criteria are classified as ductal structure contour regions: The percentage of contour pixels of the epithelial region belonging to positive signal regions in the myoepithelial distribution semantic layer is calculated, i.e., the proportion of contour points with a pixel value of 1 to the total number of contour points in the epithelial region. If this percentage exceeds a preset first threshold, the epithelial region is determined to be enveloped by myoepithelial tissue, conforming to the histological characteristics of a normal duct or intraductal lesion, and is therefore reclassified as a ductal structure contour region. If the percentage does not exceed the first threshold, the initial classification of the epithelial region is retained, implying that it may be an abnormal structure not completely enveloped by myoepithelial tissue (such as an invasive nest or a bridging region in some high-grade ductal carcinoma in situ).

[0115] The first proportion threshold mentioned here is not a fixed value, but should be determined experimentally based on the colorimetric characteristics, staining intensity, and sensitivity of the segmentation algorithm used in the immunohistochemical staining. A reasonable method is to collect a set of samples with known pathological diagnoses (clearly benign ductal carcinoma or typical ductal carcinoma in situ), extract the contours of typical ductal structures in each sample, and calculate the proportions as described above, statistically analyzing the proportion distribution of all typical ductal structures. The lower bound of this distribution (e.g., the 5th percentile or the mean minus two standard deviations) is taken as the initial value of the first proportion threshold. Subsequently, this threshold is fine-tuned on an independent validation set to maximize the correct identification rate of ductal structures while minimizing the probability of misclassifying invasive nests as ductal structures. Typically, this threshold can be set between 60% and 80%. For example, for p63 (nuclear staining) markers, the threshold can be set higher (e.g., 75%) due to the clear signal localization; for Calponin (cytoplasmic staining), the threshold may be appropriately lowered (e.g., 65%) due to slightly poorer signal continuity.

[0116] After accurately identifying and classifying the tissue components, the identified tissue components are mapped to the corresponding nodes in the tissue structure knowledge graph, and spatial relationships (edges) between nodes are established, thereby generating an instantiated graph unique to the current slice.

[0117] Specifically, connected regions identified as ductal structure outlines in the pixel-level classification label image are mapped to terminal or large duct nodes in the tissue structure knowledge graph. The distinction between terminal and large duct nodes can be based on geometric features such as the area and aspect ratio of the connected region, with thresholds set accordingly. Fiber-stromal regions are mapped to fiber-stromal nodes. Typically, a large, spatially continuous stroma region can be considered as a single node, or it can be divided according to a specific network. Adipose regions are mapped to adipose tissue nodes.

[0118] The core of defining this spatial relationship is defining the normal adjacency between the ductal structure and the surrounding stroma / fat. For each ductal structure contour node, the following calculations are performed:

[0119] Calculate the geometric center of the ductal profile. Search in space for the nearest fibrous mesenchymal and adipose tissue nodes to this geometric center. Distance calculations typically use Euclidean distance. Calculate the distances from the ductal node to these two nearest nodes, respectively.

[0120] Each potential adjacent edge (such as a duct-interstitial edge) is assigned an attribute value to represent the tightness of the spatial relationship. One specific implementation is to define an adjacency strength coefficient. For example, this can be quantified using the reciprocal of the distance or a Gaussian decay function: Adjacency Strength = Where d is the distance, The scale parameter controls the decay rate. When the distance d is less than a preset adjacency distance threshold (e.g., 200 micrometers, equivalent to the width of a few cells), a valid spatial adjacency relationship is considered to exist, and the calculated adjacency strength is used as an attribute of that edge. In this way, the edges of the graph not only represent connectivity relationships but also contain quantified information about spatial proximity.

[0121] For example, suppose a DCNN identifies an epithelial region with a total of 1000 contour points. In the myoepithelial distribution semantic layer, 720 contour point locations are displayed as positive signals (value = 1). Then calculate:

[0122] The proportion of positive contour points is 720 / 1000 = 72%. If the first proportion threshold is set to 70%, then 72% > 70%, therefore this region is determined to be the contour region of the duct structure. This region is mapped to a duct node. Its geometric center coordinates are calculated. Assuming the nearest interfibrous medial node center is found to be... Then, calculate their Euclidean distance. If the calculated Euclidean distance is equal to 150 micrometers, and the preset adjacency distance threshold is 200 micrometers, then the duct node is determined to be adjacent to the interstitial fiber node. Setting the scale parameter to 100 micrometers, the critical strength is approximately 0.325, and this value is used as the edge attribute connecting the two nodes.

[0123] Step 4: For each identified epithelial cell nest, perform feature analysis and contradiction resolution; the feature analysis refers to determining the spatial relationship of the epithelial cell nest in the myoepithelial distribution semantic layer and quantifying its encapsulation state, while determining its contextual position in the instantiated local tissue atlas; the contradiction resolution, based on the feature analysis results, determines whether the epithelial cell nest is an in situ carcinoma component or an invasive carcinoma component.

[0124] In this embodiment, quantifying the encapsulation state of the epithelial cell nest specifically includes:

[0125] Encapsulation state analysis aims to quantify the basement membrane integrity of target epithelial cell nests, based on the biological basis that an intact myoepithelial cell layer is a key marker for distinguishing intraductal lesions (carcinoma in situ) from invasive carcinoma.

[0126] The percentage of continuous length is a core indicator for measuring the integrity of the package. It is defined as the percentage of the total length of the line segments continuously covered by the myoepithelial positive signal on the boundary of the epithelial cell nest outline, relative to the entire perimeter of the outline. In this embodiment, the pixel-level outline C of the target epithelial cell nest obtained in step 2 consists of a series of ordered pixels. The structure is defined as follows: where i is the pixel index and n is the total number of pixels. In the myoepithelial distribution semantic layer, the query is performed on each pixel on the contour C. corresponding value Traversal arrive Identify all sequences of pixels with consecutive values ​​of 1. For example, if the contour sampling result is [0,1,1,1,0,0,1,1,0], then there are two consecutive positive segments with lengths of 3 and 2 pixels respectively. Calculate the physical length of each consecutive positive segment in the actual image space (which can be converted from pixel size). Add the physical lengths of all these consecutive positive segments to obtain the total continuous coverage length. Calculate the entire physical perimeter of contour C. Continuous length percentage = (total continuous coverage length / total physical perimeter) * 100%. For example, assuming the perimeter of a cell nest contour is 200 micrometers, and the total length of the portion continuously covered by myoepithelial positive signals is 150 micrometers, then its continuous length percentage = (150 / 200) * 100% = 75%. Scattered, isolated positive points are not included in the continuous coverage length because pathologically, interruptions in the continuity of the myoepithelial layer are emphasized as a more reliable indication of infiltration.

[0127] Based on the calculated percentage of continuous length, the package status is divided into three categories:

[0128] Completely packaged: The percentage of continuous length is greater than the first threshold, indicating that the myoepithelial layer is basically continuous and intact.

[0129] Incomplete wrapping: Percentage of continuous length ∈ [second threshold, first threshold] indicates that there is a clear interruption in the continuity of the myoepithelial layer, but it is not completely missing.

[0130] Unwrapped: The percentage of continuous length is less than the second threshold, indicating that the myoepithelial layer is basically missing or only sporadic fragments remain.

[0131] The values ​​of the first and second thresholds are not fixed and should be determined through statistical optimization based on a large-scale training dataset with consistent annotations by pathology experts. A typical method for determining these thresholds is as follows:

[0132] A large number of samples with confirmed diagnoses (including areas of carcinoma in situ and invasive carcinoma) were collected. Pathologists precisely labeled the cell nest outlines on the registered images, identifying areas considered to be completely myoepithelially enclosed (corresponding to carcinoma in situ) and areas with clearly missing myoepithelial tissue (corresponding to invasive carcinoma). The percentage values ​​of continuous length for all these labeled areas were calculated, and a distribution histogram was plotted. The first threshold was selected near the lower limit of the percentage distribution of continuous length for the completely enclosed group (e.g., the 5th percentile) to ensure high specificity. For example, if the percentage values ​​of continuous length for the completely enclosed group were mainly concentrated above 85%, the first threshold could be set to 85%. The second threshold was selected near the upper limit of the percentage distribution of continuous length for the clearly missing group (e.g., the 95th percentile) to ensure high sensitivity. For example, if the percentage values ​​of continuous length for the clearly missing group were mainly concentrated below 20%, the second threshold could be set to 20%. The interval [20%, 85%] was defined as the ambiguous region of incomplete enclosure. This thresholding method based on data distribution maximizes the fit to expert consensus and makes the model's decisions statistically reasonable.

[0133] The contextual position of the epithelial cell nest in the instantiated local tissue atlas is determined by judging the spatial relationship between the epithelial cell nest and the nodes in the instantiated local tissue atlas. The specific method for determining this is as follows:

[0134] The spatial inclusion relationship between the geometric contour of the epithelial cell nest and the polygonal regions defined by all ductal structure contour nodes in the instantiated local tissue atlas is determined. Specifically, this can be achieved by calculating the inclusion relationship between polygons and points (such as using the ray method). If all pixels of the epithelial cell nest are located within a certain polygonal region, it is determined that it is completely within the ductal structure contour node, and its context location label is determined to be inside the duct.

[0135] Conversely, an isolation determination is performed: the Euclidean distance from the geometric center of the epithelial cell nest to the nearest fibrous mesenchymal node and adipose tissue node is calculated. If the Euclidean distance to the nearest adipose tissue node is less than the Euclidean distance to the nearest fibrous mesenchymal node, and less than a preset isolation distance threshold, then an intra-adipose isolated context location label is assigned to it; otherwise, an intra-messenchymal suspected context location label is assigned to it.

[0136] The isolation distance threshold is used to define whether a cell nest can be considered isolated outside the original ductal unit. Its value should reflect the spatial perception of microinvasives or isolated cell clusters in pathology. It can be determined empirically; based on common pathological knowledge, isolated epithelial cell clusters located more than 1-2 high-power fields (approximately 0.5-1.0 mm) from the original ductal boundary are generally considered suspicious invasive foci. Therefore, a reasonable empirical initial value for the isolation distance threshold could be set at 500 micrometers (0.5 mm). A more scientific approach is to have pathologists label recognized isolated invasive cell nests within the stroma and budding or pseudo-invasive cell nests still belonging to intraductal lesions on the training dataset, and measure their distances to the nearest normal ductal boundary. The isolation distance threshold can be taken as the boundary value between these two distance distributions, ultimately determined by maximizing classification performance (e.g., F1 score).

[0137] The aforementioned contradiction analysis refers to determining whether an epithelial cell nest is a component of carcinoma in situ or invasive carcinoma based on the results of feature analysis through a hierarchical decision process. The specific judgment rules are as follows:

[0138] An epithelial cell nest is considered an invasive carcinoma component if it meets all of the following criteria: its contextual location is labeled as isolated within fat or suspected within the stroma, and its encapsulation status is unencapsulated. This criterion is based on the strong evidence that both locational abnormalities (detachment from ductal structures) and basement membrane absence (myoepithelial absence) are present simultaneously. This rule has the highest priority and specificity.

[0139] If the epithelial cell nest does not meet the criteria for invasive carcinoma, it shall be re-evaluated according to the following sub-rules:

[0140] Sub-rule 1: The context location label is "inside the duct", the encapsulation status is "completely encapsulated", and the morphological outline features show high irregularity and / or significant nuclear polymorphism features;

[0141] Sub-rule 2: The context location label is "inside the catheter," the wrapping status is "not wrapped or incompletely wrapped," and the reactive characteristics of the surrounding stroma show significant pro-fibrotic connective tissue proliferation or inflammatory infiltration.

[0142] If the epithelial cell nest satisfies either sub-rule one or sub-rule two above, it is determined to be a carcinoma in situ component.

[0143] The high degree of irregularity refers to the irregularity of the epithelial cell nest being greater than a preset first morphological threshold, and its face-to-circumference ratio being less than a preset first proportional threshold.

[0144] The first morphology threshold was determined by collecting a large number of images of normal ducts or benign proliferative regions with typical smooth contours, confirmed by pathologists, as well as images of epithelial cell nests diagnosed as high-grade ductal carcinoma in situ, exhibiting sieve-like, micropapillary, or solid but highly irregular borders. The irregularity of the epithelial cell nests (defined as the ratio of the region area to the convex hull area) was calculated in both sets of images. The irregularity of normal regions is typically close to 1, while the irregularity of high-grade lesions is significantly reduced. Through statistical analysis (e.g., plotting distribution curves for the two sets of data), the first morphology threshold can be set between the lower limit of the irregularity distribution in the normal group (e.g., the 5th percentile) and the upper limit of the irregularity distribution in the high-grade lesion group (e.g., the 95th percentile), and further adjusted using a validation set to maximize the accuracy in identifying high irregularity. For example, the threshold determined by this method may be 0.75, which directly quantifies the morphological changes of irregular boundaries or radial disorder described in pathology, transforming subjective morphological descriptions into objective and repeatable measurement standards, and the selection of the threshold is rooted in the statistical differences between actual lesions and normal structures.

[0145] The method for determining the first ratio threshold complements the morphological threshold, but focuses on describing the complexity of the contour. The area-to-perimeter ratio, the ratio of area to the square of the perimeter, is extremely sensitive to the tortuosity of the contour. Similarly, the area-to-perimeter ratio is calculated for datasets of normal ductal contours and high-grade atypical contours. Normal ducts are typically approximately circular, with a high area-to-perimeter ratio; while complex contours have a very low area-to-perimeter ratio. This threshold should be set at a critical value that effectively distinguishes between simple and complex contours. In practice, feature selection methods from machine learning can be used to observe the importance of the area-to-perimeter ratio in classification, and grid search combined with cross-validation can be used to find the threshold point that achieves the best classification performance when used in conjunction with irregularity features to distinguish between high and low-grade lesions. For example, a possible empirical value is 0.06. The reason for this determination is that the area-to-perimeter ratio complements the shortcomings of the single indicator of irregularity, jointly defining the comprehensive morphological concept of high irregularity, ensuring that the model captures truly clinically significant abnormal structures, rather than minor morphological fluctuations.

[0146] The significant nuclear polymorphism characteristic refers to the fact that the standard deviation of the area of ​​all nuclei in the epithelial cell nest is greater than a preset first nuclear state threshold, and the average of the long and short axis ratios of all nuclei is greater than a preset first axis ratio threshold.

[0147] The first nuclear state threshold (the standard deviation threshold of nuclear area) and the first axial ratio threshold (the average threshold of the ratio of the long and short axes of the nuclear area) are both significant in quantifying nuclear polymorphism. The determination method must start at the nuclear level. First, nuclear segmentation and measurement are performed in normal ductal epithelial regions and in regions with clearly defined high-grade intraepithelial neoplasia. Normal cells have relatively uniform nuclear size and shape, with small standard deviations in area and small average axial ratios; while high-grade lesions exhibit significant polymorphism, i.e., varying sizes (large standard deviation of area) and elongated or irregular shapes (large average axial ratio). The first nuclear state threshold can be set as a baseline of 1.5 to 2 times the standard deviation of normal nuclear area, and the first axial ratio threshold can be set with an initial value of 1.8, referencing pathological common sense (the axial ratio of round nuclei is approximately 1, and significantly elongated nuclei are greater than 1.5). A more precise method is to calculate the ROC curves of these two features in distinguishing between normal and high-grade nuclei, and select the feature value corresponding to the maximum Youden's index as the threshold. The rationale for this approach is that it directly corresponds to nuclear polymorphism. By using thresholds statistically derived from a large amount of cell nuclear data, the model's assessment of nuclear polymorphism is based on objective population comparisons rather than subjective impressions.

[0148] Based on the pixel-level outline of the epithelial cell nest, a band-shaped region with a preset width is generated by morphological dilation operation (e.g., dilation radius of 50 micrometers), which only includes the outer side of the original outline. When the fibroblast density calculated in this band-shaped region is greater than the preset first cell density threshold, it is judged that there is significant fibrotic connective tissue proliferation. When the inflammatory cell density calculated in this band-shaped region is greater than the preset second cell density threshold, it is judged that there is significant inflammatory infiltration.

[0149] The determination of the first cell density threshold (used to determine fibrous connective tissue proliferation) and the second cell density threshold (used to determine inflammatory cell infiltration) requires focusing on specifically defined band-shaped regions. During the training phase, pathologists must mark areas of dense fibroblasts or inflammatory cell aggregation at the tumor-stromal junction (i.e., the band-shaped region generated by morphological expansion in this invention) in images of carcinoma in situ cases with significant stromal reactions; simultaneously, control areas are marked in normal stroma or areas containing only mild inflammation. Subsequently, the densities (number of cells per unit area) of fibroblasts and inflammatory cells within these marked regions are calculated. The first cell density threshold can be set at the high percentile (e.g., the 90th percentile) of the cell density distribution in normal / mildly reacting areas to ensure high specificity for significant reactions. For example, if the density of fibroblasts in normal stroma is typically below X cells / mm², then the threshold can be set as a multiple of X. Its rationale lies in the fact that the threshold is not to determine whether there is an interstitial reaction, but to determine whether the reaction reaches the level of a significant or prominent clinical diagnostic description. This quantitative standard helps to unify the differences in the subjective description of significance among different observers, so that the model can stably identify host response patterns with clinical significance.

[0150] Step 5: Based on the results of feature analysis and contradiction analysis, perform a comprehensive diagnostic classification of the entire tissue region represented by the panoramic morphological image and generate a visualization result map; the comprehensive diagnostic classification includes at least: ductal carcinoma in situ, ductal carcinoma in situ with significant stromal reaction, invasive ductal carcinoma, and ductal carcinoma in situ with microinvasiveness.

[0151] In this embodiment, a comprehensive diagnostic classification of the entire tissue region is performed, and decision integration is carried out through the following rules:

[0152] Based on the first feature set, the average stromal reactivity intensity of the entire image is calculated. If all epithelial cell nests are identified as carcinoma in situ components, and the average stromal reactivity intensity of the entire image calculated based on the first feature set exceeds a preset stromal reactivity threshold, then the comprehensive diagnostic classification is ductal carcinoma in situ with significant stromal reaction. If only all epithelial cell nests are identified as carcinoma in situ components, then the comprehensive diagnostic classification is ductal carcinoma in situ.

[0153] The interstitial lung reaction threshold also needs to be optimized and determined through supervised learning on a labeled training set. A specific method could be to collect a batch of cases of ductal carcinoma in situ and ductal carcinoma in situ with significant interstitial lung reaction, confirmed by pathology expert consensus, and calculate the average interstitial lung reaction feature intensity value for each case. By plotting receiver operating characteristic (ROC) curves, a critical value for the average interstitial lung reaction feature intensity value that best distinguishes these two diagnoses can be found, and this critical value is set as the interstitial lung reaction threshold. The rationale is that this threshold is the optimal decision boundary learned from clinical gold standard data, ensuring consistency between the model's diagnosis and expert diagnosis.

[0154] The calculation logic for the overall average interstitial reactive feature intensity is as follows: For all identified epithelial cell nests in the panoramic morphological image, two sub-feature values—fibroblast density and inflammatory cell infiltration density—are extracted from the reactive features of the surrounding interstitium. The arithmetic mean of these two sub-feature values ​​is calculated for all epithelial cell nests, serving as the global fibroblast reaction intensity and inflammatory response intensity. The fibroblast reaction intensity and inflammatory response intensity are then weighted and summed using preset importance coefficients to obtain the overall average interstitial reactive feature intensity. The values ​​of the two weighting coefficients should be determined based on the statistical analysis results of large-scale retrospective clinicopathological data. For example, logistic regression or machine learning feature importance analysis can be performed on a dataset of definitively diagnosed cases to assess the contribution of fibroblast reaction and inflammatory response to distinguishing ordinary carcinoma in situ from carcinoma in situ with significant interstitial reaction, and the weights can be set accordingly. In an exemplary implementation, if the analysis finds that the influence weight of fibroblast density on diagnosis is approximately twice that of inflammatory cell density, the weighting coefficient for fibroblast reaction intensity can be set to 0.67, and the weighting coefficient for inflammatory response intensity to 0.33.

[0155] If at least one epithelial cell nest is identified as an invasive carcinoma component, a microinvasive assessment is initiated. This involves: calculating the equivalent diameter of all epithelial cell nests identified as invasive carcinoma components; if the largest equivalent diameter does not exceed 1 mm, the overall diagnostic classification is ductal carcinoma in situ with microinvasiveness; if the equivalent diameter of any invasive carcinoma component exceeds 1 mm, the overall diagnostic classification is invasive ductal carcinoma. The equivalent diameter is defined as the diameter of a circle with the same pixel area as the epithelial cell nest in the panoramic morphological image. The spatial resolution of the image (e.g., micrometers per pixel) must be considered during the calculation, converting the pixel area to the actual physical area. The threshold of 1 mm given here directly adopts the currently widely used clinicopathological standards (e.g., the AJCC Cancer Staging Manual) to define the maximum upper limit of microinvasiveness. This ensures that the diagnostic criteria of this method are in line with current clinical practice guidelines and have recognized rationality and authority.

[0156] The visualization result is specifically a transparent overlay layer with the same size as the panoramic image. In this layer, all epithelial cell nests identified as carcinoma in situ components are semi-transparently filled with a first preset color (blue). This operation clearly shows the distribution range of carcinoma in situ, while allowing the morphological details of the cells underneath to be seen through the color.

[0157] Use a second preset color (red) to semi-transparently fill in all epithelial cell nests identified as invasive carcinoma components. The high contrast of red will better highlight clinically significant invasive areas.

[0158] Meanwhile, in the instantiated local tissue atlas, the boundary lines of the ductal structure outline nodes are delineated with a third preset color (green) and displayed overlaid. Using the spatial framework of the constructed normal or abnormal ductal structures as a reference helps to understand the relationship between lesions and background tissue.

[0159] In this embodiment, the generation method is as follows: In software implementation, a fully transparent RGBA image can be created as a canvas. The masks of all in situ carcinoma components are traversed, and the corresponding pixel positions are filled with (R=0, G=0, B=255, A=128) (blue, 50% transparency). Similarly, the invasive components are filled with (R=255, G=0, B=0, A=128) (red, 50% transparency). Finally, using a graphics drawing API, green lines (R=0, G=255, B=0) are drawn along the coordinate sequence of the ductal structure outline nodes. This finally generated overlay layer is then alpha-blended with the original H&E image and displayed.

[0160] Please see Figure 2 The present invention also provides a pathological image-assisted diagnostic model construction and image classification system for multi-source heterogeneous data fusion. This system is used to implement the aforementioned pathological image-assisted diagnostic model construction and image classification method for multi-source heterogeneous data fusion, and includes:

[0161] The multi-source data acquisition and registration module is used to acquire panoramic morphological images, molecular identity images, and tissue structure knowledge graphs of the same breast tissue specimen, and to perform pixel-level spatial registration of the panoramic morphological images and molecular identity images.

[0162] The multimodal feature and semantic layer extraction module is used to identify epithelial cell nests in panoramic morphological images and extract the first feature set; it identifies and segments regions expressing myoepithelial cell markers from molecular identity images and generates a binary myoepithelial distribution semantic layer.

[0163] The structural parsing and atlas instantiation module is used to segment and identify key structural components on panoramic morphological images; it maps the identified structural components to corresponding nodes in the tissue structure knowledge graph, thus constructing an instantiated local tissue atlas for the panoramic morphological image.

[0164] The encapsulation state and context analysis module is used to perform feature analysis and contradiction resolution for each identified epithelial cell nest. The feature analysis refers to determining the spatial relationship of the epithelial cell nest in the myoepithelial distribution semantic layer and quantifying its encapsulation state, while determining its context position in the instantiated local tissue atlas. The contradiction resolution determines whether the epithelial cell nest is an in situ carcinoma component or an invasive carcinoma component based on the feature analysis results.

[0165] The comprehensive diagnosis and visualization output module is used to perform comprehensive diagnosis and classification of the entire tissue region represented by the panoramic morphological image based on the results of feature analysis and contradiction resolution, and generate a visualization result map; the comprehensive diagnosis and classification includes at least: ductal carcinoma in situ, ductal carcinoma in situ with significant stromal reaction, invasive ductal carcinoma, and ductal carcinoma in situ with microinvasiveness.

[0166] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0167] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for constructing a pathological image-assisted diagnostic model and classifying images for multi-source heterogeneous data fusion, characterized in that, The specific steps include: Step 1: Obtain panoramic morphological images, molecular identity images, and tissue structure knowledge graphs of the same breast tissue specimen, and perform pixel-level spatial registration on the panoramic morphological images and molecular identity images; Step 2: Identify epithelial cell nests in the panoramic morphology image and extract the first feature set; identify and segment regions expressing myoepithelial cell markers from the molecular identity image, and generate a binary myoepithelial distribution semantic layer; Step 3: Segment and identify key structural components on the panoramic morphological image; map the identified structural components to the corresponding nodes of the tissue knowledge graph to construct an instantiated local tissue graph for the panoramic morphological image; Step 4: For each identified epithelial cell nest, perform feature analysis and contradiction resolution; the feature analysis refers to determining the spatial relationship of the epithelial cell nest in the myoepithelial distribution semantic layer and quantifying its encapsulation state, while determining its contextual position in the instantiated local tissue atlas; the contradiction resolution, based on the feature analysis results, determines whether the epithelial cell nest is an in situ carcinoma component or an invasive carcinoma component. Step 5: Based on the results of feature analysis and contradiction analysis, perform a comprehensive diagnostic classification of the entire tissue region represented by the panoramic morphological image and generate a visualization result map; the comprehensive diagnostic classification includes at least: ductal carcinoma in situ, ductal carcinoma in situ with significant stromal reaction, invasive ductal carcinoma, and ductal carcinoma in situ with microinvasiveness.

2. The method for constructing a pathological image-assisted diagnostic model and classifying images based on multi-source heterogeneous data fusion as described in claim 1, characterized in that, The panoramic morphological image is a hematoxylin-eosin stained section image; the molecular identity image refers to an immunohistochemical staining digital image of consecutive sections from the same tissue block; the immunohistochemical staining includes at least a marker for myoepithelial cells; the marker includes at least a p63 protein marker or a Calponin protein marker. The tissue structure knowledge graph is a predefined graph model based on the histological anatomy of the breast, used to describe the topological relationships of the normal ductal lobule system of the breast; the nodes of the graph model include at least terminal ducts, lobular alveoli, large ducts, fibrous stroma, and adipose tissue. Edges in a graph model are used to define the inherent spatial connections between different types of nodes; Pixel-level spatial registration includes selecting several sets of corresponding feature points distributed in different tissue regions from the panoramic morphological image and the molecular identity image; Based on the same set of feature points, an affine transformation or elastic transformation algorithm is used to calculate the spatial transformation model. The spatial transformation model is then used to resample the molecular identity image, so that it is accurately aligned with the panoramic morphological image at the cellular and tissue structure level.

3. The method for constructing a pathological image-assisted diagnostic model and classifying images based on multi-source heterogeneous data fusion as described in claim 1, characterized in that, The epithelial cell nests are obtained by processing the registered panoramic morphological image through a pre-trained segmentation model; the segmentation model is a deep learning-based instance segmentation model, which is trained on a panoramic morphological image with pre-annotated epithelial cell nest regions.

4. The method for constructing a pathological image-assisted diagnostic model and classifying images based on multi-source heterogeneous data fusion according to claim 2, characterized in that, The first feature set includes at least the morphological outline features of epithelial cell nests, nuclear polymorphism features, and reactivity features of the surrounding stroma; The morphological contour features of epithelial cell nests include the irregularity of the outer contour of the epithelial cell nest, the face-to-circumference ratio, and the depth of the convex hull defect; Nuclear polymorphism features are obtained by image segmentation and measurement of the nuclei in the epithelial cell nest region, specifically including the average area of ​​all segmented nuclei, the ratio of the major and minor axes of each nucleus, and the uniformity of chromatin texture. The reactivity characteristics of the surrounding stroma were obtained by analyzing images of the stroma region immediately adjacent to the boundary of the epithelial cell nest, specifically including the density of fibroblasts and the infiltration density of inflammatory cells; Using a threshold-based segmentation method or a pre-trained deep learning semantic segmentation model, positive signal regions that are colored by immunohistochemical staining are identified in the molecular identity image; the positive signal regions correspond to regions expressing myoepithelial cell markers; the pixel values ​​of the positive signal regions in the molecular identity image are set as the first value, and the pixel values ​​of the non-positive signal regions are set as the second value, thereby generating a binarized myoepithelial distribution semantic layer. When the molecular identity image is an immunohistochemical staining image targeting the p63 protein, the deep learning semantic segmentation model is specifically trained to recognize positive signals in the nuclear staining pattern; when the molecular identity image is an immunohistochemical staining digital image targeting the Calponin protein, the deep learning semantic segmentation model is specifically trained to recognize positive signals in the cytoplasmic staining pattern.

5. The method for constructing a pathological image-assisted diagnostic model and classifying images based on multi-source heterogeneous data fusion according to claim 4, characterized in that, Key structural components were segmented and identified from panoramic morphological images, specifically including: Segmentation and identification of structural components are performed using a pre-trained deep convolutional neural network semantic segmentation model. The input of the deep convolutional neural network semantic segmentation model is a panoramic morphological image, and its output is a pixel-level classification label map of the same size as the input image. Each pixel of the pixel-level classification label map is classified into one of the following categories: epithelial region, fibrous stroma region, fat region, ductal structure outline region, and background region. The epithelial regions identified by the deep convolutional neural network semantic segmentation model are screened, and the epithelial regions that meet the following conditions are determined as ductal structure contour regions: the proportion of the number of outer contour pixels of the epithelial region belonging to the positive signal region in the myoepithelial distribution semantic layer is counted. If the proportion of the number exceeds the preset first proportion threshold, the epithelial region is determined to be wrapped by myoepithelial tissue and the epithelial region is classified as ductal structure contour region. The identified organizational structure components are mapped to corresponding nodes in the organizational structure knowledge graph, specifically as follows: The segmented and identified ductal structure contour regions are mapped to terminal ducts or large duct nodes in the tissue structure knowledge graph; the fibrous stroma regions are mapped to fibrous stroma nodes; and the adipose regions are mapped to adipose tissue nodes. At the same time, based on the geometric center and spatial distribution of the ductal structure contour regions, their spatial connection relationships as ductal structure nodes are reconstructed, thereby generating an instantiated local tissue map. In the instantiated local tissue knowledge graph, by calculating the spatial distance and connectivity between nodes, the adjacency relationship between each ductal structure contour node and the nearest fibrous interstitial node and adipose tissue node is defined, and this adjacency relationship is quantified as the edge attribute of the graph to represent the normal spatial adjacency relationship.

6. The method for constructing a pathological image-assisted diagnostic model and classifying images based on multi-source heterogeneous data fusion according to claim 4, characterized in that, The quantification of the encapsulation state of epithelial cell nests specifically includes: The encapsulation status of epithelial cell nests includes fully encapsulated, partially encapsulated, and not encapsulated. The specific judgment logic is as follows: The algorithm calculates the percentage of continuous length by which the contour boundary of the epithelial cell nest is covered by the positive signal region in the myoepithelial distribution semantic layer. If the percentage of continuous length is higher than a first preset threshold, the epithelial cell nest is considered to be completely wrapped. If the percentage of continuous length is lower than a second preset threshold, the epithelial cell nest is considered to be unwrapped. If the percentage of continuous length is between the first and second preset thresholds, the epithelial cell nest is considered to be incompletely wrapped. The calculation logic for the percentage of continuous length is as follows: extract the pixel-level contour of the epithelial cell nest in the panoramic morphological image; sample along the pixel-level contour in the myoepithelial distribution semantic layer and record whether each sampling point is a positive signal; calculate the total length of the continuous line segment formed by all sampling points that are positive signals, and calculate its percentage of the perimeter of the entire pixel-level contour. This percentage is defined as the percentage of continuous length. The contextual position of the epithelial cell nest in the instantiated local tissue atlas is determined by judging the spatial relationship between the epithelial cell nest and the nodes in the instantiated local tissue atlas. The specific method for determining this is as follows: Spatial inclusion relationship is determined between the geometric contour of the epithelial cell nest and the polygonal region defined by all ductal structure contour nodes in the instantiated local tissue map. If all pixels of the epithelial cell nest are located within a certain polygonal region, it is determined that it is completely within the ductal structure contour node corresponding to the polygonal region, and then a contextual position label within the duct is assigned to it. Conversely, an isolation determination is performed: the Euclidean distance from the geometric center of the epithelial cell nest to the nearest fibrous mesenchymal node and adipose tissue node is calculated; if the Euclidean distance to the nearest adipose tissue node is less than the Euclidean distance to the nearest fibrous mesenchymal node and less than the preset isolation distance threshold, then an intra-adipose isolated context location label is assigned to it; otherwise, an intra-messenchymal suspected context location label is assigned to it. The aforementioned contradiction analysis refers to determining whether an epithelial cell nest is a component of carcinoma in situ or invasive carcinoma based on the results of feature analysis through a hierarchical decision process. The specific judgment rules are as follows: An epithelial cell nest is considered an invasive carcinoma component if it meets all of the following criteria: The context location label is isolated within fat or suspected within the interstitium, and the wrapping status is unwrapped; If the epithelial cell nest does not meet the criteria for invasive carcinoma, it shall be re-evaluated according to the following sub-rules: Sub-rule 1: The context location label is "inside the duct", the encapsulation status is "completely encapsulated", and the morphological outline features show high irregularity and / or significant nuclear polymorphism features; Sub-rule 2: The context location label is "inside the catheter," the wrapping status is "not wrapped or incompletely wrapped," and the reactive characteristics of the surrounding stroma show significant pro-fibrotic connective tissue proliferation or inflammatory infiltration. If the epithelial cell nest satisfies either sub-rule one or sub-rule two above, it is determined to be a carcinoma in situ component. The high irregularity refers to the irregularity of the epithelial cell nest being greater than a preset first morphological threshold, and its face-to-circumference ratio being less than a preset first proportional threshold. The significant nuclear polymorphism characteristic refers to the fact that the standard deviation of the area of ​​all nuclei in the epithelial cell nest is greater than the preset first nuclear state threshold, and the average of the long and short axis ratios of all nuclei is greater than the preset first axis ratio threshold. Based on the pixel-level outline of the epithelial cell nest, a band-shaped region with a preset width is generated by morphological dilation operation, which only includes the outer side of the original outline. When the fibroblast density calculated in this band-shaped region is greater than the preset first cell density threshold, it is judged that there is significant fibrotic connective tissue proliferation. When the inflammatory cell density calculated in this band-shaped region is greater than the preset second cell density threshold, it is judged that there is significant inflammatory infiltration.

7. The method for constructing a pathological image-assisted diagnostic model and classifying images based on multi-source heterogeneous data fusion according to claim 6, characterized in that, A comprehensive diagnostic classification of the entire organizational region is conducted, and decision-making is integrated based on the following rules: The average stromal reactivity intensity of the entire image is calculated based on the first feature set. If all epithelial cell nests are identified as carcinoma in situ components, and the average stromal reactivity intensity of the entire image calculated based on the first feature set exceeds the preset stromal reactivity threshold, then the comprehensive diagnosis classification is ductal carcinoma in situ with significant stromal reaction; if only all epithelial cell nests are identified as carcinoma in situ components, then the comprehensive diagnosis classification is ductal carcinoma in situ. The calculation logic for the overall average interstitial reactive feature intensity is as follows: For all identified epithelial cell nests in the panoramic morphological image, extract two sub-feature values ​​from the reactive features of the surrounding interstitium: fibroblast density and inflammatory cell infiltration density. Calculate the arithmetic mean of the two sub-feature values ​​in all epithelial cell nests, which are then used as the global fibroblast reactive intensity and inflammatory response intensity. The fibroblast reactive intensity and inflammatory response intensity are then weighted and summed using a preset importance coefficient to obtain the overall average interstitial reactive feature intensity. If at least one epithelial cell nest is identified as an invasive carcinoma component, a microinvasive assessment is initiated. This involves: calculating the equivalent diameter of all epithelial cell nests identified as invasive carcinoma components; if the largest equivalent diameter does not exceed 1 mm, the overall diagnosis is classified as ductal carcinoma in situ with microinvasiveness; if the equivalent diameter of any invasive carcinoma component exceeds 1 mm, the overall diagnosis is classified as invasive ductal carcinoma. The equivalent diameter is defined as the diameter of a circle with the same pixel area as the epithelial cell nest in the panoramic morphological image. The visualization result is specifically a transparent overlay layer with the same size as the panoramic image. In this layer, all epithelial cell nest areas identified as carcinoma in situ are filled with a first preset color in a semi-transparent manner. Use the second preset color to semi-transparently fill all epithelial cell nest regions identified as invasive carcinoma components; at the same time, in the instantiated local tissue atlas, outline the boundary lines of duct structure contour nodes with the third preset color and display them overlaid.

8. A pathological image-assisted diagnostic model construction and image classification system for multi-source heterogeneous data fusion, characterized in that, The pathological image-assisted diagnostic model construction and image classification system for multi-source heterogeneous data fusion is used to implement the pathological image-assisted diagnostic model construction and image classification method for multi-source heterogeneous data fusion as described in any one of claims 1-7, including: The multi-source data acquisition and registration module is used to acquire panoramic morphological images, molecular identity images, and tissue structure knowledge graphs of the same breast tissue specimen, and to perform pixel-level spatial registration of the panoramic morphological images and molecular identity images. The multimodal feature and semantic layer extraction module is used to identify epithelial cell nests in panoramic morphological images and extract the first feature set; it identifies and segments regions expressing myoepithelial cell markers from molecular identity images and generates a binary myoepithelial distribution semantic layer. The structural parsing and atlas instantiation module is used to segment and identify key structural components on panoramic morphological images; it maps the identified structural components to corresponding nodes in the tissue structure knowledge graph, thus constructing an instantiated local tissue atlas for the panoramic morphological image. The encapsulation state and context analysis module is used to perform feature analysis and contradiction resolution for each identified epithelial cell nest. The feature analysis refers to determining the spatial relationship of the epithelial cell nest in the myoepithelial distribution semantic layer and quantifying its encapsulation state, while determining its context position in the instantiated local tissue atlas. The contradiction resolution determines whether the epithelial cell nest is an in situ carcinoma component or an invasive carcinoma component based on the feature analysis results. The comprehensive diagnosis and visualization output module is used to perform comprehensive diagnosis and classification of the entire tissue region represented by the panoramic morphological image based on the results of feature analysis and contradiction resolution, and generate a visualization result map; the comprehensive diagnosis and classification includes at least: ductal carcinoma in situ, ductal carcinoma in situ with significant stromal reaction, invasive ductal carcinoma, and ductal carcinoma in situ with microinvasiveness.

Images