Liver tissue pathological image lesion feature liver area spatial distribution analysis method and system

Abstract

The application provides a liver tissue pathological image lesion feature liver area spatial distribution analysis method and system, and a hierarchical architecture is constructed based on liver tissue physiological structure logic. The system accurately segments the internal region of the analysis object, the analysis object itself and the liver lobule object of the pathological image through an object segmentation model, respectively extracts the lesion feature of the analysis object, the unit of the analysis object and the relative spatial position information of the analysis object. Further, the lesion feature and the relative spatial position information are fused to construct a liver area coordinate algorithm, the liver area coordinate calculation of each site in the whole pathological image is realized, and the subjectivity and quantitative error of the traditional method are overcome. Finally, the system statistically analyzes the lesion feature and the liver area coordinate information of the analysis object, accurately and objectively analyzes the lesion feature in the liver tissue pathological image according to the liver area spatial distribution, and solves the technical bottleneck of the whole image quantitative analysis of the liver area spatial distribution of the liver tissue pathological image.

Description

Technical Field

[0001] This invention relates to a method and system for analyzing the spatial distribution of lesion features in liver tissue pathological images, belonging to the fields of biomedical and artificial intelligence technologies. Background Technology

[0002] Liver tissue pathological assessment technology based on H&E staining histopathological images serves as a core technical support for the clinical diagnosis and treatment of liver diseases, preclinical drug toxicity evaluation, and new drug development. By directly observing the multi-scale structural lesion characteristics of liver tissue, it provides key validation evidence for derivative technologies such as biochemical index detection and imaging examinations.

[0003] The unique physiological and pathological structure of the liver determines the complexity of its disease research: the liver lobules, as the basic functional units of the liver, consist of portal areas, central veins, and radially arranged hepatocytes, forming a gradient metabolic microenvironment. This continuous gradient distribution leads to significant regional differences in metabolism, toxic reactions, pathological mechanisms, and structural lesion characteristics among the periportal vein area, the pericentral vein area, and the intermediate transition zone. Therefore, accurately quantifying and analyzing the dynamic changes in the spatial distribution of lesion features along the liver region in histopathological images can provide crucial information for revealing the mechanisms of disease occurrence and development.

[0004] However, the assessment and quantification of the spatial distribution of lesion features in liver tissue pathological images still heavily relies on manual qualitative analysis, which presents the following technical bottlenecks: First, manual interpretation is affected by the heterogeneity of complex pathological features and differences in observer experience, making it difficult to guarantee the objectivity and anatomical accuracy of liver region division. Second, two-dimensional H&E-stained histopathological images lose the three-dimensional spatial information of liver lobules and biomarker information such as genes / proteins that can assist in liver region localization, resulting in many image areas being unable to achieve accurate liver region assignment. Although bioinformatics association can be achieved by combining serial sections with H&E staining and specific protein immunolabeling, this method is complex, costly, and has a low success rate, making it difficult to achieve high-throughput assessment. More importantly, the expression and distribution of biomarkers change dynamically with the variability of liver tissue pathology, and the pattern of these changes is not yet clear, greatly limiting the applicability of this method to various lesion samples. Third, the discretized liver region classification strategy ignores the continuous characteristics of lesion distribution within liver lobules and cannot accurately capture the dynamic evolution of lesions in the gradient metabolic microenvironment.

[0005] The aforementioned technical limitations directly lead to two core problems: First, existing methods cannot establish a complete mapping relationship between the lesion features at each point and the physiological division of the liver lobules across the entire pathological image, limiting spatial resolution capabilities. Second, the lack of effective quantitative analysis tools and methods makes it difficult to accurately assess the distribution characteristics of liver lesions. This restricts the application and technological breakthroughs of spatial distribution information of liver tissue pathology in various scenarios, including disease mechanism research (such as joint analysis of pathological images and spatial omics data), personalized diagnostic model development (heterogeneous etiology inference based on spatial distribution characteristics), and the construction of innovative drug evaluation systems (high-throughput efficacy assessment, drug screening, and discovery of efficacy characteristics). Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a method and system for analyzing the spatial distribution of microscopic lesion features in liver tissue pathological images. This method calculates the coordinates of the liver region in any liver tissue pathological image. Based on these coordinates, it achieves comprehensive and accurate quantification and analysis of the spatial distribution of microscopic lesion features in the liver region at every point in the entire pathological image.

[0007] The first aspect of this invention provides a method for analyzing the spatial distribution of microscopic lesion features in liver tissue pathological images, comprising the following steps:

[0008] Segment and acquire analytical objects from pathological images;

[0009] Extract the lesion features of the analyzed object;

[0010] Segmenting liver lobule objects in pathological images to obtain the location information of liver lobule objects in pathological images;

[0011] A liver region coordinate algorithm is established, which uses the positional information of the liver lobule to calculate the liver region coordinates of each point in the pathological image, and obtains the liver region coordinate results.

[0012] The distribution of lesion features of the object with the change of liver region coordinates is statistically analyzed to obtain the spatial distribution analysis results of lesion features of pathological images with the liver region.

[0013] The physiological structure and pathological changes within liver lobules are complex, and a significant amount of structural features and locational information is lost in two-dimensional pathological images. This makes it difficult for traditional manual interpretation methods, relying solely on liver tissue H&E pathological images, to generate accurate and quantifiable liver region coordinate information. Furthermore, they cannot quantify the distribution of lesion features as a function of liver region coordinates. Current state-of-the-art methods can only roughly identify liver regions (regions 1, 2, and 3) by dividing the image, thus obtaining coarse assessment results. They still struggle to generate truly quantitative data (they cannot distinguish and quantify the lesion features of different types of analyzed objects, cannot continuously quantify liver region coordinates, and cannot continuously quantify the spatial distribution of lesion features as a function of liver regions). This makes them even more difficult to apply in complex liver tissue pathological images.

[0014] This method segmentes and calculates features within liver lobules to provide quantifiable lesion information, addressing the issues of unclear analysis objects and difficulty in quantifying lesion features. Simultaneously, by segmenting and locating liver lobules and constructing a liver region coordinate algorithm, it outputs continuous and quantifiable physiological spatial location information, solving the problem of difficulty in continuously quantifying liver region location information. Based on this, by integrating the lesion features of the analyzed objects with the liver region coordinate results, it is possible to achieve accurate, continuous, and quantifiable distribution quantification of lesion features as they change with liver region coordinates, based solely on liver tissue H&E pathological images. Experimental results show that this analysis process is applicable to the analysis of liver tissue H&E pathological images of various genera, lesion types, and different disease states.

[0015] Preferably, segmenting and acquiring the analytical objects in the pathological image includes the following steps:

[0016] Annotate the objects to be analyzed in pathological images and train the object segmentation model;

[0017] A parsing object segmentation mask image corresponding to the pathological image is obtained using a parsing object segmentation model.

[0018] The pathological image of each parsed object is obtained by segmenting the corresponding pathological image using the parsed object segmentation mask image.

[0019] For analytical objects without clear boundaries, the pathological image is segmented into visual field blocks, which are then used as the pathological image units for analysis.

[0020] Traditional methods typically use large image regions as evaluation objects, which has limitations such as inaccurate evaluation results and unclear distinction between analysis objects. In particular, it is impossible to accurately distinguish and quantify the number, spatial location, and independent lesion characteristics of each analysis object.

[0021] This method utilizes artificial intelligence image segmentation technology to first accurately distinguish different types of analytical objects, and then precisely calculate and analyze the pathological features and spatial location of each independent object (i.e., pathological image unit). Preferably, the analytical objects include liver parenchymal cells, inflammatory cells, arteries, veins, bile ducts, fibrotic areas, and pathological image field blocks.

[0022] Based on the ability to distinguish between different types of analytical objects, this application trained an analytical object segmentation model to segment the above-mentioned types of analytical objects. This classification method covers the main physiological structures and analytical objects of liver tissue pathological evaluation.

[0023] Preferably, extracting the lesion features of the object to be analyzed includes the following steps:

[0024] Construct morphological algorithms to extract interpretable features from parsed objects;

[0025] The pathological image units of the parsed objects are integrated to form a pathological image unit dataset. An unsupervised feature extraction model is trained to extract deep learning features of the pathological image units.

[0026] A downstream evaluation algorithm is constructed to evaluate the interpretable features and deep learning features of the parsed object and calculate the downstream evaluation result features.

[0027] The interpretable features, deep learning features, and downstream evaluation result features of each parsed object are collectively used as the pathological features of the parsed object.

[0028] Traditional quantitative methods, when analyzing the lesion characteristics of the object, mainly rely on morphological parameters (such as area, proportion, distance, density, etc.). The selected features are highly subjective, and the quantitative perspective is relatively limited.

[0029] This method further constructs an unsupervised feature extraction model for the analyzed object (pathological image unit) to comprehensively capture its multidimensional feature information. Based on this, various downstream evaluation algorithms—including cluster analysis, correlation regression analysis, and model prediction—are introduced to integrate and refine the extracted lesion feature data. Ultimately, the obtained features (including interpretable features, deep learning features, and downstream evaluation results) can all serve as lesion representations of the analyzed object and be used in subsequent computational processes of this method.

[0030] Preferably, a morphological algorithm is constructed to extract interpretable features of the parsed object, including the following steps:

[0031] The internal regions of the object to be analyzed are labeled, and the internal region segmentation model of the object to be analyzed is trained to output the internal region segmentation mask map of the object to be analyzed corresponding to the pathological image.

[0032] The internal region segmentation mask image of the parsed object is cut into the corresponding internal region segmentation mask image of the parsed object to obtain the internal region segmentation mask image unit of the parsed object;

[0033] By using the pathological image units of the parsing object and the corresponding internal region segmentation mask image units, the shape parameters, quantity parameters, distance parameters, color parameters and data statistics parameters of the parsing object and its internal regions are calculated as interpretable features of the parsing object.

[0034] For the analysis object (pathological image unit), traditional methods usually quantify it by calculating interpretable features such as color distribution, overall area, and shape coefficient, but their information extraction capability is relatively limited.

[0035] This method further constructs a segmentation model for the internal regions of the analyzed object, which can finely divide its internal structure, thereby significantly expanding the range of extractable interpretable features and improving the precision. Preferably, the internal regions of the analyzed object include the blank lipid droplet region, the blank background cavity region, the red-stained normal cytoplasm region, the red-stained light-stained cytoplasm region, the blue-stained normal nucleus region, the blue-stained dark-stained nucleus region, and the heterochromatic region in the pathological image.

[0036] This classification method has good versatility and can cover the classification and segmentation tasks of internal regions of various analytical objects in H&E pathological images, avoiding the complex work of designing, labeling and training independent internal region segmentation models for each type of analytical object.

[0037] Preferably, the liver lobule object includes the portal vein outline and the central vein outline.

[0038] Compared to traditional methods of liver region segmentation, the method of segmenting liver lobules using the contours of the portal vein and central vein provides clearer object definitions, simplifies annotation, and provides basic information for the subsequent continuous quantification of liver region coordinates.

[0039] Preferably, segmenting liver lobule objects in pathological images includes the following optimization steps:

[0040] Annotate liver lobule objects in pathological images to form a liver lobule object annotation dataset;

[0041] The internal region segmentation mask of the parsed object is merged with the segmentation mask of the parsed object to form a merged segmentation mask. Specific segmentation regions in the merged segmentation mask are filtered and retained. The resolution of the merged segmentation mask is compressed to form a refined segmentation mask.

[0042] The liver lobule object segmentation model is trained using a refined segmentation mask image and a corresponding labeled dataset of liver lobule objects to obtain a segmentation mask image of the corresponding liver lobule object for any pathological image.

[0043] Traditional image segmentation methods often use pathological images directly as input to the segmentation model. Since liver lobules are primarily cavities (lacking useful information), their determination relies on high-resolution images from a large external area. This places extremely high demands on the computational power of the segmentation model and results in low accuracy.

[0044] This method uses refined segmentation mask images as input data to train the liver lobule object segmentation model. It can compress the amount of pathological image data, extract and retain key microscopic information, eliminate batch difference features, and improve the robustness of the segmentation model to external datasets. By compressing the image magnification, it can input pathological image information from a larger field of view into the liver lobule object segmentation model with the same computing power. It is especially suitable for objects such as portal vein and central vein, where most of the internal features of the object region are useless features such as lumens, and the identification and classification of objects require the help of fine feature information from a large area around the object region, which greatly improves the accuracy of the model.

[0045] Preferably, a liver region coordinate algorithm is established, which uses the positional information of the liver lobule to calculate the liver region coordinates of each point in the pathological image, and obtains the liver region coordinate results, including the following steps:

[0046] Calculate the minimum distance between the contour edges of each portal vein and the contour edges of the adjacent central vein within the segmentation mask map of the liver lobule scale corresponding to the pathological image, and calculate the median or average value of the minimum distance as the reference liver lobule radius of the pathological image.

[0047] Using the baseline liver lobule radius as the radiation coverage range of the portal vein contour and the central vein contour, calculate the minimum distance between each point within the radiation coverage range and the edge of the corresponding portal vein contour or central vein contour.

[0048] Using the baseline hepatic lobule radius as the standard distance unit, the minimum distance between each point and the edge of the portal vein or central vein contour is corrected to obtain the standardized minimum distance of each point. For points belonging to the central vein contour, their liver region coordinates are equal to the standard distance unit of the point minus the standardized minimum distance of the point. For points belonging to the portal vein contour, their liver region coordinates are equal to the standardized minimum distance of the point.

[0049] This method achieves continuous quantization of liver region coordinates by utilizing the positional information of liver lobule objects. Based on this, it standardizes different liver lobules in various pathological images by calculating the radius of a baseline liver lobule, thus achieving comparability of liver region coordinate results between different liver lobules and between different pathological images. Preferably, the method establishes a liver region coordinate algorithm, calculates the liver region coordinates of each point in the pathological image using the positional information of the liver lobule objects, and obtains the liver region coordinate results. The method also includes the following optimization steps:

[0050] Statistical analysis of lesion feature data of each analytical object in the peripheral field of each point of the pathological image is performed to obtain the statistical results of lesion features at each point of the pathological image.

[0051] A liver region coordinate prediction model is trained using the statistical results of lesion characteristics and the corresponding liver region coordinate results.

[0052] Using a liver region coordinate prediction model, based on the statistical results of lesion features, the liver region coordinates of each point in the pathological image are predicted, thus obtaining the predicted liver region coordinates of the entire pathological image.

[0053] Direct measurement methods can achieve continuous quantification of some areas, but their continuous quantification capability is significantly limited in terms of field of view and range due to the limited number of liver lobules visible in pathological images.

[0054] Based on this, this method further relies on the feature information of the parsed object to construct a liver region coordinate prediction model, so as to realize the liver region coordinate calculation for each point in the entire range of the pathological image.

[0055] Preferably, a liver region coordinate algorithm is established, which uses the positional information of the liver lobule to calculate the liver region coordinates of each point in the pathological image to obtain the liver region coordinate results. The algorithm also includes the following optimization steps:

[0056] The weight matrix is ​​calculated based on the liver region coordinates obtained by direct measurement. This matrix is ​​then used to weight and merge the liver region coordinates obtained by direct measurement at each point with the liver region coordinates predicted by the liver region coordinate prediction model.

[0057] Direct measurement and model prediction methods have different quantification accuracies for liver region coordinates in different pathological image areas. This method relies on weighted merging to retain the results of the high-accuracy areas of the two methods, cross-correct each other, and merge to form an accurate full-image prediction of liver region coordinates.

[0058] Preferably, the method for analyzing the spatial distribution of lesion features in liver tissue pathological images of the present invention further includes the following steps:

[0059] Using pathological images and the spatial distribution analysis results of corresponding lesion features in the liver region as paired datasets, a fast analysis model is trained.

[0060] The spatial distribution of lesion features in any liver tissue pathological image was predicted using a fast analytical model.

[0061] The aforementioned process of this method largely relies on the accurate localization of liver lobules. To extend its application to pathological images, including those with incomplete information on liver lobules, this method constructs a predictive model to obtain the spatial distribution analysis results of lesion features in the liver region solely based on direct prediction from the pathological image. This broadens the applicability of the method.

[0062] A second aspect of the present invention provides a system for analyzing the spatial distribution of lesion features in liver tissue pathological images, comprising:

[0063] The analysis object acquisition and feature extraction module segments the pathological image, acquires the analysis object, and extracts the lesion features of the analysis object;

[0064] The liver lobule object segmentation module segments liver lobule objects in a pathological image to obtain the location information of the liver lobule objects in the pathological image.

[0065] The liver region coordinate analysis module establishes a liver region coordinate algorithm, which uses the positional information of the liver lobule to calculate the liver region coordinates of each point in the pathological image and obtains the liver region coordinate results.

[0066] The liver region spatial distribution analysis module statistically analyzes the distribution of lesion features of the analyzed objects as the coordinates of the liver region change, and obtains the analysis results of the spatial distribution of lesion features of pathological images with the liver region.

[0067] The present invention provides a method and system for analyzing the spatial distribution of lesion features in liver tissue pathological images, guided by the physiological structural logic of liver tissue. By constructing segmentation models for different objects in different steps, it is possible to accurately identify and segment the internal structural regions of the analysis objects, the analysis objects themselves, and liver lobule objects such as the portal vein and central vein in liver tissue pathological images, providing different information such as lesion features, analysis objects, and physiological spatial locations for subsequent analysis processes of the system.

[0068] By combining lesion features and segmentation results of liver lobules, a liver region coordinate algorithm is further constructed. This algorithm can accurately calculate the liver region coordinates in liver tissue pathological images based on the lesion features at each point and their relative position information with the liver lobules.

[0069] This system integrates the analysis results of microscopic lesion features with the spatial information of liver region coordinates to achieve comprehensive and accurate analysis of the spatial distribution of microscopic lesion features in liver tissue pathological images. Attached Figure Description

[0070] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly described below.

[0071] Figure 1 This is a flowchart illustrating a method for analyzing the spatial distribution of microscopic lesion features in liver tissue HE pathological images according to an embodiment of the present invention.

[0072] Figure 2 This is a schematic diagram of a parsing object instance segmentation model according to an embodiment of the present invention;

[0073] Figure 3This is a schematic diagram of the semantic segmentation model inside the parsed object according to an embodiment of the present invention;

[0074] Figure 4 This is a schematic diagram illustrating the object acquisition and microscopic lesion feature extraction according to an embodiment of the present invention;

[0075] Figure 5 This is a schematic diagram of the visualization of the microscopic lesion features of the object to be analyzed according to an embodiment of the present invention;

[0076] Figure 6 This is a schematic diagram of a macroscopic object semantic segmentation model according to an embodiment of the present invention;

[0077] Figure 7 This is a schematic diagram of the liver region coordinate calculation process according to an embodiment of the present invention;

[0078] Figure 8 This is a schematic diagram of the liver region coordinate calculation results according to an embodiment of the present invention;

[0079] Figure 9 This is a schematic diagram illustrating the analytical process of spatial distribution of microscopic lesion features in the liver region according to an embodiment of the present invention;

[0080] Figure 10 This is the analytical result of the spatial distribution of lesion features with the liver region according to an embodiment of the present invention. Detailed Implementation

[0081] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0082] Unlike the physiological and pathological structures of other tissues, liver tissue possesses a complex, multi-layered physiological structure. Among these, the portal areas, central veins, and the hepatocytes arranged between them form a unique gradient metabolic structure—the liver lobules. During the development of liver diseases, lesions within the liver lobules can exhibit complex and heterogeneous spatial distribution patterns. Analyzing this spatial distribution information can help researchers quickly understand the pathogenesis and etiological composition of liver diseases.

[0083] HE staining provides a direct visual representation of histopathological changes and is a crucial core diagnostic and evaluation method in the medical field, encompassing disease diagnosis, research, drug evaluation, and development. Researchers fix, embed, section, and stain biological tissues with HE to create pathological sections, which are then acquired, observed, and evaluated using digital acquisition methods such as microscopic imaging or microscopic image scanning.

[0084] The diagnosis and evaluation of liver histopathological images (HE images) are of fundamental significance for the clinical diagnosis and treatment of liver diseases, preclinical drug evaluation, and pharmaceutical research and development. As the most intuitive gold standard for diagnosis, it promotes the development of many derivative detection technologies such as clinical biochemical indicators and imaging.

[0085] In the following embodiments, HE pathological images are used as the data type for parsing to describe the parsing steps of the present invention in detail.

[0086] Figure 1 This is a flowchart illustrating a method for analyzing the spatial distribution of liver tissue pathological images with varying liver regions, according to an embodiment of the present invention.

[0087] like Figure 1 As shown in this embodiment, the method for analyzing the spatial distribution of lesion features in liver tissue HE pathological images along the liver region includes the following steps:

[0088] Step S1: Establishment and standardization of liver tissue pathological image dataset.

[0089] A liver tissue pathological image dataset was established, and the pathological image dataset was standardized to obtain a standardized pathological image dataset.

[0090] Step S2: Analysis of object acquisition and feature extraction.

[0091] Standardized pathological images in a standardized pathological image dataset are segmented and parsed at the cell scale to obtain pathological image units of the parsed objects. Feature extraction and parsing algorithms are constructed to extract the internal structural features of the parsed objects as lesion features.

[0092] Step S3: Segmentation of liver lobule objects.

[0093] The liver lobule objects in the standardized pathological image dataset are labeled to train a liver lobule object segmentation model. The liver lobule object segmentation model is then used to infer the standardized pathological images to obtain the liver lobule object segmentation mask map corresponding to the standardized pathological images, which provides the location information of the liver lobule objects in the pathological images.

[0094] Step S4: Spatial location analysis of the liver region.

[0095] A liver region coordinate algorithm is established to calculate the liver region coordinates of each point in the pathological image using the positional information of the liver lobule object. The spatial coordinate results of the liver region are obtained, which reflect the physiological position of each point in the standardized pathological image and the object being analyzed within the liver lobule.

[0096] Step S5: Spatial distribution analysis of lesion characteristics in the liver region.

[0097] The distribution of lesion features of the analyzed object with the change of liver region coordinates is statistically analyzed to obtain the spatial distribution analysis results of lesion features in the liver region of the standardized pathological image; the standardized pathological image and the corresponding spatial distribution analysis results of lesion features in the liver region are used as a paired dataset to train a fast analysis model; the fast analysis model is used to quickly predict any liver tissue pathological image to obtain the corresponding spatial distribution analysis results of lesion features in the liver region.

[0098] The present invention relates to an algorithm and system for analyzing the spatial distribution of microscopic lesion features in liver tissue pathological images, guided by the physiological structural logic of liver tissue. By constructing segmentation models for different objects in different steps, it can accurately identify and segment the internal structural regions of the analysis objects, the analysis objects themselves, and liver lobule objects such as the portal vein and central vein in liver tissue pathological images. This provides different information for subsequent analysis processes of the system, including the lesion features within the analysis objects, the analysis objects themselves, and their physiological spatial location.

[0099] A liver region coordinate algorithm is constructed by combining the lesion features of the analyzed object and the segmentation results of the liver lobule objects. This algorithm can fully utilize the lesion features of the analyzed object at each point in the pathological image, and the relative position information of each point with liver lobule objects such as the portal vein and central vein, to accurately calculate the liver region coordinates of each point in the liver tissue pathological image.

[0100] This system integrates the analysis results of lesion features with the coordinate results of the liver region, enabling comprehensive and accurate analysis of the spatial distribution of lesion features in liver tissue HE pathological images.

[0101] In one specific embodiment, step S1, establishing and standardizing the liver tissue HE pathological image dataset, includes the following steps:

[0102] Step S11: Establish a liver tissue HE pathological image dataset.

[0103] Step S12: Standardize the staining style of HE pathological images.

[0104] In step S12, the staining style of HE pathological images is standardized, which specifically includes the following steps:

[0105] Existing technologies such as Macenko, Reinhard, and STST can be used to perform staining style standardization processing on each pathological image in the liver tissue HE pathological image dataset to obtain the corresponding standardized HE pathological images, thus forming a standardized HE pathological image dataset.

[0106] The standardization process is used to correct batch variations and preserve the true pathological differences to improve the accuracy of subsequent system analysis results.

[0107] In step S2, the cell-scale analysis objects acquired may include liver parenchymal cells, inflammatory cells, blood vessels (arteries, veins, bile ducts), fibrotic areas, and pathological image field blocks, etc.

[0108] In the following embodiments, the analysis steps will be described primarily using hepatocytes as an example.

[0109] In one embodiment, step S2 specifically includes the following steps:

[0110] Step S21: Parse object segmentation and acquisition.

[0111] Step S22: Extract the lesion features of the parsed object.

[0112] In step S21, the standardized pathological images in the standardized pathological image dataset are segmented and the parsing objects are collected to obtain the pathological image units of the parsing objects.

[0113] Based on the different parsing objects, standardized pathological images are labeled. The labeled parsing objects may include cell outlines, vascular outlines, and fibrosis outlines. Vascular outlines include arteries, veins, bile ducts, etc. The parsing object labeling is used to train and obtain a parsing object segmentation model.

[0114] The parsing object segmentation model is used to infer standardized pathological images and obtain parsing object segmentation mask images corresponding to the standardized pathological images.

[0115] The pathological image units of each parsed object are obtained by segmenting the corresponding standardized pathological image using the parsed object segmentation mask image.

[0116] For analytical objects without clear boundaries, taking fibrosis as an example, the standardized pathological image is segmented into visual field blocks to serve as the pathological image units of the analytical object. The segmentation mask map of the analytical object with the fibrosis contour is used to help extract the lesion features of the analytical object, which is consistent with the method of using the segmentation mask map of the internal region of the analytical object in subsequent steps.

[0117] Traditional pathological image analysis often uses large visual field blocks such as 512 × 512 as the object of feature extraction and analysis. However, the meaning of the analysis object itself and the extracted features is unclear. This method cannot calculate the liver region coordinates using the lesion features of the analysis object in liver tissue pathological images, nor can it analyze the changes in the lesion features of the analysis object as the liver region changes. Therefore, this embodiment uses key physiological and pathological structures within the liver lobules as the analysis object. The analysis object itself has important analytical significance and can also provide key lesion features to help improve the accuracy of the liver region coordinate analysis results.

[0118] Meanwhile, for analysis objects without clear boundaries, although this embodiment still uses the segmented pathological image field of view as the analysis object, the segmentation size of the pathological image field of view will be adjusted in subsequent steps according to the baseline liver lobule radius of each pathological image (baseline liver lobule radius D(lobule) / 15 ~ baseline liver lobule radius D(lobule) / 5). This method of dynamically adjusting the size of the pathological image field of view based on the baseline liver lobule radius can significantly improve the accuracy of the liver region coordinate results and the results of the changes in lesion characteristics with the distribution of the liver region.

[0119] The above methods can be adapted to the collection needs of various analytical objects of liver tissue, and can realize the collection, feature extraction and lesion feature integration of any type of analytical object, which can be used together in the subsequent calculation of liver region coordinates and analysis of lesion features with the spatial distribution of liver region.

[0120] Figure 2 This is a schematic diagram of a parsing object segmentation model according to an embodiment of the present invention.

[0121] like Figure 2 As shown, taking hepatocytes as an example of the parsing object, the outlines of each cell in the original pathological image 21 are labeled to obtain a cell outline labeled image 22. The labeled dataset consisting of cell outline labeled images 22 and the corresponding original pathological image 21 constitute a paired dataset, which is used to train the parsing object segmentation model (cell) 24. The parsing object segmentation model (cell) 24 is used to segment any standardized pathological image 23 to obtain the corresponding parsing object segmentation mask image (cell) 25.

[0122] The same method can be used to train the vessel segmentation model and obtain the vessel segmentation mask. The parsing object segmentation model (cell) 24, along with the vessel segmentation model, serves as the parsing object segmentation model. The parsing object segmentation mask (cell) 25 and the vessel segmentation mask 43 (see...) Figure 6 These, along with others, serve as the parsing object for the segmentation mask image.

[0123] In one embodiment, step S21 specifically includes the following steps:

[0124] Step S211, data preprocessing and enhancement.

[0125] The parsing object segmentation model takes a pathological image with a resolution of 20× and a size of 512 × 512 px^2 as input, and obtains the corresponding parsing object segmentation mask image through training.

[0126] To improve the generalization ability of the image segmentation model, a series of random data augmentation measures are implemented on the paired dataset before training the model. These augmentation measures include, but are not limited to, the following: rotation and flipping, scaling transformation, elastic deformation, random Gaussian noise, exposure and color adjustment, random blurring and dirt removal, etc.

[0127] Step S212, parse the object segmentation model (cell) 24 and build it.

[0128] The parsing object segmentation model (cell) 24 architecture mainly consists of three modules: the backbone network, the connection module, and the task module.

[0129] The backbone network is used to extract features from the input image, including a downsampling layer, a feature extraction layer, a 3-layer split convolutional module, and a spatial pyramid pooling module.

[0130] The downsampling layer generates a downsampled feature map by performing a downsampling operation on the input image.

[0131] The feature extraction layer consists of a convolutional layer, a batch normalization layer, and an activation function, and is used to extract preliminary features of the image.

[0132] The split convolution module receives the output of the feature extraction layer or other split convolution modules, and divides the input feature map into two paths according to the channels. One path performs complex feature extraction by concatenating multiple residual blocks, and the other path merges with the output of the residual module through short-circuit connection, thereby improving the nonlinear representation ability and generalization ability of the network.

[0133] The spatial pyramid pooling module receives the feature map output by the split convolution module, extracts features at different scales of the image through hierarchical pooling operations, enhances the model's robustness to scale changes, and converts the feature map into a feature vector of fixed size.

[0134] The connection module is used to fuse multi-scale feature maps output by the backbone network, including: feature pyramid network and path fusion network.

[0135] The Feature Pyramid Network enhances the model's ability to detect targets at different scales by fusing feature maps from different levels through top-down path enhancement and lateral connections.

[0136] The path fusion network adds a bottom-up path to the feature pyramid network to achieve bidirectional fusion of feature maps.

[0137] The task module is used to complete object detection and instance segmentation tasks based on the feature maps output by the connection module. It includes: a multi-layer convolution module, which performs operations on the input feature maps and outputs the segmentation mask map of the cell, the bounding box coordinates of the cell, and the confidence score.

[0138] The loss function used is the cross-entropy loss function, which calculates the difference between the segmentation mask image generated by the model and the annotation result during the model training process to guide the updating of model parameters.

[0139] Using the parsing object segmentation model (cell) 24, infer any standardized pathological image 23 to obtain the parsing object segmentation mask (cell) 25 corresponding to the arbitrary standardized pathological image 23.

[0140] The corresponding standardized pathological image 23 is cut using the segmentation mask image (cell) 25 of the parsing object to obtain the pathological image unit 55 with each liver parenchymal cell as the parsing object.

[0141] In step S22, a morphological algorithm is constructed to extract interpretable features of each parsed object.

[0142] The pathological image units of the analyzed object are integrated to form a pathological image unit dataset. An unsupervised feature extraction model is trained to extract deep learning features of the pathological image units of the analyzed object.

[0143] A downstream evaluation algorithm is constructed to evaluate the interpretable features and deep learning features of the parsed object and calculate the downstream evaluation result features.

[0144] The interpretable features and deep learning features of each analytical object, as well as the derived downstream evaluation results based on the interpretable features and deep learning features, are used together as the pathological features of the analytical object.

[0145] In this embodiment, the extraction of interpretable features can satisfy the extraction and analysis of key pathological feature data with clear meaning; it can also help optimize the process and method of some analysis steps in this invention to obtain better results; and it can provide a clear interpretation of the analysis results obtained in each analysis step.

[0146] Deep learning feature extraction can fully extract the features of the parsed object, including feature data that cannot be extracted by morphological calculations.

[0147] Interpretable features, deep learning features, and downstream evaluation results features are used together as the lesion features of the analysis object. This can broadly cover most of the feature data of the analysis object, greatly improve the accuracy of the liver region coordinate results, and the accuracy and analysis range of the analysis results as the lesion features change with the distribution of the liver region.

[0148] In one embodiment, step S22 specifically includes the following steps:

[0149] Step S221, Interpretable Feature Extraction Based on Morphological Computation. A morphological algorithm is constructed to extract interpretable features of the parsed object.

[0150] Figure 3This is a schematic diagram of the internal region segmentation model of the parsing object according to an embodiment of the present invention.

[0151] like Figure 3 As shown, the internal regions of the parsed object are labeled to form the internal region labeled image 32 of the parsed object, forming an internal region labeled dataset, which is paired with the original pathological image 21 to form a dataset for training to obtain the internal region segmentation model 34 of the parsed object; the internal region segmentation model 34 of the parsed object is used to infer any standardized pathological image 23 to obtain the internal region segmentation mask image 35 of the parsed object corresponding to the standardized pathological image 23.

[0152] Figure 4 This is a schematic diagram illustrating the object acquisition and lesion feature extraction according to an embodiment of the present invention.

[0153] like Figure 4 As shown, the internal region segmentation mask image of the parsed object is cut using the parsed object segmentation mask image to obtain the internal region segmentation mask image unit 54 of the parsed object.

[0154] For physiological and pathological objects and regions without clear boundaries, the internal region segmentation mask image 35 of the object to be analyzed is cut into visual field blocks and used as the internal region segmentation mask image unit 54 of the object to be analyzed.

[0155] Using the pathological image unit 55 of the parsing object and the corresponding internal region segmentation mask image unit 54, a morphological algorithm 56 is constructed to calculate the shape parameters, quantity parameters, distance parameters, color parameters, and data statistics parameters of the parsing object and its internal regions, which serve as interpretable features of the parsing object.

[0156] Step S221 specifically includes the following steps:

[0157] Step S2211: Parse the internal region segmentation of the object.

[0158] The internal region segmentation model 34 of the analyzed object, based on the physiological and pathological attributes and color gradations, hues, and morphologies of each region in the standardized pathological image 23, annotates and segments the internal regions of the analyzed object. Taking a liver tissue HE pathological image as an example, the internal regions of the analyzed object may include: blank lipid droplet regions, blank background cavity regions, red-stained normal cytoplasm regions, red-stained light-stained cytoplasm regions, blue-stained normal nucleus regions, blue-stained dark-stained nucleus regions, and heterochromatic regions. The annotated images 32 of the internal regions of the analyzed object constitute the internal region annotation dataset. The annotated images 32 of the internal regions of the analyzed object are used to train the internal region segmentation model 34 of the analyzed object. The internal region segmentation model 34 takes a standardized pathological image 23 with a resolution of 20× and a size of 512 × 512px^2 as input, and outputs the corresponding internal region segmentation mask image 35 of the analyzed object through training.

[0159] To improve the generalization ability of the image segmentation model, before training the internal region segmentation model 34 of the parsed object, a series of random data augmentation measures are implemented on the paired dataset consisting of the internal region annotation dataset and the original pathological image 21. These augmentation measures can be similar to step S211.

[0160] The internal region segmentation model 34 for parsing objects adopts an encoder-decoder architecture, with the encoder including a backbone network and a multi-scale feature fusion module.

[0161] The backbone network is a deep residual network structure, in which some 3×3 convolutional layers are replaced by deformable convolutional V3 (DCNv3) to enhance the model's ability to extract global information from the image.

[0162] The multi-scale feature fusion module is connected after the backbone network. It adopts the idea of ​​hollow spatial pyramid pooling to extract and fuse feature maps of different scales and output a multi-scale fused feature map.

[0163] The decoder receives the multi-scale fused feature map output by the encoder and the feature map output by each residual block through short-circuit connection, and performs upsampling and fusion processing, specifically including: upsampling the received feature maps of various types to match the resolution; fusing the feature maps through a 3×3 convolutional layer to obtain fused feature information; and transforming the fused feature information back to the resolution of the original image through an interpolation upsampling module to obtain the classification result of each pixel, forming a segmentation mask map 35 of the internal region of the parsed object.

[0164] The loss function used is the cross-entropy loss function, which calculates the difference between the internal region segmentation mask map 35 generated by the model and the annotation results during the model training process, so as to guide the updating of model parameters.

[0165] Step S2212: Calculate the interpretable features of the parsed object.

[0166] A morphological algorithm 56 is constructed. Taking hepatocytes as an example, it calculates the shape parameters of each segmented region in the pathological image unit 55 (which is the object of analysis) and the segmentation mask image unit 54 (which is the internal region segmentation mask image unit of the object of analysis), such as area and shape coefficient; the color parameters of each region, such as the average value and standard deviation of HSV channels; and the distance and quantity parameters between each region, such as the distance between the cell membrane and the cell nucleus and the number of lipid droplets. These serve as interpretable features of hepatocytes.

[0167] In this embodiment, compared with relying solely on the pathological image units of the parsing object for interpretable feature extraction, the setting of the parsing object internal region segmentation model 34 and the parsing object internal region segmentation mask map 35, combined with the parsing object internal region segmentation mask map information, can significantly improve the extraction capability and range of the parsing object's interpretable features.

[0168] The internal region classification method of the internal region segmentation mask image of the parsed object can be adapted to the internal structure classification needs of various parsed objects of liver tissue. This internal region classification method uses the differences in basic structural and color attributes as the basis for region classification, and the number of classifications is kept to a minimum, so as to improve the consistency of judgment among pathology experts, reduce the workload of annotation, and improve the accuracy of model segmentation.

[0169] Step S222, deep learning feature extraction based on unsupervised feature extraction model, specifically includes the following steps:

[0170] Step S2221, data preprocessing and enhancement.

[0171] like Figure 4 As shown, after processing the large number of samples in the liver tissue HE pathological image dataset established in step S11 with pathological image standardization in step S12 and parsing object segmentation and acquisition in step S2, the processing results are integrated to form a parsing object pathological image unit dataset for training the unsupervised feature extraction model 57.

[0172] The pathological image units 55 of the parsed object are uniformly scaled and filled into the center of a 64×64 pixel white background image to unify the size of the single-cell pathological images, which are then used as model input.

[0173] To improve the generalization ability of the unsupervised feature extraction model 57, a series of random data augmentation measures are implemented on the pathological image unit 55 during the training of the feature extraction model. These augmentation measures can be the same as those in step S211.

[0174] Step S2222: Building and training the unsupervised feature extraction model.

[0175] The architecture of the unsupervised feature extraction model 57 includes an image cropping module, a first image feature extraction module, and a second image feature extraction module.

[0176] The pathological image unit 55 inputs to the image cropping module, which cuts the image to obtain an even number of large-field-of-view images as the global image and an even number of locally cropped field-of-view images. Each image undergoes random data augmentation processing in step S2221, and then is input to the first image feature extraction module and the second image feature extraction module.

[0177] The first image feature extraction module includes: an image input module, an image segmentation module, an embedding module, a location encoding module, an encoder module, and a feature output module.

[0178] The image input module is used to receive two-dimensional image data to be processed.

[0179] The image segmentation module is connected to the image input module and is used to segment the received two-dimensional image data into multiple image blocks of preset size.

[0180] The embedding module, connected to the image segmentation module, is used to convert each image patch into a corresponding embedding vector.

[0181] The position encoding module, connected to the embedding module, is used to add a corresponding position encoding to each embedding vector to reflect the position information of the image patch in the original image.

[0182] The encoder module is connected to the position encoding module and contains several sequentially connected encoding layers. Each encoding layer contains a self-attention mechanism, a feedforward neural network, a residual connection module, and a layer normalization module, which are used to process the embedding vector after adding position encoding and output the encoded feature vector.

[0183] The input vector of the multi-head self-attention mechanism is linearly transformed into three parts: query, key, and value. Attention weights are obtained by performing a dot product between the query vector and all key vectors and normalizing the result. The normalized weights are then weighted and summed with the corresponding value vectors to obtain a new representation for each word. This process is performed in parallel by multiple heads. Finally, the outputs of all heads are concatenated and linearly transformed to obtain the final output.

[0184] The feedforward neural network performs an independent nonlinear transformation on the output at each location. The network includes two linear transformations and a ReLU activation function.

[0185] The residual connection module is used to add the input and output of the sub-layer after the self-attention mechanism and feedforward neural network of each encoder layer.

[0186] The layer normalization module is used to normalize the output after each encoder layer to ensure that the output of each layer remains stable during training.

[0187] The feature output module is connected to the encoder module and is used to output the final 1×768-dimensional feature vector as the deep learning features of the image.

[0188] The second image feature extraction module is completely identical to the first image feature extraction module in terms of module architecture. The difference is that the second image feature extraction module only accepts the global image as input.

[0189] When calculating the contrast loss, during model training, global images from the same source and cropped view images are considered positive samples, while global images from different sources and cropped view images are considered negative samples. The similarity between positive samples and the similarity between negative samples are calculated, and the contrast loss is obtained based on a set threshold.

[0190] The cross-entropy loss function is used to measure the similarity between the outputs of the first image feature extraction module and the second image feature extraction module.

[0191] The first image feature extraction module is trained to mimic the output of the second image feature extraction module; the model weights of the first image feature extraction module are updated using cross-entropy loss and contrast loss.

[0192] The weights of the second image feature extraction module are not updated directly through the loss function, but are updated by calculating the exponential moving average of the weights of the first image feature extraction module.

[0193] The trained second image feature extraction module is used as the final unsupervised feature extraction model 57. The 1×768-dimensional feature vector obtained by the unsupervised feature extraction model 57 for each pathological image unit 55 is the deep learning feature of the pathological image unit.

[0194] Further downstream evaluation algorithms can be constructed to perform cluster analysis, continuous change analysis, and model prediction on the interpretable features and deep learning features of the microscopic analysis objects, and calculate the downstream evaluation result features.

[0195] Downstream evaluation result features may include: categorical evaluation result features, reflecting the lesion classification information of the microscopic analysis object; and continuous evaluation result features, reflecting the lesion severity information of the microscopic analysis object. Interpretable features, deep learning features, and downstream evaluation result features together serve as the microscopic lesion features of the analysis object.

[0196] Similar to the parsing process in step S2, the parsing process with blood vessels (arteries, veins, bile ducts) as the parsing object can be completed, and the blood vessel segmentation model, output blood vessel segmentation mask map, blood vessel internal region segmentation mask map, blood vessel pathological image unit and blood vessel lesion features (interpretable features, deep learning features, and downstream evaluation result features) can be obtained.

[0197] Similar to the analysis process in step S2, the analysis process with fibrosis as the analysis object can be completed to obtain the fibrosis segmentation model, output the fibrosis segmentation mask map, fibrosis pathological image units and microscopic lesion features of blood vessels (interpretable features, deep learning features, and downstream evaluation result features).

[0198] Figure 5 This is a schematic diagram of the visualization of the lesion features of the object to be analyzed according to an embodiment of the present invention.

[0199] like Figure 5 As shown, the color reflects the lesion feature analysis results of the analyzed object 61, demonstrating the spatial distribution specificity of the lesion features.

[0200] In step S3, liver lobule objects in the standardized pathological image dataset are labeled to form a liver lobule object labeled dataset, which is used to train a liver lobule object segmentation model. The liver lobule object segmentation model is used to infer the standardized pathological image 23 to obtain a liver lobule object segmentation mask corresponding to the standardized pathological image 23, which provides the location information of the liver lobule objects. The parsed object internal region segmentation mask 35 output by the parsed object internal region segmentation model 34 and the parsed object segmentation mask output by the parsed object segmentation model are merged to form a merged segmentation mask. Specific segmentation regions in the merged segmentation mask are filtered and retained, and resolution compression is performed to form a refined segmentation mask.

[0201] The refined segmentation mask image is used as the input data for the liver lobule object segmentation model. It is paired with the corresponding liver lobule object annotation dataset to form a paired dataset, which is used to train the liver lobule object segmentation model to obtain the corresponding liver lobule object segmentation mask image for any pathological image.

[0202] Figure 6 This is a schematic diagram of a liver lobule object segmentation model according to an embodiment of the present invention.

[0203] like Figure 6 As shown, in step S3, the internal region segmentation mask map 35 of the parsing object output by the parsing object internal region segmentation model 34 is merged with the segmentation mask maps of each parsing object output by the parsing object segmentation model (parsing object segmentation mask map (cell) 25, vascular segmentation mask map 43), specific segmentation regions are filtered and retained, and resolution compression is performed to form a refined segmentation mask map 44.

[0204] The refined segmentation mask image 44 is used as the input data for the liver lobule object segmentation model 45. It is paired with the liver lobule object annotation dataset to form a dataset for training the liver lobule object segmentation model 45 and generating the liver lobule object segmentation mask image 46.

[0205] In one specific embodiment, step S3 specifically includes the following steps:

[0206] Step S31, Data preprocessing and enhancement;

[0207] Step S32: Construction of liver lobule object segmentation model.

[0208] In step S31, data preprocessing and enhancement specifically include the following steps:

[0209] The liver lobule object segmentation model 45, the labeled and segmented objects include the portal vein contour and the central vein contour.

[0210] In this embodiment, the portal vein contour and the central vein contour are used as liver lobule objects for identification and segmentation. The portal vein and the central vein are key physiological structures that help determine the coordinates of the liver region. Since the size of the vein lumen varies, the liver region coordinates can be calculated by relying on the edge position information of the vein contour rather than the centroid or center point of the vein.

[0211] Since the identification of liver lobule objects requires comprehensive consideration of information from a large number of peripheral cell-scale objects under a large field of view, the input data format of the liver lobule object segmentation model 45 is different from that of the parsing object segmentation model and the parsing object internal region segmentation model 34 in steps S212 and S2211.

[0212] The internal region segmentation mask 35 output by the internal region segmentation model 34 of the parsing object is merged with the segmentation mask (segmentation mask (cell) 25, vascular segmentation mask 43) output by the parsing object segmentation model. The segmentation region information of each parsing object contour, such as vascular contour and liver parenchymal cells, and the internal region segmentation mask of each parsing object outside the contour of each parsing object is retained and cut into a 2048×2048 px^2 field-of-view image, compressed into a 512×512 px^2 image, as the refined segmentation mask 44.

[0213] The refined segmentation mask image 44 and the corresponding liver lobule object labeled dataset are used as a pairing dataset for training the liver lobule object segmentation model 45.

[0214] In this embodiment, a refined segmentation mask image is used as the input data training model for the liver lobule object segmentation model. This can compress the amount of pathological image data, extract and retain key microscopic information, eliminate batch difference features, and improve the robustness of the segmentation model to external datasets. Compressing the image magnification enables the input of pathological image information from a larger field of view into the liver lobule object segmentation model 45 with the same computing power. This is especially suitable for objects such as the portal vein and central vein, where most of the internal features of the object region are useless features such as lumens, and the identification and classification rely on the fine feature information of a large area around the object region. This greatly improves the accuracy of the model.

[0215] In step S32, the model architecture of the liver lobule object segmentation model 45 can be constructed using the same architecture as the parsed object segmentation model 34 in step S2211.

[0216] The segmentation mask map 44 is refined using the liver lobule object segmentation model 45 to obtain the liver lobule object segmentation mask map 46 corresponding to the standardized pathological image 23, providing the location information of the portal vein and central vein contours.

[0217] In step S4, based on the location information of the liver lobule, a liver region coordinate algorithm is established to calculate the liver region coordinate results, which reflect the physiological spatial position of each microscopic point in the liver tissue HE pathological image within the liver lobule.

[0218] Figure 7 This is a schematic diagram of the liver region coordinate calculation process according to an embodiment of the present invention.

[0219] like Figure 7 As shown, using the liver lobule object segmentation mask image 46, the directly measured liver region coordinates 72 within the radial coverage area of ​​the portal vein and central vein are obtained through direct distance calculation. The liver region coordinate prediction model 74 is trained using the parsed object lesion features 58 combined with the directly measured liver region coordinates 72. The liver region coordinate prediction model 74 is then used to infer the lesion features 58 at each point to obtain the model-predicted liver region coordinates 75. Finally, the directly measured liver region coordinates 72 and the model-predicted liver region coordinates 75 are weighted and summed to obtain the full-image liver region coordinates 76.

[0220] In one embodiment, step S4 specifically includes the following steps:

[0221] Step S41, direct measurement and calculation of liver region coordinates;

[0222] Step S42, model prediction of liver region coordinates.

[0223] In step S41, the direct measurement calculation of the liver region coordinates includes the following steps:

[0224] For any standardized pathological image 23, calculate the minimum distance D(pc) between the contour edges of each portal vein and the contour edges of the adjacent central vein within the corresponding liver lobule object segmentation mask image 46, and calculate the median or average value of the minimum distance D(pc) as the reference liver lobule radius D(lobule) of the standardized pathological image 23.

[0225] The formula for calculating the baseline liver lobule radius D is as follows:

[0226]

[0227] D(lobule) = median(D(pc))

[0228] in,

[0229] sqrt() calculates the square root;

[0230] min() calculates the minimum value;

[0231] median() calculates the median;

[0232] xp and yp represent the x and y coordinates of a pixel on the edge of a portal vein contour;

[0233] xc and yc represent the x and y coordinates of a pixel on the edge of a central vein contour;

[0234] D(pc) represents the minimum distance between the edge of the portal area outline and the edge of the adjacent central vein outline;

[0235] D(lobule) represents the baseline liver lobule radius.

[0236] The baseline hepatic lobule radius D (lobule) is used as the radiation coverage area of ​​the portal vein and central vein; the minimum distance D (min) between each point within the radiation coverage area and the edge of the corresponding portal vein or central vein contour is calculated.

[0237] The formula for calculating the minimum distance D (min) is as follows:

[0238]

[0239] in,

[0240] xpc and ypc represent the x and y coordinates of a pixel on the edge of a portal vein or a central vein.

[0241] xi and yi represent the x and y coordinates of a certain point;

[0242] D (min) represents the minimum distance between site i and the edge of the portal vein or central vein outline to which it belongs.

[0243] Using the baseline hepatic lobule radius D as the standard distance unit, the minimum distance D (min) between each point and the edge of its respective vein contour is corrected to obtain the standardized minimum distance D (standard) for each point; for a site belonging to the central vein contour, its liver region coordinate D is equal to the standard distance unit of that site minus the standardized minimum distance D (standard) for that site; for a site belonging to the portal vein contour, its liver region coordinate is equal to the standardized minimum distance of that site.

[0244] The formula for calculating the standardized minimum distance D is as follows:

[0245] D(standard) (i) = D (min) (i) / D(lobule)

[0246] D(calculate) (i) = D(standard) (i) ( i ∈ PP )

[0247] D(calculate) (i) = 1 - D(standard) (i) ( i ∈ PC )

[0248] in,

[0249] D (lobule) represents the baseline liver lobule radius;

[0250] D(min)(i) represents the minimum distance between point i and the edge of the vein contour to which it belongs;

[0251] D(standard)(i) represents the standardized minimum distance at site i;

[0252] i ∈ PP indicates that site i belongs to the periphery of the portal vein;

[0253] i ∈ PC indicates that site i belongs to the central vein periphery;

[0254] D(calculate)(i) represents the liver region coordinates of site i.

[0255] Compared with the traditional method of selecting a typical local field of view to calculate the coordinates of the liver region, the above calculation process can obtain the coordinates of most sites in the pathological image of liver tissue. The absolute distance between each point in the pathological image and the vein itself is not meaningful. It is necessary to use the radius of the liver lobule to standardize the absolute distance. The above calculation method can achieve a more accurate calculation of the baseline radius of the liver lobule.

[0256] In step S42, a liver region coordinate algorithm is established to calculate the liver region coordinates on the standardized pathological image 23, and the following optimization steps are also included:

[0257] Set 100×100 px^2 as the statistical range and 30×30 px^2 as the step size, or you can set D (lobule). / 10~ D(lobule) / 5 px^2 represents the statistical range, D (lobule) / 20 ~ D(lobule) Using a step size of 10 px^2, we statistically analyze the lesion feature data of each parsed object within the field of view of each point, and obtain the statistical results of lesion feature 58 for each point.

[0258] For parsing objects that use pathological image field blocks as pathological image units, the cutting size of the pathological image field blocks can be readjusted to D(lobule) / 20 ~ D(lobule) / 10 px^2, and pathological features can be extracted and parsed again.

[0259] Using the statistical results of lesion features 58 belonging to independent pathological image samples and the direct measurement results of liver region coordinates of corresponding sites 72, a paired dataset is formed, and a liver region coordinate prediction model 74 is trained to obtain only for the pathological image samples.

[0260] The liver region coordinate prediction model 74 can select a fast-training model architecture such as xgboost, random forest, etc., to shorten the training and inference parsing time of the liver region coordinate prediction model for various pathological images.

[0261] Using the statistical results of the lesion features at each point in the pathological image 58, the liver region coordinate prediction model 74 is used to predict the predicted liver region coordinates at each point in the pathological image 75.

[0262] Specifically, due to the significant differences in lesion features across different liver regions in pathological images of various liver tissues, a paired dataset is used to train the liver region coordinate prediction model 74, rather than integrating statistical results of lesion features 58 from a large number of pathological image samples with directly measured liver region coordinates 72. This paired dataset, belonging only to independent pathological image samples, allows for rapid training to obtain a liver region coordinate prediction model 74 specific to those pathological image samples. This approach provides a more accurate model prediction of liver region coordinates 75 for each specific pathological image sample, avoiding the influence of differences in liver region lesion features between pathological image samples on the prediction of liver region coordinates 75.

[0263] The model's predicted liver region coordinates were corrected using smoothing algorithms such as Gaussian blurring, resulting in a smoother outcome.

[0264] The directly measured liver region coordinates 72 and the model-predicted liver region coordinates 75 are weighted and merged to obtain the full-image liver region coordinates 76 for any location in the standardized pathological image 23.

[0265] The weight matrix w is calculated based on the liver region coordinates obtained by direct measurement. It is used to weight and merge the liver region coordinates 72 obtained by direct measurement of each point with the liver region coordinates 76 predicted by the liver region coordinate prediction model 74, thereby further improving the accuracy of the liver region coordinates 76.

[0266] The formula for calculating the weighted and merged coordinates of the liver region in the entire image (76) is as follows:

[0267]

[0268] D(i) = w × D(alculate)(i) + (1 - w) × D(predict)(i)

[0269] in,

[0270] D(predict)(i) represents the model prediction of the liver region coordinates at site i;

[0271] abs() represents absolute value calculation;

[0272] w represents the weighting coefficient;

[0273] D(i) represents the coordinates of the entire liver region at site i.

[0274] Compared with the method of directly measuring the liver region coordinates, the above method is used to construct a liver region coordinate prediction model and predict the liver region coordinates by analyzing the lesion characteristics of the object. This can obtain the liver region coordinate results of all sites in the liver tissue pathology image. Since the histopathological image is a two-dimensional plane, it is difficult to accurately reflect the liver region coordinate results in the three-dimensional tissue by relying on direct distance measurement in the two-dimensional plane. However, based on the lesion characteristics of each point in the pathology image, more accurate liver region coordinate results can be predicted.

[0275] Within the pathological images, a weighted merging method is used to combine the liver region coordinates obtained from direct measurements with the liver region coordinates predicted by the model. For sites closer to the vein contour edge, the directly measured liver region coordinates are more accurate; however, due to edge effects in the model, the model-predicted liver region coordinates for sites closer to the vein contour edge are less accurate. Conversely, for sites farther from the vein contour edge, the directly measured liver region coordinates are extremely inaccurate, while the predicted liver region coordinates are more accurate. Therefore, this method calculates weighting coefficients to merge the more accurate portions of both direct measurements and model predictions, obtaining accurate liver region coordinates for the entire pathological image.

[0276] Figure 8 This is a schematic diagram of the liver region coordinate calculation results according to an embodiment of the present invention.

[0277] like Figure 8 As shown, the standardized pathological image 23 is obtained by calculating the liver region coordinates 23 through the liver region coordinate algorithm established in step S4, and the corresponding direct measurement liver region coordinates 72, model predicted liver region coordinates 75, and full-image liver region coordinates 76.

[0278] In this embodiment, a liver region coordinate prediction model 74 is constructed using lesion features, which can predict the liver region coordinates of all points in the entire liver tissue pathological image. This effectively overcomes the problems of subjectivity, inaccurate quantification, and difficulty in full-image analysis in existing liver region coordinate analysis methods.

[0279] Figure 9 This is a schematic diagram illustrating the spatial distribution analysis process of lesion features in the liver region according to an embodiment of the present invention.

[0280] like Figure 9 As shown, in step S5, the variation distribution of the lesion features 58 of the analyzed object with the spatial coordinates of the liver region provided by the model prediction liver region coordinate result 76 is statistically analyzed to obtain the spatial distribution analysis result 94 of the lesion features of the liver region of the standardized pathological image 23.

[0281] Figure 10 The spatial distribution analysis result of the lesion features in the liver region is shown in Figure 94, which is an embodiment of the present invention.

[0282] like Figure 10 As shown, each sector represents a different lesion feature of the analyzed object; the center of the polygon represents the central vein side, and the outer side represents the portal vein side; the color of each region represents the quantification result of the lesion feature, and the color changes from blue to green to yellow to red, corresponding to the lesion feature intensity from low to high.

[0283] Furthermore, the standardized pathological image 23 and the corresponding spatial distribution analysis results 94 of the liver region with lesion features are used as a paired dataset to train the fast analysis model 95. The fast analysis model 95 is used to quickly predict any standardized pathological image 23 of liver tissue and obtain the corresponding spatial distribution analysis results 94 of the liver region with lesion features.

[0284] In one specific embodiment, step S5 includes the following steps:

[0285] Step S51: Statistical analysis of the spatial distribution of lesion characteristics in the liver region.

[0286] Step S52: Construction of a rapid analytical model for the spatial distribution of the liver region.

[0287] In step S51, the distribution of lesion features 58 of the analyzed object is statistically analyzed along with the changes in liver region coordinates 76 in the entire image, obtaining the analysis result 94 of the spatial distribution of lesion features of the standardized pathological image 23 along with the liver region. The above analysis process is used to achieve the spatial distribution analysis of microscopic lesion features of any standardized pathological image 23 of liver tissue within the liver region.

[0288] In step S52, a rapid analysis model 95 can be further constructed to quickly predict the analysis results 94 of the spatial distribution of lesion features with the liver region, relying solely on standardized pathological images 23 and skipping each analysis step.

[0289] Step S52 specifically includes the following steps:

[0290] Step S521, data preprocessing.

[0291] A paired dataset is constructed using standardized pathological images 23 of size 512×512 pixels and the corresponding spatial distribution analysis results of lesion features in the liver region 94 for training the fast analysis model 95.

[0292] Step S522: Quickly analyze and build the model architecture.

[0293] The fast parsing model 95 can be based on the internal region segmentation model 34 architecture of the parsing object in the aforementioned step S2211. It retains the encoder part of the model architecture and replaces the decoder with a neural network structure with multiple neural layers and ReLU activation function. The data format of the parsing result 94 of the spatial distribution of lesion features in the liver area corresponds to the terminal neural layer.

[0294] Mean squared error loss is used to measure the difference between the model's output and the results of the distribution of microscopic lesions in the liver region, and is used to train and adjust the model parameters.

[0295] Using a trained fast analysis model 95, it is possible to quickly analyze the lesion features and spatial distribution of the liver region in any liver tissue pathological image 94.

[0296] The present invention also provides a system for analyzing the spatial distribution of lesion features in liver tissue pathological images with respect to the liver region. This system can realize the steps of the above-mentioned method for analyzing the spatial distribution of lesion features in liver tissue pathological images with respect to the liver region. The system includes a liver tissue pathological image dataset establishment and standardization module, an analysis object acquisition and feature extraction module, a liver lobule object segmentation module, a liver region coordinate analysis module, and a liver region spatial distribution analysis module.

[0297] The liver tissue pathological image dataset establishment and standardization module establishes a liver tissue pathological image dataset and performs standardization processing on the pathological image dataset to obtain a standardized pathological image dataset.

[0298] The parsing object acquisition and feature extraction module can segment and acquire parsing objects in standardized pathological images; it constructs a lesion feature extraction and parsing algorithm to extract lesion features from the parsing objects.

[0299] The liver lobule object segmentation module uses a liver lobule object segmentation model to segment liver lobule objects in standardized pathological images, providing location information of the liver lobule objects.

[0300] The liver region coordinate analysis module establishes a liver region coordinate algorithm, calculates the liver region coordinates of each point in the pathological image using the position information of the liver lobule object, and obtains the liver region coordinate results, which reflect the physiological position of each point in the standardized pathological image within the liver lobule.

[0301] The liver region spatial distribution analysis module statistically analyzes the distribution of lesion features of each analysis object with changes in liver region coordinates, and obtains the analysis results of the spatial distribution of lesion features with liver region in standardized pathological images; or it uses a fast analysis model to predict any liver tissue pathological image and obtain the analysis results of the spatial distribution of corresponding lesion features with liver region.

[0302] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for analyzing the spatial distribution of lesion characteristics in a liver histopathological image, characterized in that, The method comprises the following steps: segmenting and collecting the analysis objects in the pathological image, and extracting the lesion characteristics of the analysis objects; segmenting the liver lobule objects in the pathological image to obtain the position information of the liver lobule objects in the pathological image; establishing a liver area coordinate algorithm, and calculating the liver area coordinates of each site in the pathological image by using the position information of the liver lobule objects to obtain the liver area coordinate results; The method comprises the following steps: calculating the minimum distance between the profile edges of each hepatic portal vein and the adjacent central vein profile edges in the liver lobule scale object segmentation mask graph corresponding to the pathological image, calculating the median value or the average value of the minimum distance as the reference liver lobule radius of the pathological image; taking the reference liver lobule radius as the radiation coverage range of the hepatic portal vein profile and the central vein profile, and calculating the minimum distance between each site in the radiation coverage range and the profile edge of the hepatic portal vein or the profile edge of the central vein; using the reference liver lobule radius as a standard distance unit, correcting the minimum distance between each site and the profile edge of the hepatic portal vein or the profile edge of the central vein to obtain the standardized minimum distance of each site; for the site belonging to the central vein profile, the liver area coordinate is equal to the standard distance unit of the site minus the standardized minimum distance of the site; for the site belonging to the hepatic portal vein profile, the liver area coordinate is equal to the standardized minimum distance of the site; statistically analyzing the distribution of the lesion characteristics of the analysis objects with the liver area coordinates to obtain the analysis result of the spatial distribution of the lesion characteristics of the pathological image.

2. The method according to claim 1, wherein the method is characterized by, The method comprises the following steps: annotating the analysis objects in the pathological image, and training an analysis object segmentation model; obtaining the analysis object segmentation mask graph corresponding to the pathological image by using the analysis object segmentation model; cutting the corresponding pathological image by using the analysis object segmentation mask graph to obtain the pathological image unit of each analysis object; for the analysis objects without clear boundaries, performing field block cutting on the pathological image as the pathological image unit of the analysis objects.

3. The method according to claim 1, wherein the method is characterized by, The analysis objects include liver parenchymal cells, inflammatory cells, arteries, veins, bile ducts, fibrosis regions, and field blocks of the pathological image.

4. The method according to claim 2, wherein the method is characterized by, The method for extracting the lesion characteristics of the analysis objects comprises the following steps: constructing a morphological algorithm to extract the interpretable features of the analysis objects; integrating the pathological image units of the analysis objects to form a pathological image unit data set, training an unsupervised feature extraction model, and extracting the deep learning features of the pathological image units; constructing a downstream evaluation algorithm to perform downstream evaluation on the interpretable features and the deep learning features of the analysis objects, and calculating to obtain the downstream evaluation result features; the interpretable features, the deep learning features, and the downstream evaluation result features of each analysis object are collectively used as the lesion characteristics of the analysis objects.

5. The method according to claim 4, wherein the method is characterized by, The method for constructing a morphological algorithm to extract the interpretable features of the analysis objects comprises the following steps: annotating the internal regions of the analysis objects, training an analysis object internal region segmentation model to output an analysis object internal region segmentation mask graph corresponding to the pathological image; cutting the corresponding analysis object internal region segmentation mask graph by using the analysis object segmentation mask graph to obtain the internal region segmentation mask image unit of the analysis object. The shape parameters, quantity parameters, distance parameters, color parameters and data statistical parameters of the analysis object and its internal regions are calculated as the interpretable features of the analysis object by using the pathological image unit of the analysis object and the corresponding internal region segmentation mask image unit.

6. The lesion feature liver region spatial distribution analysis method of liver histopathological images according to claim 5, characterized in that, The internal regions of the analysis object include the blank lipid droplet region, the blank background cavity region, the red phase normal staining cytoplasm region, the red phase light staining cytoplasm region, the blue phase normal staining nucleus region, the blue phase deep staining nucleus region and the heterochromatic staining region of the pathological image.

7. The method according to claim 6, wherein the method is characterized by, The hepatic lobule object includes the hepatic portal vein contour and the central vein contour.

8. The method according to claim 7, wherein the method is characterized by, The hepatic lobule object in the pathological image is segmented, including the following optimization steps: The hepatic lobule object in the pathological image is labeled to form a hepatic lobule object labeled data set; The internal region segmentation mask of the analysis object and the analysis object segmentation mask are merged to form a merged segmentation mask, and specific segmentation regions of the merged segmentation mask are screened and retained, and resolution compression is performed to form a refined segmentation mask; The hepatic lobule object segmentation model is trained by using the refined segmentation mask and the corresponding hepatic lobule object labeled data set to obtain the corresponding hepatic lobule object segmentation mask of any pathological image.

9. The method according to claim 8, wherein the method is characterized by, The hepatic zone coordinate algorithm is established, the position information of the hepatic lobule object is used to calculate the hepatic zone coordinates of each site in the pathological image, and the hepatic zone coordinate result is obtained, and the following steps are further included: The lesion feature data of each analysis object in the peripheral field of view of each site of the pathological image is counted to obtain the lesion feature statistical result of each site of the pathological image; The hepatic zone coordinate prediction model is trained by using the lesion feature statistical result and the hepatic zone coordinate result of the corresponding site; The hepatic zone coordinate prediction model is used to predict the hepatic zone coordinate result of each site of the pathological image based on the lesion feature statistical result, and the full-image predicted hepatic zone coordinate result of the pathological image is obtained.

10. The method according to claim 9, wherein the method is characterized by, The hepatic zone coordinate algorithm is established, the position information of the hepatic lobule object is used to calculate the hepatic zone coordinates of each site in the pathological image, and the hepatic zone coordinate result is obtained, and the following steps are further included: The weight matrix is calculated according to the directly measured hepatic zone coordinate result, which is used to weight and merge the directly measured hepatic zone coordinate result of each site and the full-image predicted hepatic zone coordinate result predicted by the hepatic zone coordinate prediction model.

11. The method according to any one of claims 1-10, wherein the method is characterized by, Further including the following steps: The pathological image and the corresponding lesion feature hepatic zone spatial distribution analysis result are used as a paired data set to train a rapid analysis model; The rapid analysis model is used to predict the lesion feature hepatic zone spatial distribution analysis result of any liver tissue pathological image.

12. A liver lesion feature liver region spatial distribution analysis system for liver histopathological images, characterized in that, The system for implementing the lesion feature hepatic zone spatial distribution analysis method of the liver tissue pathological image according to any one of claims 1-10, the system comprises: An analysis object acquisition and feature extraction module which segments the pathological image and acquires analysis objects and extracts lesion features of the analysis objects; A hepatic lobule object segmentation module which segments the hepatic lobule object in the pathological image to obtain the position information of the hepatic lobule object in the pathological image; A hepatic zone coordinate analysis module which establishes a hepatic zone coordinate algorithm and calculates the hepatic zone coordinates of each site in the pathological image by using the position information of the hepatic lobule object to obtain a hepatic zone coordinate result. The liver region spatial distribution analysis module statistically analyzes the change distribution of the lesion characteristics of the analysis object with the liver region coordinates, and obtains the lesion characteristic spatial distribution analysis result of the pathological image. The liver region spatial distribution analysis module statistically analyzes the change distribution of the lesion characteristics of the analysis object with the liver region coordinates, and obtains the lesion characteristic spatial distribution analysis result of the pathological image.

Images