A digital pathological image segmentation method and system for esophageal cancer tissue in thoracic surgery

Abstract

The present invention relates to the field of medical image segmentation technology, and in particular to a method and system for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery. The method uses an initial U-Net model to segment a region of interest in a sample image, quantifies the degree of infiltration based on the discrete amount of the cell direction in the extended area of ​​the region of interest, and identifies the abnormal extended area. Based on the local tissue fiber orientation and cell density on the main migration path, a comprehensive infiltration resistance coefficient is obtained. The abnormal extended area is divided into multiple window areas, and the cascade constraint fusion coefficient is quantified, thereby dynamically regulating the segmentation edge retention parameters of the U-Net model decoder. The present invention integrates the histological characteristics of the case and the microenvironment dynamic constraints into the segmentation process of the U-Net model, and has a strong segmentation accuracy for transition areas with high morphological heterogeneity and invasive growth.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a method and system for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery. Background Art

[0002] Current digital pathology image segmentation of esophageal cancer tissue primarily relies on deep convolutional neural networks to automatically segment lesions based on distinct pixel features. However, because esophageal cancer tissue exhibits a high degree of morphological and textural heterogeneity, and the junction between the cancerous region and the surrounding stromal tissue often exhibits invasive growth, existing segmentation-based methods have limitations when addressing this transitional region. In particular, existing techniques are prone to producing overly smooth or jagged boundaries when identifying the boundaries of cancerous regions, resulting in an inability to accurately capture the detailed features of the cancerous infiltration, thereby losing important morphological information. Summary of the Invention

[0003] To address the technical problem that the existing digital pathology image segmentation results of esophageal cancer tissue produce overly smooth or jagged boundaries, resulting in the inability to accurately capture the detailed features of cancerous infiltration, the present invention aims to provide a method and system for digital pathology image segmentation of esophageal cancer tissue in thoracic surgery. The technical solutions adopted are as follows:

[0004] The present invention proposes a method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery, the method comprising:

[0005] Inputting a sample image of esophageal cancer into a pre-trained U-Net model to determine a region of interest in the sample image; obtaining an extended region of the region of interest, and obtaining an infiltration degree of the extended region based on a discrete degree of cell orientation distribution at a local position of the extended region; and screening out abnormal extended regions in all extended regions of the region of interest based on the infiltration degree;

[0006] In the abnormal expansion area, the synergy of each cell is obtained based on the difference in cell direction distribution between each cell and adjacent cells. Seed cells are selected based on the synergy and streamline tracking is performed to obtain the main migration path of cancerous infiltration;

[0007] Sampling points are set in the main migration path, and the local tissue fiber direction is determined based on the distribution characteristics of collagen fiber pixels within a preset neighborhood of the sampling point. The comprehensive invasion resistance coefficient at the sampling point location is obtained based on the variance difference between the local tissue fiber direction and the direction of the main migration path at the sampling point location, as well as the cell density at the sampling point location;

[0008] Divide the abnormal expansion area into multiple window areas, and obtain a cascade constraint fusion coefficient for each window area based on the comprehensive infiltration resistance coefficient of the sampling point closest to the window area and the infiltration degree of the abnormal expansion area;

[0009] The initial segmentation edge retention parameters in the U-Net model are adjusted according to the cascade constraint fusion coefficient to obtain the adaptive segmentation edge retention parameters of each window area, and the U-Net model is used to perform image segmentation.

[0010] Furthermore, the method for obtaining the wettability includes:

[0011] Uniformly selecting boundary pixels on the boundary of the extended region, obtaining the gradient direction of the selected boundary pixels, and constructing a local window with the gradient direction as the central axis of the window; obtaining the difference in cell direction distribution discreteness between adjacent local windows with the same gradient direction, and obtaining a distance weight based on the distance between each local window and the boundary of the extended region;

[0012] The distance weights of all local windows are used to perform weighted averaging on the differences in the discreteness of the cell direction distribution to obtain the clarity of the expanded area, and the clarity is negatively correlated with the mapping to obtain the infiltration degree.

[0013] Furthermore, the method for obtaining the difference in cell direction distribution discreteness includes:

[0014] The major axis direction angle of the minimum circumscribed ellipse of the cell nucleus region is taken as the cell direction angle; the information entropy of the cell direction angle in the local window is taken as the discreteness of the cell direction distribution; in a gradient direction, starting from the local window farthest from the boundary pixel point, the differences in the discreteness of the cell direction distribution between adjacent local windows in the gradient direction are obtained in sequence.

[0015] Furthermore, the method for obtaining the distance weight includes:

[0016] The distance between the center point of the local window and the boundary pixel point corresponding to the gradient direction is taken as the window distance; the variance of all window distances is taken as a reference value; for any local window, the ratio of the square of the window distance of the local window to twice the reference value is taken as the distance feature of the local window, the distance feature is negatively correlated and normalized to obtain the distance weight.

[0017] Furthermore, the method for obtaining the synergy includes:

[0018] Voronoi-Delaunay double partitioning is used to establish an adjacency network of cells in the abnormal expansion area, and the neighboring cells of each cell are obtained based on the adjacency network. For each cell, the average cell direction difference between the cell and all neighboring cells is obtained, and the average cell direction difference is negatively correlated with the difference to obtain the synergy of each cell.

[0019] Furthermore, the method for obtaining the local tissue fiber orientation includes:

[0020] According to the preset hue range, saturation range and brightness range of the collagen fibers, a collagen fiber area composed of collagen fiber pixels is screened out in the abnormal extension area; an 8-directional Gabor filter group is used to perform convolution processing on the collagen fiber area to obtain the maximum response direction of each collagen fiber pixel; a histogram of the maximum response directions of the collagen fiber pixels within the preset neighborhood of the sampling point is constructed, and the maximum response direction corresponding to the main peak of the histogram is selected as the local tissue fiber direction.

[0021] Furthermore, the method for obtaining the comprehensive wetting resistance coefficient includes:

[0022] The cell nucleus density within the neighborhood of the sampling point is taken as the cell density at the sampling point;

[0023] The acute angle between the direction of the local tissue fibers and the direction of the main migration path at the sampling point is obtained; and the sum of the square of the sine value of the acute angle and the cell density is used as the comprehensive infiltration resistance coefficient.

[0024] Furthermore, the method for obtaining the cascade constraint fusion coefficient includes:

[0025] For each window area, a confidence weight is obtained based on the distance between the window area and the nearest sampling point. The comprehensive infiltration resistance coefficient of the nearest sampling point is weighted using the confidence weight to obtain a weighted infiltration resistance coefficient. The product of the weighted infiltration resistance coefficient and the infiltration degree of the abnormal extension area is used as the cascade constraint fusion coefficient of each window area.

[0026] Furthermore, the method for obtaining the adaptive segmentation edge preservation parameter includes:

[0027] An increase ratio of the initial segmentation edge preservation parameter is obtained according to the cascade constraint fusion coefficient, and the increase ratio is multiplied by the initial segmentation edge preservation parameter to obtain the adaptive segmentation edge preservation parameter.

[0028] The present invention proposes a digital pathological image segmentation system for esophageal cancer tissue in thoracic surgery, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the system implements any one of the steps of a digital pathological image segmentation method for esophageal cancer tissue in thoracic surgery.

[0029] The present invention has the following beneficial effects:

[0030] The present invention first uses the initial U-Net model to segment the region of interest in the sample image. Because the present invention is aimed at extracting the invasive growth characteristics of cancerous tissue, this type of invasive growth area is a transition area, so the extended area of ​​the region of interest is obtained, and the degree of infiltration is quantified according to the discrete amount of the cell direction in the extended area and the infiltration transition area, that is, the abnormal extension area, is identified. Then, the detailed infiltration characteristics are further analyzed in the abnormal extension area, and the main migration path of the cancerous infiltration is determined according to the distribution direction of the cells. The present invention further takes into account the microenvironment dynamic constraints, based on the local tissue fiber direction and cell density on the migration main path, quantitatively characterizes the physical constraints of the tissue microstructure on the infiltration morphology, and then obtains the comprehensive infiltration resistance coefficient. The abnormal extension area is divided into multiple window areas, and a small-scale cascade analysis is performed on the transition area, and the cascade constraint fusion coefficient is quantified. Then, the segmentation edge retention parameter of the U-Net model decoder is dynamically adjusted, so that the final segmentation result improves the segmentation accuracy of the cancerous tissue boundary. The present invention integrates case histological characteristics and microenvironment dynamic constraints into the segmentation process of the U-Net model, and has strong segmentation accuracy for transition areas with high morphological heterogeneity and invasive growth. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A flow chart of a method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery provided by one embodiment of the present invention;

[0033] Figure 2 A schematic diagram of a sample image provided by an embodiment of the present invention;

[0034] Figure 3 An input image of a pre-trained U-Net model provided by one embodiment of the present invention;

[0035] Figure 4 An embodiment of the present invention provides Figure 3 The corresponding output image;

[0036] Figure 5 A schematic diagram of cell nucleus area distribution provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0037] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a method and system for digital pathological image segmentation of esophageal cancer tissue in thoracic surgery. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0038] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0039] The specific scheme of the digital pathological image segmentation method and system for esophageal cancer tissue in thoracic surgery provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0040] See also Figure 1 , which shows a flow chart of a method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery provided by one embodiment of the present invention. The method includes:

[0041] Step S1: Input a sample image of esophageal cancer into a pre-trained U-Net model to determine a region of interest in the sample image; obtain an extended region of the region of interest, and obtain the infiltration degree of the extended region based on the discreteness of the cell direction distribution at the local position of the extended region; and screen out abnormal extended regions in all extended regions of the region of interest based on the infiltration degree.

[0042] This embodiment of the present invention aims to preserve more detailed edges by adjusting the segmentation edge preservation parameters within the U-Net model. Therefore, this embodiment first uses a pre-trained U-Net model to identify regions of interest (ROIs) within a sample image. Specifically, these regions are defined as distinct cancerous areas and exclude invasive transitional areas.

[0043] It should be noted that the sample images in the embodiments of the present invention are tissue sample images of patients with esophageal cancer. Each patient provides 1 to 5 sets of paraffin-embedded serial section samples, covering the tumor core area, the invasion front, and the normal tissue area adjacent to the cancer. Each set of slice samples is scanned using a high-throughput whole-slice digital scanner (objective magnification: 40×, scanning resolution: 0.25μm / pixel) to generate a continuous high-resolution whole-slice sample image sequence. An automated algorithm is used to correct for staining differences between batches. The embodiments of the present invention use the Reinhard color normalization method to improve image comparability. Please refer to [refer to the 'Reinhard color normalization method' for details]. Figure 2 , which shows a schematic diagram of a sample image provided by an embodiment of the present invention.

[0044] It should be noted that the training method of the U-Net model is a technical content well known to those skilled in the art and will not be described in detail. The U-Net model in the embodiment of the present invention also includes a dynamic boundary perception network architecture, which automatically sends a data push request to the dynamic boundary perception network architecture after determining the region of interest, and the data transmission is performed based on the HL7 message queue protocol of the DICOM standard. The dynamic boundary perception network architecture is the architecture of the method executed by the embodiment of the present invention. By executing the embodiment of the present invention through the dynamic boundary perception network architecture, the adaptive segmentation edge retention parameters of the U-Net model in the local area of ​​the infiltration area in each sample image can be obtained, thereby achieving the retention of detail edges. Please refer to Figure 3 , which shows an input image of a pre-trained U-Net model provided by one embodiment of the present invention; please refer to Figure 4 , which shows an embodiment of the present invention provides Figure 3 The corresponding output image. Figure 4 The output image can be used as a mask to extract the region of interest in the sample image.

[0045] Because the region of interest is only the cancerous area initially segmented by the neural network, and the invasive area targeted by the embodiment of the present invention is the area generated during the transformation of the cancerous area to the normal area, during the invasive process, cancer cells lose their adhesion due to uncontrolled proliferation and invasion behavior and gain migration ability, causing the cells and arrangement in the lesion invasive area to lose polarity and form irregular nested or cord-like structures. Figure 5 , which shows a schematic diagram of cell nucleus area distribution provided by one embodiment of the present invention, Figure 5The darker area is the nuclear region, and based on the distribution of the nuclear region, it is clearly visible that it has an irregular nested or cord-like structure. Therefore, to screen for infiltrating areas, it is necessary to first expand the region of interest and then analyze the discreteness of the cell direction distribution at each local location in the expanded region. That is, the greater the discreteness, the more likely the cell distribution is to have the irregular nested or cord-like structure described above. Therefore, the infiltration degree of the expanded region can be obtained based on this. Based on the infiltration degree, abnormal expansion areas can be screened out in all expanded areas of the region of interest. That is, abnormal expansion areas are the identified infiltrating areas.

[0046] In the embodiment of the present invention, the boundary of the region of interest is used as a reference, and the image is expanded with a length unit of 50 μm in reality to obtain the expanded region. It should be noted that the calibration method of the real world coordinate system and the image coordinate system is a technical means well known to those skilled in the art and will not be described in detail here.

[0047] Preferably, in an embodiment of the present invention, the method for obtaining the wettability includes:

[0048] Boundary pixels are uniformly selected on the boundary of the extended area, and the gradient direction of the selected boundary pixels is obtained. A local window is constructed using the gradient direction as the window center axis. In an embodiment of the present invention, the spacing between boundary pixels is set to 10 μm in the world coordinate system (10 μm in the world coordinate system is approximately two cell diameters). The size of the local window is 10×10 μm. The first local window is obtained with the selected boundary pixel as the center. All local windows are obtained by continuously sliding 10 μm toward the region of interest. If the last local window contains both extended area information and region of interest information, it is removed.

[0049] The difference in cell orientation distribution dispersion between adjacent local windows in the same gradient direction is obtained. Distribution dispersion can characterize the characteristics of cell orientation distribution in a local window. A larger difference in cell orientation dispersion between adjacent local windows indicates a rapid change in tissue properties within a smaller space, resulting in a clearer boundary within the region and a non-infiltrating area. Conversely, a smaller difference indicates a smoother edge within the space, representing a fuzzy infiltration front and a higher degree of infiltration.

[0050] Furthermore, considering that in the above analysis, the edge infiltration analysis should focus on the periphery of the region of interest, the farther the local window area is from the region of interest, the weaker the reference of the analysis result should be. Therefore, the distance weight can be obtained according to the distance between each local window and the boundary of the extended area; the difference in the discreteness of the cell direction distribution is weighted and averaged using the distance weights of all local windows to obtain the clarity of the extended area. The smaller the clarity, the greater the infiltration of the extended area. Therefore, the clarity can be negatively correlated with the mapping to obtain the infiltration.

[0051] In this embodiment of the present invention, the negative correlation mapping method is to raise the inverse of the clarity to the power of an exponential function with a natural constant as the base. The output of the exponential function is the wetness. After this exponential mapping, the wetness can also be normalized to facilitate subsequent calculations.

[0052] It should be noted that, because the infiltration degree has been normalized, the greater the infiltration degree, the more likely the extended area is to be an infiltration area. Therefore, the embodiment of the present invention sets the infiltration degree threshold to 0.6, and regards the extended area with an infiltration degree greater than the infiltration degree threshold as an abnormal extended area, that is, the abnormal extended area is the infiltration area in the process of cancer cell transformation.

[0053] Furthermore, the method for obtaining the difference in cell direction distribution discreteness includes:

[0054] The embodiment of the present invention can first separate the cell nucleus region in the image according to the morphological watershed algorithm. The cell nucleus region is usually not a standard circle, and the embodiment of the present invention needs to determine the distribution direction of the cells, so the minimum circumscribed ellipse of the cell nucleus region can be further obtained. When the cell migrates, the cell nucleus will stretch along the direction of movement, that is, the long axis direction is consistent with the direction of actin flow. Here, for a specific cell, the long axis direction of its cell nucleus is regarded as its cancerous infiltration and migration direction. That is, the long axis direction angle of the minimum circumscribed ellipse of the cell nucleus region is used as the cell direction angle. Thereby, the quantification of the cell direction is achieved. It should be noted that, in the embodiment of the present invention, the cell direction angle is the angle between the long axis direction and the horizontal direction.

[0055] The information entropy of the cell orientation angles in the local window is used as the discreteness of the cell orientation distribution. The larger the information entropy, the more discrete the distribution of cell orientation angles in the local window.

[0056] In a gradient direction, starting with the local window farthest from the boundary pixel, the differences in cell direction distribution dispersion between adjacent local windows along the gradient direction are sequentially obtained. That is, no local window corresponding to a boundary pixel selected on the boundary of the extended region has a corresponding distribution dispersion difference, and each distribution dispersion difference corresponds to a local window. In this embodiment of the present invention, the difference in cell direction distribution dispersion is the absolute value of the difference in cell direction distribution dispersion.

[0057] As an example, wettability is expressed as:

[0058] ;in is the wettability of the bth expansion area, is the clarity of the bth expanded area; exp( ) is an exponential function with a natural constant as the base.

[0059] In the above formula, is a weighted average value, which can be expressed as: ;in is the number of windows excluding the local windows on the boundary of the extended region, is the distance weight of the jth local window except the local window on the boundary of the extended area, The discreteness of the cell orientation distribution of the jth local window except the local window on the boundary of the extended region, is the discreteness of the cell direction distribution of the next local window adjacent to the j-th local window along the gradient direction.

[0060] Furthermore, in an embodiment of the present invention, a method for obtaining a distance weight includes:

[0061] The distance between the center point of the local window and the boundary pixel point corresponding to the gradient direction is used as the window distance; the variance of all window distances is used as a reference value; for any local window, the ratio of the square of the window distance of the local window to twice the reference value is used as the distance feature of the local window. The distance feature is negatively correlated and normalized to obtain the distance weight. In an embodiment of the present invention, the negative correlation mapping and normalization operation is: the opposite number of the distance feature is used as the power of an exponential function with a natural constant as the base, and the output result of the exponential function is the distance weight. The method of entering the reference value and performing function mapping according to the embodiment of the present invention can effectively quantify the weight sharing of the local window spatial position on the infiltration analysis, so that the data is within a reasonable range.

[0062] As an example, in an embodiment of the present invention, the distance weight is expressed by the formula:

[0063] ;in is the distance weight of the j-th local window, is the window distance of the jth local window, is the variance of all window distances, and exp( ) is an exponential function with a natural constant as the base.

[0064] Step S2: In the abnormal expansion area, the synergy of each cell is obtained based on the difference in cell direction distribution between each cell and adjacent cells. Seed cells are screened based on the synergy and streamline tracking is performed to obtain the main migration path of cancerous infiltration.

[0065] In step S1, the abnormal expansion area is obtained based on the analysis of the tissue morphological characteristics of cancer cell tissue during the infiltration process. It is further necessary to integrate the microenvironment dynamic constraints into the segmentation process. It is necessary to determine the infiltration direction and path of the cell infiltration and migration process in the abnormal expansion area, and then determine the comprehensive infiltration resistance coefficient in the infiltration process based on the path information in the subsequent process to realize the feature extraction of the microenvironment dynamics.

[0066] During the actual invasion and migration of esophageal cancer tissue, the collective infiltration and migration of cancerous cells at the esophageal cancer invasion front exhibits a "leader-follower" behavior pattern: the leader cell guides the direction of the local cell group through actin flow, while the follower cells maintain mechanical coupling in the local area through cadherin. Therefore, within the abnormal expansion area, the synergy of each cell can be determined based on the differences in cell orientation distribution between each cell and its neighbors. Specifically, the smaller the difference in cell orientation distribution, the more likely it is to be the "leader cell" leading the migration of neighboring cells. Therefore, seed cells can be selected based on synergy, and streamline tracing based on these seed cells can reveal the main migration path of the cancerous invasion.

[0067] Preferably, in an embodiment of the present invention, the method for obtaining synergy includes:

[0068] The Voronoi-Delaunay double meshing is used to establish the adjacency network of cells in the abnormal expansion area, and the neighboring cells of each cell are obtained according to the adjacency network.

[0069] For each cell, the average cell direction difference between the cell and all adjacent cells is obtained, and the average cell direction difference is negatively correlated to obtain the synergy of each cell. It should be noted that negative correlation mapping is a technique well known to those skilled in the art and has been described in the above embodiments. The same method can be used to perform negative correlation mapping and achieve normalization, and is not limited or elaborated upon here.

[0070] In the embodiment of the present invention, the smaller the average cell direction difference, the greater the synergy of the corresponding cells. The greater the synergy, the more likely the cell is to be a "leader cell." Therefore, the synergy threshold is set to 0.7, and cells with synergy greater than the synergy threshold are used as seed cells for streamline tracking. During the streamline tracking process, the migration direction is set as the local dominant migration vector: ,in is the cell direction of the i-th seed cell; the migration vector along the seed cell Streamline tracing was performed with a step size of 10 μm. During the streamline tracing process, the cell that best matched the migration vector of the seed cell was searched for. Iterations were performed, and iterations were terminated when the streamline curvature suddenly changed by more than 15° or when it exceeded the abnormal expansion region. It should be noted that finding the cell that matched the migration vector of the seed cell directly obtained the cosine similarity of the migration vectors of adjacent cells. The adjacent cell with the greatest cosine similarity was selected as the best match. As iterations progressed, the best match was used as a new pathpoint, and matching information for adjacent cells was continuously obtained. This allowed iterations to obtain all the main migration paths in the expansion region.

[0071] Step S3: Set a sampling point in the main migration path, determine the local tissue fiber direction based on the collagen fiber pixel distribution characteristics within the preset neighborhood of the sampling point, and obtain the comprehensive invasion resistance coefficient at the sampling point based on the variance difference between the local tissue fiber direction and the migration main path direction at the sampling point position, as well as the cell density at the sampling point position.

[0072] During cancer cell invasion, the morphology of invasion is influenced by the microenvironmental structure. For example, dense normal tissue forces cancer cells to migrate in a distributed manner, forming a fragmented infiltrating layer, while loose tissue allows cancer cells to advance in sheets. In other words, dense tissue creates greater resistance to cancer cell invasion, while loose tissue creates less resistance. This allows us to quantify the resistance characteristics of various locations along the main migration pathway within the abnormal expansion region during migration.

[0073] First, for each main migration path, a sampling point is set on the path, and the sampling point is used as the target to analyze the resistance of each position of the main migration path during the infiltration migration process. As described above, the resistance can be reflected in the characteristics of the tissue morphology in the microenvironment near the sampling point. Therefore, the embodiment of the present invention uses the sampling point as the center to analyze the information within the preliminary neighborhood of the sampling point, and determines the local tissue fiber direction in the local range of the sampling point based on the distribution characteristics of the collagen fiber pixels. The greater the difference between the local tissue fiber direction and the path direction, the greater the resistance of the main migration path at the sampling point position; and the greater the cell density at the sampling point position, the more likely it is to be in a dense tissue area, and the greater the resistance. Therefore, the embodiment of the present invention further obtains the comprehensive infiltration resistance coefficient of the sampling point position based on the variance difference between the local tissue fiber direction and the migration main path direction at the sampling point position, as well as the cell density at the sampling point position.

[0074] In the embodiment of the present invention, the neighborhood range of the sampling point is an area with the sampling point as the center and a radius of 20 μm. On the migration trunk path, the sampling points are evenly arranged at intervals of 5 μm.

[0075] Preferably, in an embodiment of the present invention, the method for obtaining the orientation of local tissue fibers includes:

[0076] Based on the preset hue, saturation, and lightness intervals of the collagen fibers, a collagen fiber region consisting of collagen fiber pixels is screened out within the abnormally extended region. In embodiments of the present invention, the sample images are typically stained, and the Masson-stained images are converted to the HSV color space. A collagen fiber binary mask is extracted using thresholds of hue (200°-260°), saturation (>0.3), and lightness (0.1-0.9). That is, in the sample image, the region containing the collagen fiber binary mask represents the collagen fiber region.

[0077] The collagen fiber region is convolved using an 8-directional Gabor filter bank to obtain the maximum response direction of each collagen fiber pixel. A histogram of the maximum response directions of collagen fiber pixels within a preset neighborhood of the sampling point is constructed, with the horizontal axis of the histogram representing the type of maximum response direction and the vertical axis representing the number of collagen fiber pixels corresponding to each type of maximum response direction. The maximum response direction corresponding to the main peak of the histogram is selected as the local tissue fiber orientation.

[0078] The methods for obtaining the comprehensive wetting resistance coefficient include:

[0079] The cell nucleus density within the neighborhood of the sampling point is taken as the cell density at the sampling point. It should be noted that the cell nucleus density is the ratio of the total area of ​​the cell nucleus region to the area of ​​the neighborhood.

[0080] Obtain the acute angle between the local tissue fiber orientation and the main migration path at the sampling point. The smaller the acute angle, the more parallel the two directions are, and the lower the resistance to cell invasion. The larger the acute angle, the more perpendicular the two directions are, which means that the current cancer cell invasion and migration path needs to overcome greater structural resistance.

[0081] Because an acute angle is selected, the embodiment of the present invention uses the sine value of the acute angle for quantification. That is, the larger the acute angle, the larger the corresponding sine value. The sum of the square of the sine value of the acute angle and the cell density is used as the comprehensive invasion resistance coefficient. A larger comprehensive invasion resistance coefficient indicates that the sampling point is located in a dense tissue area, with greater cell invasion resistance.

[0082] Step S4: Divide the abnormal extension area into multiple window areas, and obtain the cascade constraint fusion coefficient of each window area according to the comprehensive infiltration resistance coefficient of the window area closest to the sampling point and the infiltration degree of the abnormal extension area.

[0083] In step S3, the comprehensive infiltration resistance coefficients at sampling points along the main migration path are quantitatively evaluated. To analyze other regions within the abnormal extension region, this embodiment of the present invention divides the abnormal extension region into multiple window regions. For each window region, the comprehensive infiltration resistance coefficient of the sampling point closest to the window region is selected as reference information. Furthermore, the cascade constraint fusion coefficient for each window region within the abnormal extension region is quantified based on the infiltration degree of the abnormal extension region itself. The cascade constraint fusion coefficient is the parameter result of the fusion of case histological characteristics and microenvironment dynamic constraints. A larger parameter indicates a greater need for detection sensitivity within the abnormal extension region, enabling the preservation of more detailed edges.

[0084] In one embodiment of the present invention, the window area is a window with a radius of 20 μm in the abnormal extension area. If the area space is less than 20 μm during the division process, the remaining area is directly used as a window area.

[0085] Preferably, in an embodiment of the present invention, the method for obtaining the cascade constraint fusion coefficient includes:

[0086] The embodiment of the present invention further considers the distance information between the window area and the sampling points used as reference information. The larger the distance, the lower the confidence of the reference information. Therefore, for each window area, a confidence weight is obtained based on the distance between the window area and the nearest sampling point. In the embodiment of the present invention, the reciprocal of this distance is used as the confidence weight. It should be noted that in the embodiment of the present invention, if the nearest sampling point is within the window area or on the boundary of the window area, the distance is 0, and the reciprocal does not need to be calculated. The confidence weight can be directly set to a positive integer of 1. If the nearest sampling point is outside the window area, the distance between the nearest sampling point and the nearest boundary of the window area is selected as the distance between the two.

[0087] The confidence weight is used to weight the comprehensive infiltration resistance coefficient of the nearest sampling point to obtain a weighted infiltration resistance coefficient. That is, the product of the confidence weight and the corresponding comprehensive infiltration resistance coefficient is used as the weighted infiltration resistance coefficient.

[0088] The product of the weighted infiltration resistance coefficient and the infiltration degree of the abnormal expansion area is used as the cascade constraint fusion coefficient of each window area.

[0089] Step S5: Adjust the initial segmentation edge preservation parameters in the U-Net model according to the cascade constraint fusion coefficient, obtain the adaptive segmentation edge preservation parameters of each window area, and use the U-Net model to perform image segmentation.

[0090] In the U-Net image segmentation method, a convolutional neural network (CNN) first extracts features from the input image. The U-Net network structure consists of two parts: an encoder and a decoder. The encoder gradually compresses the features of the input image, while the decoder restores the compressed features to the original image size. During this process, U-Net combines shallow features in the encoder with deep features in the decoder through skip connections to preserve more detailed information.

[0091] However, since esophageal cancer tissue exhibits a high degree of morphological and textural heterogeneity, and the junction between the cancerous area and the surrounding stromal tissue often shows invasive growth, existing segmentation models have limitations when processing this transition area, and are prone to produce overly smooth or jagged boundaries, resulting in the inability to accurately capture the detailed features of cancerous infiltration, thereby losing important morphological information.

[0092] Therefore, the embodiment of the present invention adjusts the initial segmentation edge preservation parameters in the U-Net model based on the cascade constraint fusion coefficient to obtain adaptive segmentation edge preservation parameters for each window area. By fusing features at different scales, it is possible to effectively extract detailed information from the image and accurately predict pixel-level segmentation results at the boundaries. To further optimize the segmentation results, all segmentation curves are sorted according to edge strength to obtain an edge strength sequence. When obtaining the edge curves of the final image, the adaptive segmentation edge preservation parameters are used to control the threshold for non-maximum suppression. This preserves more detailed edges at the invasion front of the digital pathology image of esophageal cancer tissue, and selects edge curves with edge strengths that meet the threshold and are closer to the front edge and meet the threshold as the final segmentation edge curve of the image. This effectively preserves more detailed information and reduces information loss caused by overlapping or thin edges. Furthermore, the embodiment of the present invention analyzes each window area in the extended region, providing a more detailed spatial analysis of the extended region, further improving the preservation of the invasion edge. Ultimately, the cancerous tissue structural boundary in the sample image is obtained, indicating that the cancerous tissue includes both distinct cancerous areas and infiltrating transition areas.

[0093] It should be noted that the specific edge segmentation algorithm of the U-Net model is a technical means well known to those skilled in the art and will not be described in detail here.

[0094] Preferably, in an embodiment of the present invention, the method for obtaining the adaptive segmentation edge preservation parameter includes:

[0095] An increase ratio of the initial segmentation edge preservation parameter is obtained according to the cascade constraint fusion coefficient, and the increase ratio is multiplied by the initial segmentation edge preservation parameter to obtain the adaptive segmentation edge preservation parameter.

[0096] In the embodiment of the present invention, the cascade constraint fusion coefficient is normalized and then added to a positive integer 1 to obtain an increase ratio, that is, the increase ratio is a proportional coefficient between 1 and 2.

[0097] In summary, the present invention first uses the initial U-Net model to segment the region of interest in the sample image, obtains the extended region of the region of interest, quantifies the infiltration degree according to the discrete quantity of the cell direction in the extended region, and identifies the abnormal extended region. The main migration path of cancerous infiltration is determined according to the distribution direction of the cells. Based on the local tissue fiber direction and cell density on the main migration path, a comprehensive infiltration resistance coefficient is obtained. The abnormal extended area is divided into multiple window areas, the cascade constraint fusion coefficient is quantified, and then the segmentation edge retention parameters of the U-Net model decoder are dynamically adjusted, so that the final segmentation result improves the segmentation accuracy of the cancerous tissue boundary. The present invention integrates the case histological characteristics and microenvironment dynamic constraints into the segmentation process of the U-Net model, and has a strong segmentation accuracy for transition areas with high morphological heterogeneity and invasive growth.

[0098] Based on the same inventive concept, the present invention also proposes a digital pathological image segmentation system for esophageal cancer tissue in thoracic surgery, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the system implements any one of the steps of the digital pathological image segmentation method for esophageal cancer tissue in thoracic surgery.

[0099] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A digital pathological image segmentation method for esophageal cancer tissue in thoracic surgery, characterized in that: The method comprises: Inputting a sample image of esophageal cancer into a pre-trained U-Net model to determine a region of interest in the sample image; obtaining an extended region of the region of interest, and obtaining an infiltration degree of the extended region based on a discrete degree of cell orientation distribution at a local position of the extended region; and screening out abnormal extended regions in all extended regions of the region of interest based on the infiltration degree; In the abnormal expansion area, the synergy of each cell is obtained based on the difference in cell direction distribution between each cell and adjacent cells. Seed cells are selected based on the synergy and streamline tracking is performed to obtain the main migration path of cancerous infiltration; Sampling points are set in the main migration path, and the local tissue fiber direction is determined based on the distribution characteristics of collagen fiber pixels within a preset neighborhood of the sampling point. The comprehensive invasion resistance coefficient at the sampling point location is obtained based on the variance difference between the local tissue fiber direction and the direction of the main migration path at the sampling point location, as well as the cell density at the sampling point location; Divide the abnormal expansion area into multiple window areas, and obtain a cascade constraint fusion coefficient for each window area based on the comprehensive infiltration resistance coefficient of the sampling point closest to the window area and the infiltration degree of the abnormal expansion area; Adjusting the initial segmentation edge retention parameters in the U-Net model according to the cascade constraint fusion coefficient to obtain the adaptive segmentation edge retention parameters of each window area, and performing image segmentation using the U-Net model; The method for obtaining the synergy includes: A Voronoi-Delaunay double partitioning method is used to establish an adjacency network of cells in the abnormal expansion region. The neighboring cells of each cell are obtained based on the adjacency network. For each cell, the average cell direction difference between the cell and all neighboring cells is obtained. The average cell direction difference is negatively correlated with the difference to obtain the synergy of each cell. The method for obtaining the comprehensive wetting resistance coefficient includes: The cell nucleus density within the neighborhood of the sampling point is taken as the cell density at the sampling point; Obtaining the acute angle between the direction of local tissue fibers and the direction of the main migration path at the sampling point; taking the sum of the square of the sine value of the acute angle and the cell density as the comprehensive invasion resistance coefficient; The method for obtaining the cascade constraint fusion coefficient includes: For each window area, a confidence weight is obtained based on the distance between the window area and the nearest sampling point. The comprehensive infiltration resistance coefficient of the nearest sampling point is weighted using the confidence weight to obtain a weighted infiltration resistance coefficient. The product of the weighted infiltration resistance coefficient and the infiltration degree of the abnormal extension area is used as the cascade constraint fusion coefficient of each window area.

2. A method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery according to claim 1, characterized in that: The method for obtaining the wettability includes: Uniformly selecting boundary pixels on the boundary of the extended region, obtaining the gradient direction of the selected boundary pixels, and constructing a local window with the gradient direction as the central axis of the window; obtaining the difference in cell direction distribution discreteness between adjacent local windows with the same gradient direction, and obtaining a distance weight based on the distance between each local window and the boundary of the extended region; The distance weights of all local windows are used to perform weighted averaging on the differences in the discreteness of the cell direction distribution to obtain the clarity of the expanded area, and the clarity is negatively correlated with the mapping to obtain the infiltration degree.

3. The method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery according to claim 2, characterized in that: The method for obtaining the difference in cell direction distribution discreteness includes: The major axis direction angle of the minimum circumscribed ellipse of the cell nucleus region is taken as the cell direction angle; the information entropy of the cell direction angle in the local window is taken as the discreteness of the cell direction distribution; in a gradient direction, starting from the local window farthest from the boundary pixel point, the differences in the discreteness of the cell direction distribution between adjacent local windows in the gradient direction are obtained in sequence.

4. The method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery according to claim 2, characterized in that: The method for obtaining the distance weight includes: The distance between the center point of the local window and the boundary pixel point corresponding to the gradient direction is taken as the window distance; the variance of all window distances is taken as a reference value; for any local window, the ratio of the square of the window distance of the local window to twice the reference value is taken as the distance feature of the local window, the distance feature is negatively correlated and normalized to obtain the distance weight.

5. The method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery according to claim 1, characterized in that: The method for obtaining the local tissue fiber orientation includes: According to the preset hue range, saturation range and brightness range of the collagen fibers, a collagen fiber area composed of collagen fiber pixels is screened out in the abnormal extension area; an 8-directional Gabor filter group is used to perform convolution processing on the collagen fiber area to obtain the maximum response direction of each collagen fiber pixel; a histogram of the maximum response directions of the collagen fiber pixels within the preset neighborhood of the sampling point is constructed, and the maximum response direction corresponding to the main peak of the histogram is selected as the local tissue fiber direction.

6. The method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery according to claim 1, characterized in that: The method for obtaining the adaptive segmentation edge preservation parameter includes: An increase ratio of the initial segmentation edge preservation parameter is obtained according to the cascade constraint fusion coefficient, and the increase ratio is multiplied by the initial segmentation edge preservation parameter to obtain the adaptive segmentation edge preservation parameter.

7. A digital pathological image segmentation system for esophageal cancer tissue in thoracic surgery, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method for segmenting digital pathological images of esophageal cancer tissue in thoracic surgery as described in any one of claims 1 to 6 are implemented.

Images