Liver grid anatomical landmark segmentation method based on nested resolution grid-graph convolution

Abstract

The invention relates to a liver grid anatomical landmark segmentation method based on nested resolution grid-graph convolution, and the method comprises the following steps: S1, obtaining high-resolution liver grid data, and obtaining low-resolution grid data through random edge sampling; s2, taking the low-resolution grid data as an input feature of a dynamic graph convolutional network to obtain a low-resolution anatomical landmark preliminary segmentation result; s3, expansion mapping and erosion mapping processing are carried out on the preliminary segmentation result; and S4, based on the expansion mapping data and the erosion mapping data after mapping processing, in an anatomical landmark refining network based on MeshConv, outputting a high-resolution segmentation result through attention fusion. By adopting a nesting resolution grid-graph convolution architecture, the method aims to solve key problems existing in automatic segmentation of liver 3D surface anatomical landmarks in an existing method, so that the accuracy and robustness of segmentation are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a liver grid anatomical landmark segmentation method based on nested resolution grid-graph convolution. Background Art

[0002] Accurate segmentation is crucial for liver image segmentation, and the detection and segmentation of 3D anatomical landmarks of the liver, such as the falciform ligament and hepatic ridge, is key. Traditional methods rely primarily on manual annotation, which is not only time-consuming and labor-intensive but also subject to significant subjective errors, making it difficult to meet high-precision requirements.

[0003] Existing deep learning-based methods have improved the automation level of detection to a certain extent, but some problems still exist. For example, PointNet++ only uses the coordinate information of points and lacks the ability to model the mesh topology; MeshCNN only performs convolution based on mesh edge features and cannot fully capture the global geometric structure of the liver. In particular, most existing methods use a fixed-resolution mesh pooling strategy. When processing liver mesh data of different resolutions, it is prone to information loss or feature over-smoothing, resulting in limited generalization of the model. Therefore, there is an urgent need for a new method to improve the accuracy and generalization of 3D anatomical landmark segmentation of the liver to meet the actual needs of clinical liver image segmentation.

[0004] The paper "Pointnet++: Deep hierarchical feature learning on point sets in ametric space" addresses the issues of generating point set partitions and extracting point set features or local features through local feature learners. By using a hierarchical local feature learner, features of larger local regions are gradually abstracted along the hierarchy. Density-adaptive Pointnet layers are then employed to achieve robust feature learning under non-uniform sampling density. However, these methods all suffer from insufficient consideration of liver data characteristics when addressing the task of segmenting anatomical landmarks on the liver surface, making them difficult to adapt to the liver's shape, appearance variations, and lack of surface texture. Faced with class imbalance, they are unable to accurately identify and segment small landmark regions. In terms of information fusion, these methods fail to effectively balance global geometry and local topology, resulting in poor performance when processing complex liver meshes. Furthermore, their generalization capabilities are limited. When encountering liver meshes from different datasets or resolutions that differ significantly from the training data, segmentation performance deteriorates significantly.

[0005] In summary, the existing technology has the following shortcomings:

[0006] (1) Limited detection accuracy: Existing methods are difficult to accurately detect liver surface landmarks. For example, point cloud-based methods tend to ignore the mesh topology structure, and graph convolutional network (GCN)-based methods are difficult to fully capture the complex features of the liver. The MeshCNN method has defects in learning the global geometric structure, resulting in the overall detection accuracy being unable to meet actual needs.

[0007] (2) Poor generalization ability: Most of them adopt a fixed resolution processing method. When faced with liver mesh data of different resolutions, processing high-resolution meshes is prone to losing details, and processing low-resolution meshes may cause excessive smoothing of features. They have poor adaptability and stability on different data sets and cannot effectively cope with the complex and diverse liver morphology.

[0008] (3) Low computational efficiency: When processing liver meshes containing a large number of edges, such as the MeshCNN method, a large number of convolution and pooling operations are required, which has high computational complexity and limited receptive field coverage. It is difficult to meet real-time requirements when processing large-scale data, which affects the actual application effect. Summary of the Invention

[0009] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a liver mesh anatomical landmark segmentation method based on nested resolution grid-graph convolution to solve or partially solve the problems of limited detection accuracy, poor generalization ability and low computational efficiency.

[0010] The purpose of the present invention can be achieved by the following technical solutions:

[0011] In one aspect of the present invention, a method for segmenting liver mesh anatomical landmarks based on nested resolution grid-graph convolution is provided, comprising the following steps:

[0012] Step S1, obtaining high-resolution liver mesh data, and obtaining low-resolution mesh data by random edge sampling;

[0013] Step S2, using the low-resolution grid data as input features of a dynamic graph convolutional network to obtain a low-resolution preliminary segmentation result of anatomical landmarks;

[0014] Step S3, performing dilation mapping and erosion mapping processing on the preliminary segmentation result respectively;

[0015] In step S4, based on the dilated map data and the eroded map data after mapping processing, a high-resolution segmentation result is output through attention fusion in the MeshConv-based anatomical landmark refinement network.

[0016] As a preferred technical solution, step S1 includes the following steps:

[0017] For the high-resolution liver mesh data, the mesh is downsampled to a low-resolution mesh with a preset number of edges through random edge sampling, and the center coordinates of the edges in the low-resolution mesh are extracted to form low-resolution mesh data.

[0018] As a preferred technical solution, step S2 includes the following steps:

[0019] Step S201, constructing a local neighborhood graph by identifying the nearest neighboring points of each edge based on the low-resolution grid data;

[0020] Step S202, encoding the local topological relationship by performing spatial transformation on the local neighborhood graph;

[0021] Step S203: extract local features of the local topological relationship through multiple dynamic graph convolution layers, wherein for each dynamic graph convolution layer, the adjacency relationship is recalculated to capture geometric features of different scales;

[0022] In step S204, the outputs of each dynamic graph convolution layer are integrated and spliced with the low-resolution grid data. A classification probability matrix is output through multi-layer perceptron-based classification, and an anatomical landmark category is assigned to each edge as the preliminary segmentation result of the anatomical landmark.

[0023] As a preferred technical solution, step S3 includes the following steps:

[0024] Step S301 , performing one-hot encoding on the preliminary segmentation result, mapping the low-resolution preliminary segmentation result to a high-resolution image through grid inverse pooling, and generating a probability propagation map;

[0025] Step S302: performing dilation mapping and erosion mapping based on the probability propagation map.

[0026] As a preferred technical solution, the expansion mapping is:

[0027] If the probability propagation value of an edge is greater than a preset first threshold, the edge is determined to belong to the target landmark category area;

[0028] The erosion map is:

[0029] If the probability propagation value of an edge is greater than a preset second threshold, it is determined that the edge belongs to the target landmark category area, wherein the second threshold is greater than the first threshold.

[0030] As a preferred technical solution, step S4 includes the following steps:

[0031] Step S401: Using the original high-resolution liver mesh edge features, mesh resolution information, dilation mapping data, and erosion mapping data as input to the anatomical landmark refinement network, a multi-sensor machine is used to extract feature vectors from the original high-resolution liver mesh edge features and mesh resolution information. These feature vectors are then concatenated with the dilation mapping data and the erosion mapping data, respectively, to serve as inputs to the dilation mapping branch and the erosion mapping branch.

[0032] Step S402: extracting local topological relationships using MeshConv for the dilation mapping branch and the erosion mapping branch respectively;

[0033] Step S403 : weighted fusion of the dilation mapping branch and the erosion mapping branch is performed through a fine-grained hierarchical aggregation mechanism to obtain a final segmentation result and the segmentation results of the dilation mapping branch and the erosion mapping branch.

[0034] As a preferred technical solution, in step S403, the following formula is used to perform weighted fusion on the expansion mapping branch and the erosion mapping branch:

[0035] Among them, W D 、W E are the response weights of the dilation mapping branch and the erosion mapping branch, respectively, D 、F E is the feature representation of the dilation mapping branch and the erosion mapping branch, Stands for element-wise multiplication.

[0036] As a preferred technical solution, the dynamic graph convolutional network is pre-trained for joint optimization based on Dice loss and cross entropy loss.

[0037] As a preferred technical solution, the anatomical landmark refinement network is pre-trained with the goal of minimizing the loss function value, wherein the loss function includes the dilation branch loss, the erosion branch loss, and the supervision loss of the final fusion feature. For any of the dilation branch loss, the erosion branch loss, and the supervision loss, the cross entropy loss and the Dice loss based on the anatomical perception of the edge length weighting are included. The Dice loss based on the anatomical perception of the edge length weighting is calculated using the following formula:

[0038]

[0039]

[0040] Among them, l i For edge e i length, is the average length of all edges under the current anatomical landmark category, p i is the predicted value, gi is the true value, is the anatomically-aware Dice loss based on edge length weighting, To consider the loss of anatomical co-occurrence characteristics, Ligament and Ridge represent the falciform ligament and liver ridge, respectively. represents the sum of the Dice losses of the falciform ligament and the hepatic ridge.

[0041] Another aspect of the present invention provides an electronic device comprising one or more processors, a memory, and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the aforementioned liver mesh anatomical landmark segmentation method based on nested resolution grid-graph convolution.

[0042] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0043] (1) Overcoming the imbalance between global and local information processing: The present invention constructs a mesh-graph convolutional model (Nested Resolution Mesh-Graph CNN, NR-MGCN, NR-MGCN) consisting of a dynamic graph convolutional network and an anatomical landmark refinement network based on MeshConv. By combining the advantages of DGCNN and MeshConv, DGCNN is first used to quickly learn the overall shape and appearance of the liver on a low-resolution mesh, obtain global information, and generate initial landmark segmentation. Then, with the help of a refined network based on MeshConv, edge features and initial segmentation results of different levels are fused on a high-resolution mesh to effectively capture local topological details, thus balancing the processing of global and local information. Existing methods such as PointNet2Plus and DGCNN have certain capabilities in acquiring global information, but they do not pay enough attention to the surface details of the liver mesh. Although MeshCNN is good at capturing local topological information, it is difficult to achieve a comprehensive global understanding.

[0044] (2) Solving the problem of grid resolution difference: Considering the large difference in liver grid resolution, which brings challenges to segmentation, traditional methods are difficult to provide high-precision segmentation priors when mapping from low-resolution grids to high-resolution grids, and neural networks are difficult to effectively integrate information at different resolutions. The NR-MGCN of the present invention provides two mapping strategies, "dilation" and "erosion", which map the initial segmentation results at the low-resolution level to the high-resolution grid surface, generating landmark mapping priors at different levels. At the same time, the resolution encoding in the refinement network can guide the network to learn the potential correct priors according to different resolutions, effectively dealing with the problem of grid resolution difference.

[0045] (3) Optimizing the loss function to improve segmentation performance: Traditional Dice loss has defects in dealing with the unevenness of the liver mesh surface and capturing the relative relationship of anatomical landmarks, resulting in limited segmentation performance, especially for structures such as the falciform ligament. The NR-MGCN of the present invention introduces anatomically aware Dice loss, which dynamically adjusts the importance of different regions by weighting the edge length, enhances the sensitivity to sparse but critical areas, and regards the falciform ligament and liver ridge as a whole. The joint Dice loss is used to capture their structural relationship, which significantly improves the segmentation performance.

[0046] (4) Enhanced model generalization ability: Existing methods have insufficient generalization ability on liver meshes of different datasets or resolutions. When faced with data that is significantly different from the training data, the segmentation performance will drop significantly. The NR-MGCN of the present invention is trained and tested on multiple liver datasets, including 200 manually annotated liver meshes and the P2ILF challenge dataset, to verify its effectiveness on liver meshes of different resolutions, thereby enhancing the generalization ability of the model and being able to better adapt to the diversity of liver data in clinical practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 Flowchart of a liver mesh anatomical landmark segmentation method based on nested resolution grid-graph convolution in an embodiment;

[0048] Figure 2 Schematic diagram of the technical route of the liver anatomical landmark segmentation method in the embodiment;

[0049] Figure 3 for Figure 2 Schematic diagram of the technical route of the refined network part;

[0050] Figure 4 Schematic diagram of an electronic device in an embodiment. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0052] The terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0053] Example 1

[0054] In response to the problems existing in the aforementioned prior art, this embodiment provides a liver mesh anatomical landmark segmentation method based on nested resolution mesh-graph convolution. Specifically, a nested resolution mesh-graph convolution model (NestedResolution Mesh-Graph CNN, NR-MGCN) is provided to solve the key problems existing in the existing methods in the automatic segmentation of liver 3D surface anatomical landmarks, thereby improving the accuracy and robustness of the segmentation, and can be applied to liver mesh data segmentation scenarios with real-time requirements and high precision requirements.

[0055] See also Figure 1 , this method comprises the following steps:

[0056] Step S1: Acquire high-resolution liver mesh data and obtain low-resolution mesh data through random edge sampling.

[0057] The liver mesh data is preprocessed by downsampling the original high-resolution mesh to a low-resolution mesh with a fixed number of edges (e.g., 3000 edges) using random edge sampling. The center coordinates (x, y, z) of these edges are extracted as the input of the subsequent DGCNN model.

[0058] Specifically, the number of edges in the original high-resolution grid varies between 3000 and 20000. To ensure computational consistency and reduce computational burden, the “random edge sampling” method is used to fix the number of low-resolution grid edges to 3000. 3000 edges are randomly selected each time, and their center coordinates (x, y, z) are extracted as the input features of DGCNN.

[0059] In step S2, the low-resolution grid data is used as the input feature of the dynamic graph convolutional network to obtain the low-resolution preliminary segmentation results of anatomical landmarks. Figure 2 The input is the original high-resolution mesh, which is sampled into a low-resolution point cloud and then used by a dynamic graph convolutional network to obtain an initial segmentation result. Then, through two mapping strategies, dilation and erosion, high-resolution liver surface input data is obtained. This is then input into the refinement network, which outputs a refined landmark segmentation result. Step S2 may include steps S201-S203.

[0060] Step S201: Low-resolution mesh feature extraction and preliminary prediction: The obtained low-resolution mesh data is fed into the trained DGCNN model. DGCNN determines local neighborhood relationships through a k-nearest neighbor (k-NN) graph. The formula is: Determine the k nearest neighbors of each edge to construct a local neighborhood graph structure.

[0061] Specifically, DGCNN takes the coordinates (x, y, z) of the center of the low-resolution grid edge as input, and determines the local neighborhood relationship through the k-nearest neighbor (k-NN) graph. The calculation formula involves the selection of neighborhood points and the determination of relationships.

[0062] Step S202: perform spatial transformation module processing.

[0063] In step S203, local features are extracted through multiple dynamic graph convolution layers (DGConv). Each layer of DGConv recalculates the adjacency relationship to capture geometric features of different scales.

[0064] In step S204, a multi-layer perceptron (MLP) is used to map the global features (i.e., the geometric features obtained in step S203) to the anatomical landmark classification space, and a preliminary prediction result of whether each edge unit belongs to the anatomical landmark category (falcine ligament, liver ridge, or background area) is output, thereby obtaining a classification probability matrix of size (H×3) (H is the number of low-resolution grid edges, such as 3000).

[0065] Specifically, the spatial transformation module transforms the input data, calculating a 3×3 transformation matrix to normalize and align the input data and encode local topological relationships. The data then passes through multiple "dynamic graph convolutional layers (DGConv)" for local feature extraction. DGConv uses a dynamic neighborhood update strategy, recalculating adjacency relationships at each layer, iteratively capturing geometric features at different scales, and constructing high-order global structural information.

[0066] The output features of each DGConv layer of DGCNN are integrated into a 1024-dimensional global feature vector through maximum pooling, and are spliced with the original feature matrix (i.e., the low-resolution grid data obtained in step S1) to form an H×1220 (H=3000) enhanced feature matrix, which is then input into a classification network composed of a multi-layer perceptron and outputs an H×3 classification probability matrix to assign an anatomical landmark category (falciform ligament, liver ridge, or background area) to each edge unit.

[0067] Step S3: Perform dilation mapping and erosion mapping on the preliminary segmentation results. Figure 3 , step S3 may include steps S301-S302:

[0068] In step S301, the preliminary prediction results of the low-resolution grid are one-hot encoded to form an (H×3) classification matrix, which is then mapped to the original high-resolution grid through an operation similar to the MeshCNN inverse pooling to generate a probability propagation map (PSM).

[0069] In step S302, considering the possible “feature dilution” problem, two mapping strategies, “dilation” and “erosion”, are adopted:

[0070] Dilation mapping: Set a lower threshold T1 (such as T1 = 0.1). If the PSM value of an edge is greater than T1, the edge is determined to belong to the target landmark category.

[0071] Erosion mapping: Set a higher threshold T2 value (such as T2 = 0.5). Only when the PSM value is greater than T2, the edge is considered to belong to the landmark area.

[0072] Specifically, the landmark output of the dynamic graph convolutional network at the low-resolution level is an H×3 classification matrix, which is then restored to the original high-resolution grid surface through grid inverse pooling to generate a preliminary probability propagation map (PSM). Considering the difference in the number of edges between the low-resolution and high-resolution grids, the "feature dilution" phenomenon occurs, so a "dilation and erosion" mapping strategy is proposed. The dilation mapping sets a lower threshold T1 (set to 0.1 in this paper). If the PSM value of an edge is greater than T1, the edge is considered to belong to the target landmark category, which is used to solve the problem of feature dilution of high-resolution grid landmarks; the erosion mapping sets a higher threshold T2 (set to 0.5). Only when the PSM value is greater than T2 is the edge considered to belong to the landmark area, which is used to improve segmentation accuracy and reduce false prediction noise.

[0073] Step S4: Based on the dilated map data and the eroded map data after mapping, a high-resolution segmentation result is output through attention fusion in the anatomical landmark refinement network based on MeshConv. Specifically, step S4 may include steps S401-S403:

[0074] In step S401, the edge features (N×5) and mesh resolution (N) of the original high-resolution mesh are fed into a MeshConv-based anatomical landmark refinement network along with the results of dilation and erosion mapping. The dilation and erosion branches of this network each consist of three MeshConv layers, which learn and optimize different landmark regions.

[0075] The input to the MeshConv-based anatomical landmark refinement network includes the original high-resolution mesh edge features (N×5), mesh resolution information (N), dilation mapping results (N×3), and erosion mapping results (N×3). The edge features and mesh resolution information are first processed through the MLP to extract a new feature vector, which is then concatenated with the dilation and erosion mapping results and then input into two independent MeshConv-based branches (dilation branch and erosion branch) for processing.

[0076] In step S402, the features of the two branches are integrated through the attention fusion module (AFM). AFM uses two fine-grained aggregation mechanisms (FGA) to calculate the weight mapping of the two branches and generate a comprehensive feature through weighted summation. The formula is:

[0077]

[0078] Among them, W D and W E are the response weights of the expansion and erosion branches, respectively, F D and F E is the feature representation of each branch, Stands for element-wise multiplication.

[0079] Step S403: obtaining the final segmentation result of the high-resolution grid.

[0080] See also Figure 3 The dilation map is combined with the erosion map, the local features of the edge, and the resolution number N as the input of the refinement network, which is finally obtained by the three segmentation heads. The best results come from the output produced by the attention fusion mechanism.

[0081] After MeshConv processing and fusion with the AFM mechanism, the final high-resolution mesh features are mapped to the anatomical landmark classification space through a multi-layer perceptron (MLP), and the final segmentation results on the high-resolution mesh are output. To ensure that meshes in different branches can obtain accurate landmark predictions, a multi-level segmentation supervision mechanism is adopted, that is, independent segmentation heads are set in the expansion branch, erosion branch, and FGA (Fine-Grained Aggregation) fusion feature layer. The MeshConv output features at the end of each branch are connected to an independent MLP segmentation head to independently generate anatomical landmark prediction results.

[0082] Preferably, the method may further include a training step for performing two-stage training on the dynamic graph convolutional network (DGCNN) of step S2 and the anatomical landmark refinement network of step S4 before formal segmentation:

[0083] Step S001: global anatomical landmark learning on a low-resolution grid.

[0084] In each epoch of this phase, the model directly randomly samples the edge center coordinates of the original high-resolution grid to obtain edge center coordinate point cloud data with a fixed size of H = 3000 and use it for DGCNN training. The training loss of DGCNN is a combination of cross entropy loss (CE Loss) and Dice loss:

[0085]

[0086] Step S002: landmark refinement on the high-resolution grid.

[0087] During the second phase of training, the parameters of the pre-trained DGCNN are fixed. The input mesh is first randomly collapsed using a mesh pooling operation, compressing it into a low-resolution mesh of H = 3000. The center point coordinates of all edges in this low-resolution mesh are then extracted as the input to the DGCNN. Preliminary predictions are obtained and projected back to the original high-resolution mesh using two propagation methods. The five-channel features of each edge on the input high-resolution mesh and the resolution encoding are then combined as the input to the anatomical refinement network. The final optimization objective function for the second phase is:

[0088]

[0089] in, and Corresponding to the supervision loss of expansion, erosion branch and final fusion feature respectively, and composed of cross entropy loss and edge-length-weighted anatomically-aware Dice loss Combination.

[0090] The anatomically aware Dice loss includes two aspects: edge length weighting and inter-category joint optimization. The formulas are:

[0091]

[0092]

[0093] Among them, l i For edge e i length, is the average length of all edges in this category, is the Dice loss after weight adjustment, p i is the predicted value, g i is the true value, The co-occurrence loss is used to account for the co-occurrence characteristics of the falciform ligament and the liver ridge on the liver surface.

[0094] The anatomically aware Dice loss introduces edge-length weighting to ensure that long edges are not misclassified due to uneven weights, and improves the structural consistency of landmark segmentation by introducing a co-occurrence loss that takes into account the relationship between anatomical structures.

[0095] Using the trained network, high-resolution mesh data is used as the input raw, and a low-resolution mesh is generated through mesh pooling operation; then, DGCNN extracts global morphological information on the low-resolution mesh and generates preliminary landmark segmentation; then, the low-resolution segmentation result is mapped back to the high-resolution mesh through dilation and erosion, and serves as the input of the anatomical refinement network; finally, the result of the fusion feature output of the anatomical refinement network is taken as the final anatomical landmark prediction.

[0096] This method has the following beneficial effects:

[0097] (1) Improved detection accuracy: Through a nested resolution design, global prior information is captured at low resolution, and local topological relationships are optimized at high resolution, enabling more precise segmentation of anatomical landmarks. Experiments have shown that this method outperforms traditional point cloud, mesh, or graph neural network-based methods in terms of metrics such as the Dice similarity coefficient and 3D chamfer distance, and can more accurately detect liver surface landmarks.

[0098] (2) Enhanced generalization ability: It can effectively process liver mesh data of different resolutions, avoiding the information loss problem caused by fixed-resolution meshes. It shows good adaptability and stability on different data sets and has strong generalization ability.

[0099] (3) Reduced computational complexity: By performing global structural modeling on a low-resolution grid, the amount of feature computation is reduced; on a high-resolution grid, the computational cost is optimized by focusing on anatomical landmark areas. Compared with traditional methods, this method improves computational efficiency and achieves faster inference speed.

[0100] Example 2

[0101] An electronic device comprises: one or more processors and a memory, wherein the memory stores one or more programs, wherein the one or more programs include instructions for executing the liver mesh anatomical landmark segmentation method based on nested resolution grid-graph convolution as described in Example 1.

[0102] like Figure 4 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0103] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0104] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution, characterized by: The steps include: Step S1, obtaining high-resolution liver mesh data, and obtaining low-resolution mesh data by random edge sampling; Step S2, using the low-resolution grid data as input features of a dynamic graph convolutional network to obtain a low-resolution preliminary segmentation result of anatomical landmarks; Step S3, performing dilation mapping and erosion mapping processing on the preliminary segmentation result respectively; In step S4, based on the dilated map data and the eroded map data after mapping processing, a high-resolution segmentation result is output through attention fusion in the MeshConv-based anatomical landmark refinement network.

2. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 1, characterized in that: The step S1 includes the following steps: For the high-resolution liver mesh data, the mesh is downsampled to a low-resolution mesh with a preset number of edges through random edge sampling, and the center coordinates of the edges in the low-resolution mesh are extracted to form low-resolution mesh data.

3. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 1, characterized in that: The step S2 comprises the following steps: Step S201, constructing a local neighborhood graph by identifying the nearest neighboring points of each edge based on the low-resolution grid data; Step S202, encoding the local topological relationship by performing spatial transformation on the local neighborhood graph; Step S203: extract local features of the local topological relationship through multiple dynamic graph convolution layers, wherein for each dynamic graph convolution layer, the adjacency relationship is recalculated to capture geometric features of different scales; In step S204, the outputs of each dynamic graph convolution layer are integrated and spliced with the low-resolution grid data. A classification probability matrix is output through multi-layer perceptron-based classification, and an anatomical landmark category is assigned to each edge as the preliminary segmentation result of the anatomical landmark.

4. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 1, characterized in that: The step S3 includes the following steps: Step S301 , performing one-hot encoding on the preliminary segmentation result, mapping the low-resolution preliminary segmentation result to a high-resolution image through grid inverse pooling, and generating a probability propagation map; Step S302: performing dilation mapping and erosion mapping based on the probability propagation map.

5. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 4, characterized in that: The expansion mapping is: If the probability propagation value of an edge is greater than a preset first threshold, the edge is determined to belong to the target landmark category area; The erosion map is: If the probability propagation value of an edge is greater than a preset second threshold, it is determined that the edge belongs to the target landmark category area, wherein the second threshold is greater than the first threshold.

6. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 1, characterized in that: The step S4 comprises the following steps: Step S401: Using the original high-resolution liver mesh edge features, mesh resolution information, dilation mapping data, and erosion mapping data as input to the anatomical landmark refinement network, a multi-sensor machine is used to extract feature vectors from the original high-resolution liver mesh edge features and mesh resolution information. These feature vectors are then concatenated with the dilation mapping data and the erosion mapping data, respectively, to serve as inputs to the dilation mapping branch and the erosion mapping branch. Step S402: extracting local topological relationships using MeshConv for the dilation mapping branch and the erosion mapping branch respectively; Step S403 : weighted fusion of the dilation mapping branch and the erosion mapping branch is performed through a fine-grained hierarchical aggregation mechanism to obtain a final segmentation result and the segmentation results of the dilation mapping branch and the erosion mapping branch.

7. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 6, characterized in that: In step S403, the following formula is used to perform weighted fusion on the expansion mapping branch and the erosion mapping branch: Among them, W D 、W E are the response weights of the dilation mapping branch and the erosion mapping branch, respectively, D 、F E is the feature representation of the dilation mapping branch and the erosion mapping branch, Stands for element-wise multiplication.

8. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 1, characterized in that: The dynamic graph convolutional network is pre-trained by joint optimization based on Dice loss and cross entropy loss.

9. The method for liver mesh anatomical landmark segmentation based on nested resolution grid-graph convolution according to claim 1, characterized in that: The anatomical landmark refinement network is pre-trained with the goal of minimizing the loss function value, wherein the loss function includes the dilation branch loss, the erosion branch loss, and the supervision loss of the final fusion feature. For any of the dilation branch loss, the erosion branch loss, and the supervision loss, the cross entropy loss and the Dice loss based on edge length weighted anatomical perception are included. The Dice loss based on edge length weighted anatomical perception is calculated using the following formula: Among them, l i For edge e i length, is the average length of all edges under the current anatomical landmark category, p i is the predicted value, h i is the true value, is the anatomically-aware Dice loss based on edge length weighting, To consider the loss of anatomical co-occurrence characteristics, Ligament and Ridge represent the falciform ligament and liver ridge, respectively. (Ligament+Ridge) represents the sum of the Dice losses of the falciform ligament and the hepatic ridge.

10. An electronic device, characterized in that: The method comprises one or more processors, a memory and one or more programs stored in the memory, wherein the one or more programs include instructions for executing the liver mesh anatomical landmark segmentation method based on nested resolution grid-graph convolution as claimed in any one of claims 1 to 9.

Images