Deep Learning-Based Adaptive Watershed Segmentation and Surgical Planning System

Abstract

The present invention relates to the technical field of surgical planning. More specifically, it relates to an adaptive watershed segmentation and surgical planning system based on deep learning, including: a chest imaging data modality layer for obtaining chest imaging data modality information including lung lobes, lung segment boundaries, and pulmonary blood vessels; an end-to-end multi-task deep learning model layer communicatively connected to the chest imaging data modality layer, including: a blood vessel segmentation module for segmenting blood vessels in the chest imaging data modality; a hierarchical watershed segmentation module for performing hierarchical watershed segmentation based on the blood vessel segmentation results; a topological watershed network module for analyzing the topological relationship of the blood vessel structure; a watershed visualization module for generating visualization results of lung lobes and lung segments; a path planning module for generating a surgical plan; an evaluation and decision-making module for evaluating and performing decision-making analysis on the surgical plan; the present invention realizes the full-process automatic processing from medical image input to surgical plan output.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical planning, and more particularly, to an adaptive watershed segmentation and surgical planning system based on deep learning. Background Art

[0002] In recent years, with the rapid development of medical imaging technology and computer-aided diagnosis technology, medical image analysis and surgical planning systems based on deep learning have become increasingly popular in clinical practice. Especially in complex lung surgeries, accurate blood vessel segmentation, watershed division, and surgical path planning are crucial for improving the success rate of surgeries and reducing the risk of complications.

[0003] In the prior art, common lung surgical planning methods mainly rely on traditional image processing techniques and simple machine learning algorithms. These methods usually use fixed threshold segmentation or edge detection-based techniques for blood vessel segmentation, and then use region growing or watershed algorithms for watershed division. However, when facing complex lung structures and individual differences, these methods often fail to achieve satisfactory results. Especially when dealing with small blood vessels and lesion areas, over-segmentation or under-segmentation problems are likely to occur, resulting in the lack of accuracy and reliability in subsequent surgical planning.

[0004] In addition, most existing surgical path planning methods adopt geometry-based algorithms such as the Dijkstra algorithm or the A algorithm. Although these methods can find the shortest path in simple cases, it is difficult to fully consider the anatomical structure characteristics and hemodynamic properties of blood vessels. Therefore, the generated surgical path may face many challenges in clinical practice, such as being unable to effectively avoid important blood vessel structures, or ignoring the impact of blood flow direction on surgical operations.

[0005] More importantly, each processing link in the prior art is often fragmented, lacking a unified framework to integrate multiple key steps such as blood vessel segmentation, watershed division, and path planning. This fragmented processing method not only increases the complexity of the system, but also makes it difficult to fully utilize the information correlation between each link, thus limiting the improvement of the overall performance.

[0006] In view of the above problems, there is an urgent need for an intelligent surgical planning system that can comprehensively consider the anatomical structure characteristics of the lungs, the topological relationship of the blood vessel network, and the hemodynamic properties. The system should be able to adaptively handle the individual differences of different patients, achieve high-precision blood vessel segmentation and watershed division, and generate safe and effective surgical paths. At the same time, the system should also have the ability of real-time evaluation and adjustment to adapt to various changes that may occur during the surgical process. Summary of the Invention

[0007] The present invention is precisely proposed in view of the above technical problems. The present invention provides an adaptive watershed segmentation and surgical planning system based on deep learning, aiming to solve the problem of fragmentation in each processing link of traditional methods, and also significantly improve the overall performance through the collaborative effect and information sharing among modules.

[0008] To solve the above technical problems, the present invention adopts the following technical solutions:

[0009] An adaptive watershed segmentation and surgical planning system based on deep learning, comprising:

[0010] A chest imaging data modality layer, configured to obtain chest imaging data modality information including lung lobes, lung segment boundaries, and pulmonary blood vessels;

[0011] An end-to-end multi-task deep learning model layer, communicatively connected to the chest imaging data modality layer, including:

[0012] A blood vessel segmentation module, configured to perform blood vessel segmentation on the chest imaging data modality;

[0013] A hierarchical watershed segmentation module, configured to perform hierarchical watershed segmentation based on the blood vessel segmentation result;

[0014] A topological watershed network module, configured to analyze the topological relationship of the blood vessel structure;

[0015] A watershed visualization module, configured to generate visualization results of lung lobes and lung segments;

[0016] A path planning module, configured to generate a surgical plan;

[0017] An evaluation and decision-making module, configured to perform evaluation and decision-making analysis on the surgical plan;

[0018] Wherein, the end-to-end multi-task deep learning model layer is configured to segment and evaluate the chest imaging data modality to obtain visualizations of lung lobes and lung segments, the topological structure of the blood vessel network, and a surgical plan, and perform decision-making analysis.

[0019] Preferably, the blood vessel segmentation module adopts an improved U-Net model, including:

[0020] A multi-view two-dimensional feature fusion unit, configured to fuse multi-scale features;

[0021] An improved decoder unit, including a skip connection structure, configured to extract features at different levels;

[0022] An attention mechanism unit, arranged at the end of the decoder, configured to enhance the feature extraction effect of important blood vessel structures.

[0023] Preferably, the hierarchical watershed segmentation module includes:

[0024] A recursive partitioning unit for recursively partitioning a lung lobe into multiple lung segments based on the blood vessel segmentation result;

[0025] A hierarchical feature selection strategy unit for adaptively selecting the best features;

[0026] Among them, the upsampled blood vessel features are selected for the lung segments used to partition the lung lobe, and the upsampled blood vessel features and the lung segment features of this layer are selected for the lung segments used to partition the lung segments.

[0027] Preferably, the topological watershed network module is based on an improved graph neural network and includes:

[0028] A blood vessel structure modeling unit for jointly modeling the blood vessel network and the pulmonary anatomical structure;

[0029] A topological relationship analysis unit for obtaining the topological relationship of blood vessels;

[0030] A flow analysis unit for performing blood vessel flow analysis.

[0031] Preferably, the watershed visualization module includes:

[0032] A target selection unit for selecting an interested target according to the hierarchical blood vessel segmentation result;

[0033] A dynamic display unit for dynamically displaying the flow relationship between the blood vessels of the currently selected lung segment and the blood vessels of adjacent lung segments in combination with the blood vessel flow analysis result;

[0034] A solution generation unit for generating a feasible personalized surgical option.

[0035] Preferably, the path planning module includes:

[0036] A path planning algorithm unit for designing a path planning algorithm according to the interested target;

[0037] A blood vessel centerline fitting unit for starting from the target nodule and fitting the blood vessel centerline and the blood vessel branch tangent along the blood vessel direction;

[0038] A reinforcement learning optimization unit for optimizing the resection path planning by using a reinforcement learning algorithm.

[0039] Preferably, the objective function adopted by the reinforcement learning optimization unit is:

[0040] ,

[0041] Among them, is the loss degree of the blood vessel structure topological network, is the loss matrix of the blood vessel diameter, is the loss matrix of the blood vessel branches, The loss matrix for making the path select the blood vessel branch with the largest diameter when entering the blood vessel branch.

[0042] Preferably, the evaluation and decision-making module includes:

[0043] A comprehensive analysis unit for evaluating the generated surgical planning scheme according to the path planning result and the blood vessel topology analysis result;

[0044] A strategy analysis unit for analyzing surgical strategies;

[0045] A decision assistance unit for providing the comprehensive analysis result to assist the doctor in making a surgical decision.

[0046] Preferably, it further includes a real-time verification module for:

[0047] Using 3D printing technology to directly print the generated surgical path at the specific position of the human lung to simulate the actual surgical process;

[0048] At the microscopic level, imaging the slice on a CT device and comparing and verifying it with the virtual surgical path;

[0049] Real-time adjusting the virtual path according to the deviation occurring in the actual surgical process.

[0050] Preferably, the end-to-end multi-task deep learning model layer adopts a knowledge distillation module for:

[0051] Training the model with soft labels designed by experts as target labels;

[0052] Reducing the difficulty of the optimization target and enhancing the robustness of the algorithm to noise;

[0053] Improving the role of blood vessel information in the path planning process;

[0054] Enhancing the accuracy of the algorithm and its ability to absorb surgical experience.

[0055] The system of the present invention has the following remarkable technical effects:

[0056] The adaptive watershed segmentation and surgical planning system based on deep learning proposed by the present invention is precisely designed for the above technical problems. By introducing an end-to-end multi-task deep learning model, the present invention realizes the full-process automatic processing from medical image input to surgical plan output. The core of the system lies in its unique end-to-end multi-task deep learning model layer, which includes multiple functional modules such as blood vessel segmentation, hierarchical watershed segmentation, topological watershed network analysis, watershed visualization, path planning, and evaluation and decision-making.

[0057] From a macroscopic perspective, the system architecture of the present invention realizes the deep integration of medical image analysis and surgical planning. By decomposing complex surgical planning problems into a series of interrelated subtasks and jointly optimizing them under a unified deep learning framework, the system can better capture the internal connections between tasks, thereby generating more reasonable and reliable surgical plans. This holistic approach not only improves the accuracy of surgical planning but also greatly reduces the need for manual intervention, enhancing the efficiency and consistency of the entire process.

[0058] At the microscopic level, each functional module of the present invention adopts advanced deep learning technologies and is specifically optimized for the characteristics of lung surgery. For example, the vascular segmentation module uses an improved U-Net model, which significantly improves the recognition ability of complex vascular structures through multi-scale feature fusion and attention mechanisms. The hierarchical watershed segmentation module introduces graph neural networks, which can adaptively extract multi-scale information from vascular features to achieve accurate division from lung lobes to lung segments. These innovations not only solve the limitations of traditional methods in dealing with complex anatomical structures but also greatly enhance the system's adaptability to individual differences.

[0059] It is particularly worth mentioning that the path planning module of the present invention adopts a method based on reinforcement learning, which can simultaneously consider the geometric features, topological structure, and hemodynamic characteristics of blood vessels. This method can not only generate safer and more efficient surgical paths but also has strong self-adaptability and can dynamically adjust the planning strategy according to the actual situation. In addition, the system also introduces innovative mechanisms such as real-time verification and knowledge distillation to further enhance the reliability of surgical planning and the system's learning ability.

[0060] In summary, the deep learning-based adaptive watershed segmentation and surgical planning system of the present invention effectively solves many problems existing in the prior art through its innovative overall architecture and advanced technical methods. The system can not only provide more accurate and personalized surgical planning solutions but also has strong adaptability and scalability. These advantages make the system have significant clinical application value in improving the success rate of surgery, reducing the risk of complications, and shortening the operation time, providing strong technical support for the planning and implementation of complex lung surgeries. Brief Description of the Drawings

[0061] Figure 1 It is the overall system flowchart of the present invention.

[0062] Figure 2 It is the flowchart of the vascular segmentation module of the present invention.

[0063] Figure 3 It is the flowchart of the hierarchical watershed segmentation module of the present invention.

[0064] Figure 4 It is a flowchart of the topological watershed network module of the present invention.

[0065] Figure 5 It is a flowchart of the watershed visualization module of the present invention.

[0066] Figure 6 It is a flowchart of the path planning module of the present invention.

[0067] Figure 7 It is a flowchart of the evaluation and decision-making module of the present invention. Detailed implementation manners

[0068] As Figure 1-7 shown, the present invention provides an adaptive watershed segmentation and surgical planning system based on deep learning, which can effectively analyze lung images and provide accurate surgical planning schemes for surgeons. The following will further elaborate on the present invention in combination with specific implementation manners.

[0069] The system of the present invention includes a chest imaging data modality layer 1 and an end-to-end multi-task deep learning model layer 2. The chest imaging data modality layer 1 is used to obtain chest imaging data modality information including lung lobes, lung segment boundaries, and pulmonary blood vessels. In a preferred embodiment of the present invention, the chest imaging data modality layer 1 can be a high-resolution CT scanning device, and its resolution is preferably 0.5mm x 0.5mm x 0.5mm. Such a resolution can ensure capturing sufficiently clear details of the lung structure.

[0070] The end-to-end multi-task deep learning model layer 2 is communicatively connected to the chest imaging data modality layer 1 and includes multiple functional modules. These modules work together to jointly achieve the full-process automation from image data input to surgical plan output. Specifically, the end-to-end multi-task deep learning model layer 2 includes: a blood vessel segmentation module 21, a hierarchical watershed segmentation module 22, a topological watershed network module 23, a watershed visualization module 24, a path planning module 25, and an evaluation and decision-making module 26.

[0071] The blood vessel segmentation module 21 is used to perform blood vessel segmentation on the chest imaging data modality. The present invention uses an improved U-Net model to achieve accurate blood vessel segmentation. Preferably, the input layer of this model uses 32 3x3 convolutional kernels, and the activation function is selected as ReLU. Such a configuration can effectively extract low-level features of blood vessels. In the encoder part, 4 max-pooling operations are used, and the number of channels doubles after each pooling, reaching a maximum of 512 channels. Such a design can gradually extract higher-level semantic features, which is beneficial to capturing complex blood vessel structures.

[0072] The hierarchical watershed segmentation module 22 performs hierarchical watershed segmentation based on the blood vessel segmentation result. This module uses a graph neural network to achieve adaptive feature extraction. In one embodiment, the Graph Convolutional Network (GCN) is used, and the output of each graph convolutional layer can be expressed as:

[0073] ,

[0074] where, represents the node feature matrix of the l-th layer, is the adjacency matrix with self-connection added, is 's degree matrix, is the learnable weight matrix, is the non-linear activation function. This design can effectively capture the local and global features of the blood vessel structure, thus achieving accurate watershed segmentation.

[0075] The topological watershed network module 23 is used to analyze the topological relationship of the blood vessel structure. This module is based on an improved graph neural network, which can not only accurately describe the spatial relationship of blood vessels, but also capture functional information such as blood flow. In practical applications, a graph neural network based on the attention mechanism is adopted, and its core formula is as follows:

[0076] ,

[0077] ,

[0078] where, represents the attention coefficient of node i to neighbor node j, and are the feature vectors of nodes i and j respectively, W is the weight matrix, a is the learnable attention vector, is the activation function. This design can adaptively focus on important blood vessel connections and improve the accuracy of topological analysis.

[0079] The blood vessel segmentation module 21 adopts an improved U-Net model, including a multi-view two-dimensional feature fusion unit 211, an improved decoder unit 212, and an attention mechanism unit 213. The multi-view two-dimensional feature fusion unit 211 is used to fuse multi-scale features. In one embodiment of the present invention, a multi-scale feature fusion strategy is adopted. Specifically, convolution kernels of three different sizes, 1x1, 3x3, and 5x5, are used to extract features in parallel, and then these features are fused together. This method can capture the blood vessel structure information of different scales at the same time and improve the accuracy of segmentation.

[0080] The improved decoder unit 212 includes a skip connection structure for extracting features at different levels. In the present invention, not only the skip connections in the traditional U-Net are used, but also dense connections are introduced. Specifically, each layer of the decoder receives not only the output of the previous layer but also the outputs of all the previous layers. This design can be expressed as:

[0081] ,

[0082] where, represents the output of the l-th layer, represents the non-linear transformation, represents concatenating the outputs of all the previous layers together. This design can utilize features at different levels more effectively and improve the accuracy of blood vessel segmentation.

[0083] The attention mechanism unit 213 is arranged at the end of the decoder for enhancing the feature extraction effect of important blood vessel structures. In the present invention, a method combining spatial attention mechanism and channel attention mechanism is adopted. The spatial attention mechanism can help the model focus on important regions in the image, while the channel attention mechanism can highlight important feature channels. The specific attention mechanism can be expressed as:

[0084] ,

[0085] ,

[0086] ,

[0087] where, F represents the input feature map, and represent channel attention and spatial attention respectively, represents the sigmoid activation function, represents element-wise multiplication. This design can effectively highlight important blood vessel structures and improve the accuracy of segmentation.

[0088] The hierarchical watershed segmentation module 22 includes a recursive partitioning unit 221 and a hierarchical feature selection strategy unit 222. The recursive partitioning unit 221 is used to recursively divide the lung lobe into multiple lung segments based on the blood vessel segmentation result. In an embodiment of the present invention, a recursive partitioning algorithm based on graph cut is adopted. First, the blood vessel segmentation result is represented as a graph G=(V, E), where V represents the nodes in the graph (corresponding to the pixels in the image), and E represents the edges between the nodes. Then, an energy function is defined:

[0089] ,

[0090] Among them, A represents the segmentation result, represents a data item, measuring the cost of pixel p being assigned to label The cost of represents a smoothness term, measuring the cost of adjacent pixels p and q being assigned to different labels. The optimal segmentation result is obtained by minimizing this energy function. This method can effectively utilize the topological structure information of blood vessels to achieve accurate lung segment division.

[0091] The hierarchical feature selection strategy unit 222 is used to adaptively select the best features. In the present invention, a dynamic feature selection mechanism is designed. Specifically, for different levels of segmentation tasks, different feature combinations are used. For example, for lobe-level segmentation, mainly deep semantic features are used; while for segment-level segmentation, shallow detail features and deep semantic features are combined. This strategy can be expressed as:

[0092] ,

[0093] Among them, represents the feature of the l-th level, represents the feature map of the i-th layer, is a learnable weight. By learning these weights, the model can adaptively select the most appropriate feature combination to improve the accuracy and robustness of segmentation.

[0094] Through the above design, the deep learning-based adaptive watershed segmentation and surgical planning system of the present invention can achieve high-precision pulmonary vascular segmentation and lung segment division, providing a reliable basis for subsequent surgical planning. The multi-module collaborative design of this system not only improves the accuracy of segmentation, but also enhances the adaptability and robustness of the system, and can better handle the individual differences of different patients.

[0095] The topological watershed network module 23 of the present invention is based on an improved graph neural network, including a vascular structure modeling unit 231, a topological relationship analysis unit 232, and a flow analysis unit 233. The vascular structure modeling unit 231 is used to jointly model the vascular network and the pulmonary anatomical structure. In a preferred embodiment of the present invention, this unit adopts a hierarchical graph structure representation method. Specifically, this method first represents the entire pulmonary vascular network as a large graph structure G, and then divides it into multiple subgraphs G_i according to anatomical knowledge, and each subgraph corresponds to a lung segment. This hierarchical representation method can be expressed in the following mathematical form:

[0096] ,

[0097] ,

[0098] Among them, and represent the node sets of the entire graph and sub - graphs respectively, and represent the edge sets, and represent the attribute sets. Preferably, node attributes include information such as spatial location, blood vessel diameter, etc., and edge attributes include characteristics such as length, curvature, etc. This representation method can not only accurately describe the topological structure of blood vessels, but also retain important anatomical information, laying a foundation for subsequent analysis.

[0099] The topological relationship analysis unit 232 is used to obtain the topological relationship of blood vessels. The method of the present invention introduces an attention mechanism in this unit to better capture key topological structures. Specifically, the method defines an attention matrix A, whose element represents the attention weight of node i to node j. The calculation formula of the attention weight is as follows:

[0100] ,

[0101] where, and represent the feature vectors of nodes i and j respectively, is a learnable parameter matrix, LeakyReLU is the activation function, and || represents the vector concatenation operation. In this way, the method can adaptively focus on important blood vessel connections and improve the accuracy of topological analysis.

[0102] The flow analysis unit 233 is used to perform blood vessel flow analysis. In the preferred embodiment of the present invention, this unit adopts a method combining a physical model and deep learning. First, the method uses a simplified Navier - Stokes equation to simulate the blood flow in blood vessels:

[0103] ,

[0104] ,

[0105] where, ρ is the blood density, v is the velocity vector, p is the pressure, and μ is the viscosity. Then, the method uses a graph neural network to learn and predict the flow of each blood vessel branch. Specifically, for each node i, its flow prediction can be expressed as:

[0106] ,

[0107] where, is a parameterized function (such as a multi - layer perceptron), is the feature of node i, is the set of neighbors of node i. In this way, this method can comprehensively consider the geometric features and topological structure of blood vessels to obtain a more accurate flow estimation.

[0108] The watershed visualization module 24 of the present invention includes a target selection unit 241, a dynamic display unit 242, and a solution generation unit 243. The target selection unit 241 is used to select the target of interest according to the hierarchical blood vessel segmentation result. In an embodiment of the present invention, this unit adopts an automatic selection method based on importance scoring. Specifically, this method calculates an importance score S for each lung segment:

[0109] ,

[0110] where V represents the volume of the lung segment, F represents the total blood flow through the lung segment, C represents the connection complexity with the surrounding lung segments, 、 and are weight coefficients. Preferably, 、 and can be adjusted according to specific clinical needs. For example, in lung cancer surgery planning, more attention may be paid to the volume and blood flow of the lung segment. Therefore, =0.4, .

[0111] The dynamic display unit 242 is used to dynamically display the flow relationship between the currently selected lung segment blood vessels and the adjacent lung segment blood vessels in combination with the blood vessel flow analysis result. The method of the present invention adopts an interactive 3D visualization technology in this unit. Specifically, this method uses color and thickness to represent the blood flow of blood vessels and uses animation effects to show the dynamic changes of blood flow. For example, the blood flow can be mapped to a gradient color spectrum from red to blue. The greater the blood flow, the more the color tends to red. At the same time, the thickness of the blood vessel is also proportional to the blood flow. This intuitive visualization method can help doctors quickly understand the complex blood vessel network structure and blood flow distribution.

[0112] The solution generation unit 243 is used to generate feasible personalized surgical options. In a preferred embodiment of the present invention, this unit adopts a solution generation algorithm based on multi-objective optimization. Specifically, this method considers the following objectives: minimizing surgical trauma, maximizing the preservation of healthy tissue, and ensuring sufficient blood supply. This multi-objective optimization problem can be expressed as:

[0113] ,

[0114] ,

[0115] ,

[0116] Among them, , and respectively represent the degrees of surgical trauma, healthy tissue loss, and insufficient blood supply, and represent various constraint conditions. This method uses an improved NSGA-II algorithm to solve this multi-objective optimization problem, thereby generating a series of alternative surgical plans.

[0117] The path planning module 25 of the present invention includes a path planning algorithm unit 251, a blood vessel centerline fitting unit 252, and a reinforcement learning optimization unit 253. The path planning algorithm unit 251 is used to design a path planning algorithm according to the target of interest. In an embodiment of the present invention, this unit adopts an improved A algorithm. Specifically, this method defines a comprehensive cost function:

[0118] ,

[0119] where g(n) is the actual cost from the starting point to the current node n, h(n) is the estimated cost from the current node to the target, and r(n) is an additional cost term related to risk. Preferably, r(n) can be calculated according to the distance to important structures (such as large blood vessels) near node n. For example, r(n)=k / d can be set, where d is the distance from node n to the nearest important structure, and k is an adjustable parameter. By introducing this risk term, this method can consider surgical safety while finding the shortest path.

[0120] The blood vessel centerline fitting unit 252 is used to start from the target nodule and fit the blood vessel centerline and blood vessel branch tangents along the blood vessel direction. The method of the present invention adopts a fitting algorithm based on principal component analysis (PCA) in this unit. Specifically, for each cross-section of the blood vessel, this method first extracts the blood vessel contour points, and then performs PCA analysis on these points. The blood vessel center can be approximated as the mean of these points, and the blood vessel direction can be given by the first principal component. By gradually advancing along the blood vessel direction, this method can accurately fit the centerline of the blood vessel.

[0121] The objective function adopted by the reinforcement learning optimization unit 253 of the present invention is:

[0122] ,

[0123] In a preferred embodiment of the present invention, these loss terms can be specifically defined as follows:

[0124] ,

[0125] where E is the set of edges in the blood vessel network, Represents the probability of transitioning from action to action .

[0126] ,

[0127] where, is the blood vessel diameter at node i, is the action taken at node i.

[0128] ,

[0129] where B is the set of blood vessel bifurcation points, represents the probability of selecting action in state .

[0130] ,

[0131] where, is the neighbor set of the node, is the indicator function. By minimizing this comprehensive objective function, the method of the present invention can generate an optimal path that not only follows the blood vessel anatomical structure but also meets the surgical requirements. This path planning method based on reinforcement learning has strong self-adaptability and robustness and can effectively cope with complex and variable blood vessel structures.

[0132] By minimizing this comprehensive objective function, the method of the present invention can generate an optimal path that not only follows the blood vessel anatomical structure but also meets the surgical requirements. This path planning method based on reinforcement learning has strong self-adaptability and robustness and can effectively cope with complex and variable blood vessel structures.

[0133] The evaluation and decision-making module 26 of the present invention includes a comprehensive analysis unit 261, a policy analysis unit 262, and a decision-making assistance unit 263. The comprehensive analysis unit 261 is used to evaluate the generated surgical planning scheme according to the path planning result and the blood vessel topology analysis result. In a preferred embodiment of the present invention, this unit adopts a method based on multi-dimensional scoring. Specifically, this method defines a comprehensive scoring function S:

[0134] ,

[0135] where, represents the safety score, represents the effectiveness score, represents the difficulty score, , and are the corresponding weight coefficients. Preferably, these scores can be calculated according to specific clinical indicators. For example, the safety score can consider the minimum distance between the planned path and important structures; the effectiveness score can consider the coverage rate of the target area; the difficulty score can consider factors such as the complexity of the path and the limitations of the operating space. By adjusting the weight coefficients, this method can flexibly adjust the evaluation criteria according to different clinical needs.

[0136] The strategy analysis unit 262 is used to analyze the surgical strategy. The method of the present invention introduces a strategy analysis method based on decision trees in this unit. Specifically, this method first constructs a decision tree, where each node represents a decision point and each edge represents a possible choice. Then, for each leaf node (i.e., the final surgical strategy), its expected utility is calculated:

[0137] ,

[0138] where s represents the strategy, represents the possible outcome, represents the probability that the outcome occurs when the strategy s is adopted, represents the utility of the outcome . This method can help doctors comprehensively evaluate the potential risks and benefits of different surgical strategies.

[0139] The decision assistance unit 263 is used to provide a comprehensive analysis result to assist doctors in making surgical decisions. In a preferred embodiment of the present invention, this unit adopts a decision support system based on fuzzy logic. Specifically, this method first fuzzifies each evaluation index, then performs reasoning through a series of fuzzy rules, and finally obtains a recommended decision. The general form of the fuzzy rule can be expressed as:

[0140] IF (condition 1 is A) AND (condition 2 is B) THEN (decision is C),

[0141] where A, B, and C are fuzzy sets. For example, a specific rule may be:

[0142] IF (safety is high) AND (effectiveness is medium) AND (difficulty is low) THEN (recommended execution is strong),

[0143] In this way, this method can convert the quantitative analysis result into a qualitative recommendation that is easier to understand and operate, providing strong support for doctors' decisions.

[0144] The present invention further includes a real-time verification module 27 for simulating the actual surgical process and performing real-time evaluation. In one embodiment of the present invention, the real-time verification module 27 first uses 3D printing technology to directly print the generated surgical path at the specific position of the human lung, simulating the actual surgical process. This method can intuitively display the position and trend of the surgical path in the actual anatomical structure, helping doctors better understand and evaluate the surgical plan.

[0145] Preferably, at the microscopic level, the method images the slices on a CT device and compares and verifies them with the virtual surgical path. Specifically, the method uses image registration technology to align the virtual path with the actual CT image. The registration process can be expressed as an optimization problem:

[0146] ,

[0147] where, represents the virtual path image, represents the actual CT image, T represents the transformation function, and D represents the image similarity metric. By solving this optimization problem, the method can accurately map the virtual path onto the actual CT image for quantitative comparison and analysis.

[0148] In addition, the method of the present invention can adjust the virtual path in real time according to the deviations occurring during the actual surgical process. For this purpose, the method introduces a dynamic path adjustment algorithm. The algorithm is based on the Kalman filter and can be expressed as:

[0149] ,

[0150] ,

[0151] where, is the path state at time k, is the control input, is the observation value, , and are the state transition matrix, control input matrix, and observation matrix respectively, and are the process noise and observation noise respectively. In this way, the method can flexibly adapt to various changes in the actual surgery while maintaining the main features of the original planned path.

[0152] The end-to-end multi-task deep learning model layer 2 of the present invention adopts a knowledge distillation module 28 for improving the model performance and robustness. In the preferred embodiment of the present invention, the knowledge distillation module 28 trains the model using soft labels designed by experts as the target labels. Specifically, the method defines a distillation loss function:

[0153] ,

[0154] where represents the cross - entropy loss, KL represents the KL divergence, y is the true label, and are the output logits of the student model and the teacher model respectively, σ is the softmax function, T is the temperature parameter, and α is the balance factor. By adjusting the temperature parameter T, this method can control the softness of knowledge transfer, thus achieving a good balance between model performance and generalization ability.

[0155] In addition, to enhance the role of vascular information in the path - planning process, the method of the present invention introduces a vascular attention mechanism during the knowledge distillation process. Specifically, this method adds a vascular attention module to the teacher model, and its output can be expressed as:

[0156] ,

[0157] where F is the input feature map, , , and are learnable parameters. Then, this method passes this attention information as additional soft labels to the student model. This way can guide the student model to better focus on important vascular structures, thereby improving the accuracy of path - planning.

[0158] Finally, to further improve the accuracy of the algorithm and its ability to absorb surgical experience, the method of the present invention adopts a progressive knowledge distillation strategy. Specifically, this method first trains the student model with simple tasks, and then gradually increases the complexity of the tasks. At each stage, a teacher model with corresponding complexity is used to guide the learning of the student model. This process can be expressed as:

[0159] ,

[0160] where N is the total number of tasks, is the weight of the i - th task, is the distillation loss of the i - th task. Through this progressive learning strategy, this method can more effectively convert complex surgical experience into the ability of the model, thereby achieving better performance and stronger generalization ability.

[0161] Through the above design, the deep learning-based adaptive watershed segmentation and surgical planning system of the present invention can not only generate high-quality surgical plans, but also be verified and adjusted in real time during actual application, and continuously improve its own performance through knowledge distillation technology. This comprehensive and advanced design enables the system to provide reliable planning and decision-making support for complex lung surgeries, and has important clinical application value.

[0162] The above are only the preferred specific embodiments of the present invention; however, the protection scope of the present invention is not limited thereto; any person familiar with the art within the scope disclosed by the present invention, according to the solution and its improved concept of the present invention, making equivalent substitutions or changes, should be covered within the protection scope of the present invention.

Claims

1. An adaptive watershed segmentation and surgical planning system based on deep learning, characterized in that Comprising: A chest imaging data modality layer for obtaining chest imaging data modality information including lung lobes, lung segment boundaries, and pulmonary blood vessels; An end-to-end multi-task deep learning model layer communicatively connected to the chest imaging data modality layer, including: A blood vessel segmentation module for segmenting blood vessels in the chest imaging data modality; A hierarchical watershed segmentation module for performing hierarchical watershed segmentation based on the blood vessel segmentation result; A topological watershed network module for analyzing the topological relationship of the blood vessel structure; A watershed visualization module for generating visualization results of lung lobes and lung segments; A path planning module for generating a surgical plan; An evaluation and decision-making module for evaluating and performing decision-making analysis on the surgical plan; Wherein, the end-to-end multi-task deep learning model layer is used to segment and evaluate the chest imaging data modality to obtain visualizations of lung lobes and lung segments, the topological structure of the blood vessel network, and a surgical plan, and to perform decision-making analysis; The path planning module includes: A path planning algorithm unit for designing a path planning algorithm according to the target of interest; A blood vessel centerline fitting unit for starting from the target nodule and fitting the blood vessel centerline and the blood vessel branch tangent along the blood vessel direction; A reinforcement learning optimization unit for optimizing the resection path planning using a reinforcement learning algorithm; The objective function adopted by the reinforcement learning optimization unit is: , Among them, is the loss degree of the vascular structure topological network, is the loss matrix of the blood vessel diameter, is the loss matrix of the blood vessel branches, is the loss matrix for making the path select the blood vessel branch with the largest diameter when entering the blood vessel branch.

2. The system according to claim 1, wherein The blood vessel segmentation module adopts an improved U-Net model, including: A multi-view two-dimensional feature fusion unit for fusing multi-scale features; An improved decoder unit containing a skip connection structure for extracting features at different levels; An attention mechanism unit arranged at the end of the decoder for enhancing the feature extraction effect of important blood vessel structures.

3. The system according to claim 1, wherein The hierarchical watershed segmentation module includes: A recursive partitioning unit for recursively dividing lung lobes into multiple lung segments based on the blood vessel segmentation result; A hierarchical feature selection strategy unit for adaptively selecting the best features; Wherein, the lung segments for dividing lung lobes select upsampled blood vessel features, and the lung segments for dividing lung segments select upsampled blood vessel features and the lung segment features of this layer.

4. The system according to claim 1, wherein The topological watershed network module is based on an improved graph neural network, including: A blood vessel structure modeling unit for jointly modeling the blood vessel network and the pulmonary anatomical structure; A topological relationship analysis unit for obtaining the topological relationship of the blood vessels; A blood flow analysis unit for performing blood vessel flow analysis.

5. The system according to claim 1, wherein The watershed visualization module includes: A target selection unit for selecting the target of interest according to the hierarchical blood vessel segmentation result; A dynamic display unit for dynamically displaying the flow relationship between the blood vessels of the currently selected lung segment and the blood vessels of adjacent lung segments in combination with the blood vessel flow analysis result; A plan generation unit for generating a feasible personalized surgical option.

6. The system according to claim 1, characterized in that, The evaluation and decision-making module includes: A comprehensive analysis unit for evaluating the generated surgical planning plan according to the path planning result and the blood vessel topological analysis result; A strategy analysis unit for analyzing the surgical strategy; A decision-making assistance unit for providing the comprehensive analysis result to assist the doctor in making a surgical decision.

7. The system according to claim 1, wherein It further includes a real-time verification module for: The generated surgical path will be directly printed at the specific position of the human lung using 3D printing technology to simulate the actual surgical process; At the microscopic level, the slices are imaged on a CT device and compared and verified with the virtual surgical path; The virtual path is adjusted in real time according to the deviations that occur during the actual surgical process.

8. The system according to claim 1, wherein The end-to-end multi-task deep learning model layer adopts a knowledge distillation module for: Training the model with soft labels designed by experts as target labels; Reducing the difficulty of the optimization objective and enhancing the robustness of the algorithm to noise; Improving the role of vascular information in the path planning process; Enhancing the accuracy of the algorithm and its ability to absorb surgical experience.

Images