Self-adaptive drainage basin segmentation and operation planning system based on deep learning

Abstract

The invention relates to the technical field of operation planning, in particular to an adaptive drainage basin segmentation and operation planning system based on deep learning, which comprises a chest imaging data modal layer used for acquiring chest imaging data modal information including lung lobes, lung segment boundaries and lung blood vessels; the end-to-end multi-task deep learning model layer is in communication connection with the chest imaging data modal layer and comprises a blood vessel segmentation module used for performing blood vessel segmentation on the chest imaging data modal; the hierarchical drainage basin segmentation module is used for carrying out hierarchical drainage basin segmentation based on the blood vessel segmentation result; the topological basin network module is used for analyzing the topological relation of the vascular structure; the drainage basin visualization module is used for generating visualization results of lung lobes and lung segments; the path planning module is used for generating an operation scheme; the evaluation and decision-making module is used for carrying out evaluation and decision-making analysis on the operation scheme; according to the invention, full-process automatic processing from medical image input to surgical scheme output is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of surgical planning, and more specifically, to an adaptive watershed segmentation and surgical planning system based on deep learning. Background Art

[0002] In recent years, with the rapid development of medical imaging technology and computer-aided diagnosis technology, medical image analysis and surgical planning systems based on deep learning have become increasingly popular in clinical practice. Especially in complex lung surgery, accurate vascular segmentation, watershed division, and surgical path planning are crucial to improving the success rate of surgery and reducing the risk of complications.

[0003] In the prior art, common lung surgery planning methods mainly rely on traditional image processing technology and simple machine learning algorithms. These methods usually use fixed threshold segmentation or edge detection-based technology to segment blood vessels, and then use region growing or watershed algorithms for watershed division. However, these methods often find it difficult to achieve satisfactory results when faced with complex lung structures and individual differences. In particular, when dealing with small blood vessels and lesion areas, over-segmentation or under-segmentation problems are prone to occur, resulting in a lack of accuracy and reliability in subsequent surgical planning.

[0004] In addition, most of the existing surgical path planning methods use geometry-based algorithms, such as Dijkstra algorithm or A Although these methods can find the shortest path in simple cases, they are difficult to fully consider the anatomical characteristics and hemodynamic characteristics of blood vessels. Therefore, the generated surgical paths may face many challenges in clinical practice, such as failing to effectively avoid important vascular structures or ignoring the impact of blood flow direction on surgical operations.

[0005] More importantly, the various processing links in the existing technology are often fragmented, lacking a unified framework to integrate multiple key steps such as blood vessel segmentation, watershed division, and path planning. This fragmented processing method not only increases the complexity of the system, but also makes it difficult to fully utilize the information association between various links, thus limiting the improvement of overall performance.

[0006] In view of the above problems, there is an urgent need for an intelligent surgical planning system that can comprehensively consider the characteristics of the lung anatomical structure, the topological relationship of the vascular network, and the hemodynamic characteristics. The system should be able to adaptively handle the individual differences of different patients, achieve high-precision vascular segmentation and watershed division, and generate a safe and effective surgical path. At the same time, the system should also have real-time evaluation and adjustment capabilities to adapt to various changes that may occur during the operation. Summary of the invention

[0007] The present invention is proposed to solve the above technical problems. The present invention provides an adaptive watershed segmentation and surgical planning system based on deep learning, which aims to solve the problem of separation of various processing links in traditional methods, and also significantly improves the overall performance through synergy and information sharing between modules.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions: Adaptive watershed segmentation and surgical planning system based on deep learning, including: A chest imaging data modality layer, used to obtain chest imaging data modality information including lung lobes, lung segment boundaries and lung blood vessels; An end-to-end multi-task deep learning model layer, in communication with the chest imaging data modality layer, comprises: A blood vessel segmentation module, used for performing blood vessel segmentation on the chest imaging data modality; A hierarchical watershed segmentation module, used for performing hierarchical watershed segmentation based on the blood vessel segmentation result; Topological watershed network module, used to analyze the topological relationship of vascular structures; The watershed visualization module is used to generate visualization results of lung lobes and segments; Path planning module, used to generate surgical plans; Evaluation and decision-making module, used to evaluate and analyze surgical plans; Among them, the end-to-end multi-task deep learning model layer is used to segment and evaluate the chest imaging data modality to obtain visualization of lung lobes and segments, vascular network topology and surgical plans, and perform decision analysis.

[0009] Preferably, the blood vessel segmentation module adopts an improved U-Net model, including: Multi-view 2D feature fusion unit, used to fuse multi-scale features; An improved decoder unit with skip connection structure to extract features at different levels; The attention mechanism unit is set at the end of the decoder to improve the feature extraction effect of important vascular structures.

[0010] Preferably, the hierarchical watershed segmentation module comprises: A recursive division unit, used for recursively dividing the lung lobe into a plurality of lung segments based on the blood vessel segmentation result; Hierarchical feature selection strategy unit, used to adaptively select the best features; Among them, the lung segment used to divide the lung lobes selects the upsampled blood vessel features, and the lung segment used to divide the lung segments selects the upsampled blood vessel features and the lung segment features of this layer.

[0011] Preferably, the topological watershed network module is based on an improved graph neural network and includes: The vascular structure modeling unit is used to jointly model the vascular network with the lung anatomical structure; A topological relationship analysis unit, used to obtain the topological relationship of blood vessels; A flow analysis unit is used to perform blood vessel flow analysis.

[0012] Preferably, the watershed visualization module comprises: A target selection unit, used to select a target of interest based on the hierarchical blood vessel segmentation result; A dynamic display unit, used to dynamically display the flow relationship between the currently selected pulmonary segment blood vessels and the adjacent pulmonary segment blood vessels in combination with the vascular flow analysis results; A plan generation unit is used to generate feasible personalized surgical options.

[0013] Preferably, the path planning module comprises: A path planning algorithm unit, used to design a path planning algorithm according to a target of interest; A vascular centerline fitting unit is used to fit the vascular centerline and vascular branch tangent line starting from the target nodule along the vascular direction; The reinforcement learning optimization unit is used to optimize the resection path planning by using the reinforcement learning algorithm.

[0014] Preferably, the objective function adopted by the reinforcement learning optimization unit is: , in, is the loss degree of the vascular structure topology network, is the loss matrix of blood vessel diameter, is the loss matrix of the blood vessel branch, The loss matrix is ​​used to select the blood vessel branch with the largest diameter when the path enters the blood vessel branch.

[0015] Preferably, the evaluation and decision-making module includes: A comprehensive analysis unit, used to evaluate the generated surgical planning scheme according to the path planning results and the vascular topology analysis results; Strategy analysis unit, used to analyze surgical strategies; The decision support unit is used to provide comprehensive analysis results to assist doctors in making surgical decisions.

[0016] Preferably, a real-time verification module is also included, for: Use 3D printing technology to print the generated surgical path directly at the specific location of the human lung to simulate the actual surgical process; At the microscopic level, the slices are imaged on a CT device and compared and verified with the virtual surgical path; The virtual path is adjusted in real time according to deviations that occur during the actual surgery.

[0017] Preferably, the end-to-end multi-task deep learning model layer adopts a knowledge distillation module for: Train the model using soft labels designed by experts as target labels; Reduce the difficulty of optimizing the target and improve the algorithm's robustness to noise; Improve the role of vascular information in the path planning process; Improve the accuracy of the algorithm and its ability to absorb surgical experience.

[0018] The system of the present invention has the following significant technical effects: The deep learning-based adaptive watershed segmentation and surgical planning system proposed in the present invention is designed to address the above technical issues. By introducing an end-to-end multi-task deep learning model, the present invention realizes full-process automated processing from medical image input to surgical plan output. The core of the system lies in its unique end-to-end multi-task deep learning model layer, which includes multiple functional modules such as vascular segmentation, hierarchical watershed segmentation, topological watershed network analysis, watershed visualization, path planning, and evaluation and decision-making.

[0019] From a macro perspective, the system architecture of the present invention achieves a deep integration of medical image analysis and surgical planning. By decomposing complex surgical planning problems into a series of interrelated subtasks and jointly optimizing them under a unified deep learning framework, the system can better capture the intrinsic connections between tasks, thereby generating more reasonable and reliable surgical plans. This holistic approach not only improves the accuracy of surgical planning, but also greatly reduces the need for manual intervention, improving the efficiency and consistency of the entire process.

[0020] At the microscopic level, each functional module of the present invention adopts advanced deep learning technology and is specially optimized for the characteristics of lung surgery. For example, the vascular segmentation module adopts an improved U-Net model, which significantly improves the recognition ability of complex vascular structures through multi-scale feature fusion and attention mechanism. The hierarchical watershed segmentation module introduces a graph neural network, which can adaptively extract multi-scale information from vascular features and achieve accurate division from lung lobes to lung segments. These innovations not only solve the limitations of traditional methods in dealing with complex anatomical structures, but also greatly improve the system's ability to adapt to individual differences.

[0021] It is particularly worth mentioning that the path planning module of the present invention adopts a reinforcement learning-based method that can simultaneously consider the geometric characteristics, topological structure and hemodynamic characteristics of blood vessels. This method can not only generate a safer and more efficient surgical path, but also has strong adaptability and can dynamically adjust the planning strategy according to actual conditions. In addition, the system also introduces innovative mechanisms such as real-time verification and knowledge distillation, which further improves the reliability of surgical planning and the learning ability of the system.

[0022] In summary, the deep learning-based adaptive watershed segmentation and surgical planning system of the present invention effectively solves many problems existing in the prior art through its innovative overall architecture and advanced technical methods. The system can not only provide more accurate and personalized surgical planning solutions, but also has strong adaptability and scalability. These advantages make the system have significant clinical application value in improving the success rate of surgery, reducing the risk of complications, shortening the operation time, etc., and provide strong technical support for the planning and implementation of complex lung surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall flow chart of the system of the present invention.

[0024] Figure 2 FIG. 4 is a flow chart of the blood vessel segmentation module of the present invention.

[0025] Figure 3 It is a flowchart of the hierarchical watershed segmentation module of the present invention.

[0026] Figure 4 It is a flow chart of the topological watershed network module of the present invention.

[0027] Figure 5 Flow chart of the watershed visualization module of the present invention.

[0028] Figure 6 It is a flow chart of the path planning module of the present invention.

[0029] Figure 7 It is a flow chart of the evaluation and decision module of the present invention. DETAILED DESCRIPTION

[0030] like Figure 1-7 As shown, the present invention provides an adaptive watershed segmentation and surgical planning system based on deep learning, which can effectively analyze lung images and provide surgeons with accurate surgical planning solutions. The present invention will be further described in detail below in conjunction with specific implementation methods.

[0031] The system of the present invention includes a chest imaging data modality layer 1 and an end-to-end multi-task deep learning model layer 2. The chest imaging data modality layer 1 is used to obtain chest imaging data modality information including lung lobes, lung segment boundaries and lung blood vessels. In a preferred embodiment of the present invention, the chest imaging data modality layer 1 can be a high-resolution CT scanning device, and its resolution is preferably 0.5mmx0.5mmx0.5mm, such a resolution can ensure that sufficiently clear lung structure details are captured.

[0032] The end-to-end multi-task deep learning model layer 2 is connected to the chest imaging data modality layer 1 and includes multiple functional modules. These modules work together to realize the automation of the entire process from image data input to surgical plan output. Specifically, the end-to-end multi-task deep learning model layer 2 includes: a blood vessel segmentation module 21, a hierarchical watershed segmentation module 22, a topological watershed network module 23, a watershed visualization module 24, a path planning module 25, and an evaluation and decision module 26.

[0033] The blood vessel segmentation module 21 is used to perform blood vessel segmentation on the chest imaging data modality. The present invention adopts an improved U-Net model to achieve accurate blood vessel segmentation. Preferably, the input layer of the model adopts 32 3x3 convolution kernels, and the activation function selects ReLU. This configuration can effectively extract low-level features of blood vessels. In the encoder part, 4 maximum pooling operations are used, and the number of channels doubles after each pooling, reaching a maximum of 512 channels. This design can gradually extract higher-level semantic features, which is conducive to capturing complex vascular structures.

[0034] The hierarchical watershed segmentation module 22 performs hierarchical watershed segmentation based on the blood vessel segmentation result. This module uses a graph neural network to achieve adaptive feature extraction. In one embodiment, a Graph Convolutional Network (GCN) is used, where the output of each graph convolution layer can be expressed as: , in, represents the node feature matrix of the lth layer, is the adjacency matrix with self-connection added, yes The degree matrix of is a learnable weight matrix, is a nonlinear activation function. This design can effectively capture the local and global features of the vascular structure, thereby achieving accurate watershed segmentation.

[0035] The topological watershed network module 23 is used to analyze the topological relationship of the vascular structure. This module is based on an improved graph neural network, which can not only accurately describe the spatial relationship of the blood vessels, but also capture functional information such as vascular flow. In practical applications, a graph neural network based on the attention mechanism is used, and its core formula is as follows: , , in, represents the attention coefficient of node i to neighbor node j, and are the feature vectors of nodes i and j respectively, W is the weight matrix, a is the learnable attention vector, is the activation function. This design can adaptively focus on important vascular connections and improve the accuracy of topological analysis.

[0036] The blood vessel segmentation module 21 adopts an improved U-Net model, including a multi-view two-dimensional feature fusion unit 211, an improved decoder unit 212 and an attention mechanism unit 213. The multi-view two-dimensional feature fusion unit 211 is used to fuse multi-scale features. In one embodiment of the present invention, a multi-scale feature fusion strategy is adopted. Specifically, three different sizes of convolution kernels, 1x1, 3x3 and 5x5, are used to extract features in parallel, and then these features are fused together. This method can simultaneously capture blood vessel structure information of different scales and improve the accuracy of segmentation.

[0037] The improved decoder unit 212 includes a skip connection structure for extracting features at different levels. In the present invention, not only the skip connection in the traditional U-Net is used, but also the dense connection is introduced. Specifically, each layer of the decoder receives not only the output of the previous layer, but also the output of all previous layers. This design can be expressed as: , in, represents the output of layer l, represents a nonlinear transformation, Indicates that the outputs of all previous layers are connected together. This design can more effectively utilize features at different levels and improve the accuracy of blood vessel segmentation.

[0038] The attention mechanism unit 213 is set at the end of the decoder to improve the feature extraction effect of important vascular structures. In the present invention, a method combining the spatial attention mechanism and the channel attention mechanism is adopted. The spatial attention mechanism can help the model focus on important areas in the image, while the channel attention mechanism can highlight important feature channels. The specific attention mechanism can be expressed as: , , , Among them, F represents the input feature map, and represent channel attention and spatial attention respectively, represents the sigmoid activation function, This design can effectively highlight important vascular structures and improve the accuracy of segmentation.

[0039] The hierarchical watershed segmentation module 22 includes a recursive partitioning unit 221 and a hierarchical feature selection strategy unit 222. The recursive partitioning unit 221 is used to recursively divide the lung lobe into multiple lung segments based on the blood vessel segmentation result. In one embodiment of the present invention, a recursive partitioning algorithm based on Graph Cut is adopted. First, the blood vessel segmentation result is represented as a graph G=(V,E), where V represents the nodes in the graph (corresponding to the pixels in the image) and E represents the edges between the nodes. Then, an energy function is defined: , Among them, A represents the segmentation result, represents a data item, measuring the pixel p assigned to the label The price, represents the smoothness term, which measures the cost of assigning adjacent pixels p and q to different labels. The optimal segmentation result is obtained by minimizing this energy function. This method can effectively utilize the topological structure information of blood vessels and achieve accurate lung segmentation.

[0040] The hierarchical feature selection strategy unit 222 is used to adaptively select the best features. In the present invention, a dynamic feature selection mechanism is designed. Specifically, different feature combinations are used for segmentation tasks at different levels. For example, for segmentation at the lobe level, deep semantic features are mainly used; while for segmentation at the segment level, shallow detail features and deep semantic features are combined. This strategy can be expressed as: , in, represents the features of the lth level, represents the feature map of the i-th layer, are learnable weights. By learning these weights, the model can adaptively select the most appropriate feature combination to improve the accuracy and robustness of segmentation.

[0041] Through the above design, the deep learning-based adaptive watershed segmentation and surgical planning system of the present invention can achieve high-precision pulmonary vascular segmentation and lung segmentation, providing a reliable basis for subsequent surgical planning. The multi-module collaborative design of the system not only improves the accuracy of segmentation, but also enhances the adaptability and robustness of the system, and can better cope with the individual differences of different patients.

[0042] The topological watershed network module 23 of the present invention is based on an improved graph neural network, and includes a vascular structure modeling unit 231, a topological relationship analysis unit 232, and a flow analysis unit 233. The vascular structure modeling unit 231 is used to jointly model the vascular network and the pulmonary anatomical structure. In a preferred embodiment of the present invention, the unit adopts a hierarchical graph structure representation method. Specifically, the method first represents the entire pulmonary vascular network as a large graph structure G, and then divides it into multiple subgraphs G_i based on anatomical knowledge, each subgraph corresponding to a lung segment. This hierarchical representation method can be expressed in the following mathematical form: , , in, and Represent the node sets of the entire graph and subgraph respectively, and represents the edge set, and Represents a set of attributes. Preferably, node attributes include information such as spatial position and vessel diameter, and edge attributes include features such as length and curvature. This representation method can not only accurately describe the topological structure of the blood vessel, but also retain important anatomical information, laying the foundation for subsequent analysis.

[0043] The topological relationship analysis unit 232 is used to obtain the topological relationship of the blood vessels. The method of the present invention introduces an attention mechanism in this unit to better capture the key topological structure. Specifically, the method defines an attention matrix A, whose elements Represents the attention weight of node i to node j. The calculation formula of attention weight is as follows: , in, and denote the feature vectors of nodes i and j respectively, is a learnable parameter matrix, LeakyReLU is the activation function, and || represents a vector concatenation operation. In this way, the method can adaptively focus on important vascular connections and improve the accuracy of topological analysis.

[0044] The flow analysis unit 233 is used to perform blood vessel flow analysis. In a preferred embodiment of the present invention, the unit adopts a method based on a combination of physical models and deep learning. First, the method uses a simplified Navier-Stokes equation to simulate the flow of blood in the blood vessel: , , Where ρ is the blood density, v is the velocity vector, p is the pressure, and μ is the viscosity. Then, this method uses a graph neural network to learn and predict the flow of each vascular branch. Specifically, for each node i, its flow prediction can be expressed as: , in, is a parameterized function (like a multilayer perceptron), is the feature of node i, is the neighbor set of node i. In this way, the method can comprehensively consider the geometric characteristics and topological structure of the blood vessels to obtain more accurate flow estimation.

[0045] The watershed visualization module 24 of the present invention includes a target selection unit 241, a dynamic display unit 242 and a solution generation unit 243. The target selection unit 241 is used to select a target of interest based on the hierarchical blood vessel segmentation result. In one embodiment of the present invention, the unit adopts an automatic selection method based on importance scoring. Specifically, the method calculates an importance score S for each lung segment: , Among them, V represents the volume of the lung segment, F represents the total blood flow through the lung segment, and C represents the complexity of the connection with the surrounding lung segments. , and is the weight coefficient. Preferably, , and It can be adjusted according to specific clinical needs. For example, in lung cancer surgery planning, more attention may be paid to the volume and blood flow of the lung segment, so =0.4, .

[0046] The dynamic display unit 242 is used to dynamically display the flow relationship between the currently selected pulmonary segment blood vessels and the adjacent pulmonary segment blood vessels in combination with the vascular flow analysis results. The method of the present invention uses interactive 3D visualization technology in this unit. Specifically, the method uses color and thickness to represent the flow of blood vessels, and uses animation effects to show the dynamic changes of blood flow. For example, the blood flow can be mapped to a gradient color spectrum from red to blue. The greater the flow, the more the color tends to red; at the same time, the thickness of the blood vessel is also proportional to the flow. This intuitive visualization method can help doctors quickly understand the complex vascular network structure and blood flow distribution.

[0047] The solution generation unit 243 is used to generate feasible personalized surgical options. In a preferred embodiment of the present invention, the unit adopts a solution generation algorithm based on multi-objective optimization. Specifically, the method considers the following objectives: minimizing surgical trauma, maximizing the retention of healthy tissue, and ensuring adequate blood supply. This multi-objective optimization problem can be expressed as: , , , in, , and Respectively represent the degree of surgical trauma, loss of healthy tissue and insufficient blood supply. and Represents various constraints. This method uses the improved NSGA-II algorithm to solve this multi-objective optimization problem, thereby generating a series of alternative surgical plans.

[0048] The path planning module 25 of the present invention includes a path planning algorithm unit 251, a blood vessel centerline fitting unit 252 and a reinforcement learning optimization unit 253. The path planning algorithm unit 251 is used to design a path planning algorithm according to the target of interest. In one embodiment of the present invention, the unit adopts an improved A Specifically, this method defines a comprehensive cost function: , Among them, g(n) is the actual cost from the starting point to the current node n, h(n) is the estimated cost from the current node to the target, and r(n) is an additional cost term related to risk. Preferably, r(n) can be calculated based on the distance to important structures (such as large blood vessels) near node n. For example, r(n)=k / d can be set, where d is the distance from node n to the nearest important structure, and k is an adjustable parameter. By introducing this risk term, this method can take into account surgical safety while finding the shortest path.

[0049] The vessel centerline fitting unit 252 is used to fit the vessel centerline and the vessel branch tangent line along the vessel direction starting from the target nodule. The method of the present invention adopts a fitting algorithm based on principal component analysis (PCA) in this unit. Specifically, for each cross section of the vessel, the method first extracts the vessel contour points and then performs PCA analysis on these points. The vessel center can be approximated as the mean of these points, and the vessel direction can be given by the first principal component. By gradually advancing along the vessel direction, the method can accurately fit the centerline of the vessel.

[0050] The objective function used by the reinforcement learning optimization unit 253 of the present invention is: , In a preferred embodiment of the present invention, these loss terms can be specifically defined as follows: , Where E is the edge set in the vascular network, Indicates from action Transfer to Action probability.

[0051] , in, is the vessel diameter at node i, is the action taken at node i.

[0052] , Where B is the set of vascular branch points, Indicates in status Next select action probability.

[0053] , in, is the node’s neighbor set, is an indicator function. By minimizing this comprehensive objective function, the method of the present invention can generate an optimal path that not only follows the vascular anatomical structure but also meets the surgical requirements. This path planning method based on reinforcement learning has strong adaptability and robustness, and can effectively cope with complex and changeable vascular structures.

[0054] By minimizing this comprehensive objective function, the method of the present invention can generate an optimal path that not only follows the vascular anatomical structure but also meets the surgical requirements. This path planning method based on reinforcement learning has strong adaptability and robustness, and can effectively cope with complex and changeable vascular structures.

[0055] The evaluation and decision module 26 of the present invention includes a comprehensive analysis unit 261, a strategy analysis unit 262 and a decision support unit 263. The comprehensive analysis unit 261 is used to evaluate the generated surgical planning scheme based on the path planning results and the vascular topology analysis results. In a preferred embodiment of the present invention, the unit adopts a method based on multi-dimensional scoring. Specifically, the method defines a comprehensive scoring function S: , in, represents the safety score, represents the effectiveness score, represents the difficulty rating, , and is the corresponding weight coefficient. Preferably, these scores can be calculated based on specific clinical indicators. For example, the safety score can consider the minimum distance between the planned path and important structures; the effectiveness score can consider the coverage of the target area; the difficulty score can consider factors such as the complexity of the path and the limitation of the operating space. By adjusting the weight coefficient, the method can flexibly adjust the evaluation criteria according to different clinical needs.

[0056] The strategy analysis unit 262 is used to analyze the surgical strategy. The method of the present invention introduces a decision tree-based strategy analysis method in this unit. Specifically, the method first constructs a decision tree, in which each node represents a decision point and each edge represents a possible choice. Then, for each leaf node (i.e., the final surgical strategy), its expected utility is calculated: , Among them, s represents the strategy, Indicates possible outcomes, Indicates the result when strategy s is adopted The probability of occurrence, Display results This approach can help doctors comprehensively evaluate the potential risks and benefits of different surgical strategies.

[0057] The decision support unit 263 is used to provide comprehensive analysis results to assist doctors in making surgical decisions. In a preferred embodiment of the present invention, the unit adopts a decision support system based on fuzzy logic. Specifically, the method first fuzzifies each evaluation index, then makes inferences through a series of fuzzy rules, and finally obtains a recommended decision. The general form of fuzzy rules can be expressed as: IF (condition 1 is A) AND (condition 2 is B) THEN (decision is C), Among them, A, B and C are fuzzy sets. For example, a specific rule may be: IF (security is high) AND (effectiveness is medium) AND (difficulty is low) THEN (recommendation for execution is strong), In this way, this method can transform quantitative analysis results into qualitative recommendations that are easier to understand and implement, providing strong support for doctors' decision-making.

[0058] The present invention also includes a real-time verification module 27 for simulating an actual surgical procedure and performing real-time evaluation. In one embodiment of the present invention, the real-time verification module 27 first uses 3D printing technology to directly print the generated surgical path at a specific location of the human lung to simulate the actual surgical procedure. This method can intuitively display the location and direction of the surgical path in the actual anatomical structure, which helps doctors better understand and evaluate the surgical plan.

[0059] Preferably, the method images the slices on a CT device at a microscopic level and compares and verifies the virtual surgical path. Specifically, the method uses image registration technology to align the virtual path with the actual CT image. The registration process can be expressed as an optimization problem: , in, represents a virtual path image, represents the actual CT image, T represents the transformation function, and D represents the image similarity measure. By solving this optimization problem, the proposed method can accurately map the virtual path to the actual CT image, thereby performing quantitative comparison and analysis.

[0060] In addition, the method of the present invention can adjust the virtual path in real time according to the deviations that occur during the actual operation. To this end, the method introduces a dynamic path adjustment algorithm. The algorithm is based on the Kalman filter and can be expressed as: , , in, is the path state at time k, is the control input, is the observed value, , and They are the state transfer matrix, control input matrix and observation matrix, and They are process noise and observation noise. In this way, the method can flexibly adapt to various changes in actual surgery while maintaining the main features of the original planned path.

[0061] The end-to-end multi-task deep learning model layer 2 of the present invention uses a knowledge distillation module 28 to improve model performance and robustness. In a preferred embodiment of the present invention, the knowledge distillation module 28 uses soft labels designed by experts as target labels to train the model. Specifically, this method defines a distillation loss function: , in, represents the cross entropy loss, KL represents the KL divergence, y is the true label, and are the output logits of the student model and the teacher model, σ is the softmax function, T is the temperature parameter, and α is the balance factor. By adjusting the temperature parameter T, this method can control the softness of knowledge transfer, thereby achieving a good balance between model performance and generalization ability.

[0062] In addition, in order to enhance the role of vascular information in the path planning process, the method of the present invention introduces a vascular attention mechanism in the knowledge distillation process. Specifically, the method adds a vascular attention module to the teacher model, and its output can be expressed as: , Among them, F is the input feature map, , , and is a learnable parameter. Then, this method passes this attention information to the student model as an additional soft label. This method can guide the student model to better focus on important vascular structures, thereby improving the accuracy of path planning.

[0063] Finally, in order to further improve the accuracy of the algorithm and its ability to absorb surgical experience, the method of the present invention adopts a progressive knowledge distillation strategy. Specifically, the method first trains the student model with simple tasks, and then gradually increases the complexity of the tasks. At each stage, a teacher model of corresponding complexity is used to guide the learning of the student model. This process can be expressed as: , Where N is the total number of tasks, is the weight of the ith task, is the distillation loss of the ith task. Through this progressive learning strategy, this method can more effectively transform complex surgical experience into the capabilities of the model, thereby achieving better performance and stronger generalization ability.

[0064] Through the above design, the deep learning-based adaptive watershed segmentation and surgical planning system of the present invention can not only generate high-quality surgical plans, but also perform real-time verification and adjustment in practical applications, while continuously improving its own performance through knowledge distillation technology. This comprehensive and advanced design enables the system to provide reliable planning and decision support for complex lung surgeries, which has important clinical application value.

[0065] The above description is only a preferred specific implementation manner of the present invention; however, the protection scope of the present invention is not limited thereto; any person familiar with the art who, within the scope disclosed by the present invention, makes equivalent replacements or changes based on the scheme and improved concepts of the present invention shall be covered within the protection scope of the present invention.

Claims

1. Adaptive watershed segmentation and surgical planning system based on deep learning, characterized by: include: A chest imaging data modality layer, used to obtain chest imaging data modality information including lung lobes, lung segment boundaries and lung blood vessels; An end-to-end multi-task deep learning model layer, in communication with the chest imaging data modality layer, comprises: A blood vessel segmentation module, used for performing blood vessel segmentation on the chest imaging data modality; A hierarchical watershed segmentation module, used for performing hierarchical watershed segmentation based on the blood vessel segmentation result; Topological watershed network module, used to analyze the topological relationship of vascular structures; The watershed visualization module is used to generate visualization results of lung lobes and segments; Path planning module, used to generate surgical plans; Evaluation and decision-making module, used to evaluate and analyze surgical plans; Among them, the end-to-end multi-task deep learning model layer is used to segment and evaluate the chest imaging data modality to obtain visualization of lung lobes and segments, vascular network topology and surgical plans, and perform decision analysis.

2. The system according to claim 1, characterized in that The blood vessel segmentation module adopts an improved U-Net model, including: Multi-view 2D feature fusion unit, used to fuse multi-scale features; An improved decoder unit with skip connections to extract features at different levels; The attention mechanism unit is set at the end of the decoder to improve the feature extraction effect of important vascular structures.

3. The system according to claim 1, characterized in that The hierarchical watershed segmentation module includes: A recursive division unit, used for recursively dividing the lung lobe into a plurality of lung segments based on the blood vessel segmentation result; Hierarchical feature selection strategy unit, used to adaptively select the best features; Among them, the lung segment used to divide the lung lobes selects the upsampled blood vessel features, and the lung segment used to divide the lung segments selects the upsampled blood vessel features and the lung segment features of this layer.

4. The system according to claim 1, characterized in that The topological watershed network module is based on an improved graph neural network and includes: The vascular structure modeling unit is used to jointly model the vascular network with the lung anatomical structure; A topological relationship analysis unit, used to obtain the topological relationship of blood vessels; A flow analysis unit is used to perform blood vessel flow analysis.

5. The system according to claim 1, characterized in that The watershed visualization module includes: A target selection unit, used to select a target of interest based on the hierarchical blood vessel segmentation result; A dynamic display unit, used to dynamically display the flow relationship between the currently selected pulmonary segment blood vessels and the adjacent pulmonary segment blood vessels in combination with the vascular flow analysis results; A plan generation unit is used to generate feasible personalized surgical options.

6. The system according to claim 1, characterized in that The path planning module includes: A path planning algorithm unit, used to design a path planning algorithm according to a target of interest; A vascular centerline fitting unit is used to fit the vascular centerline and vascular branch tangent line starting from the target nodule along the vascular direction; The reinforcement learning optimization unit is used to optimize the resection path planning by using the reinforcement learning algorithm.

7. The system according to claim 6, characterized in that The objective function adopted by the reinforcement learning optimization unit is: , in, is the loss degree of the vascular structure topology network, is the loss matrix of blood vessel diameter, is the loss matrix of the blood vessel branch, The loss matrix is ​​used to select the blood vessel branch with the largest diameter when the path enters the blood vessel branch.

8. The system according to claim 1, characterized in that The evaluation and decision-making module includes: A comprehensive analysis unit, used to evaluate the generated surgical planning scheme according to the path planning results and the vascular topology analysis results; Strategy analysis unit, used to analyze surgical strategies; The decision support unit is used to provide comprehensive analysis results to assist doctors in making surgical decisions.

9. The system according to claim 1, characterized in that Also includes real-time verification modules for: Use 3D printing technology to print the generated surgical path directly at the specific location of the human lung to simulate the actual surgical process; At the microscopic level, the slices are imaged on a CT device and compared and verified with the virtual surgical path; The virtual path is adjusted in real time according to deviations that occur during the actual surgery.

10. The system according to claim 1, characterized in that The end-to-end multi-task deep learning model layer adopts a knowledge distillation module for: Train the model using soft labels designed by experts as target labels; Reduce the difficulty of optimizing the target and improve the algorithm's robustness to noise; Improve the role of vascular information in the path planning process; Improve the accuracy of the algorithm and its ability to absorb surgical experience.

Images