Airway intervention operation auxiliary decision system based on chest ct and bronchoscopy image

Abstract

The application relates to the technical field of medical image processing, and discloses an airway intervention operation auxiliary decision system based on chest CT and bronchoscopic images, which comprises an image preprocessing module, a path planning module, a diagnosis decision module, a treatment decision module and a learning optimization module; the image preprocessing module is used for segmenting a tracheal tree, fusing chest CT and bronchoscopic real-time images, and extracting a blood vessel running path and a lesion position feature; the path planning module is used for calculating an optimal path, and performing navigation, dynamic re-planning and risk early warning during an operation; the diagnosis decision module is used for predicting a diagnosis positive rate according to a lesion feature and outputting an operation strategy; the treatment decision module is used for formulating a treatment mode, matching an operation tool, and predicting curative effects, complications and comprehensive benefits; and the learning optimization module is used for recording whole-process data, training a prediction model and correcting a decision rule. The application can improve the positioning accuracy, safety and individualized decision level of airway intervention operation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and more specifically, to an airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images. Background Technology

[0002] Bronchoscopy and endobronchial interventions are widely used in the diagnosis and treatment of diseases such as pulmonary nodules, lung cancer, airway stenosis, airway foreign bodies, and hemoptysis. Chest CT can display structures such as the tracheal tree, lesions, blood vessels, and pleura, while bronchoscopic images can display the airway lumen in real time. However, in practice, the two are usually relatively independent. Surgeons need to rely on experience to match preoperative CT images with intraoperative bronchoscopic images, which is easily affected by factors such as complex airway branches, deep lesion locations, visual field obstruction, and anatomical variations.

[0003] While some existing navigation-guided bronchoscopy systems can reconstruct the tracheal tree based on CT and provide path guidance, they primarily focus on static path planning. They do not adequately consider risk factors such as the course of blood vessels around the lesion, pleural distance, and areas with weak airway walls. Furthermore, they struggle to dynamically replan in time when airway obstruction, deviation from the planned route, or changes in lesion exposure occur during the procedure. Therefore, risks such as bleeding, pneumothorax, airway injury, and sampling failure remain when operating on lesions near blood vessels, the pleura, or areas with fragile airways.

[0004] Furthermore, existing technologies have limited ability to assist in decision-making for diagnostic and treatment procedures. The selection of procedures such as biopsy, brushing, irrigation, needle aspiration, as well as ablation, balloon dilation, stent placement, foreign body removal, and hemostasis largely relies on physician experience. There is a lack of mechanisms to predict positive rates, technical success rates, clinical efficacy, and complication probabilities by combining factors such as lesion size, location, density, blood supply, accessibility, and risk indicators. There is also a lack of individualized recommendations for parameters such as tool type, operating angle, number of samplings, ablation power, and stent size.

[0005] Meanwhile, existing systems typically lack postoperative learning optimization mechanisms, making it difficult to use imaging data, path selection, actual doctor operations, diagnostic results, treatment effects, and complications from each operation for model retraining and decision rule correction.

[0006] Therefore, there is an urgent need for an airway interventional decision support system based on chest CT and bronchoscopic images to solve these problems. Summary of the Invention

[0007] The purpose of this invention is to solve the technical problems mentioned in the background section and to provide an airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images.

[0008] The above-mentioned objective of the present invention is achieved as follows:

[0009] An airway interventional procedure support decision-making system based on chest CT and bronchoscopic images, the system comprising:

[0010] The image preprocessing module is used to segment the tracheal tree and perform real-time fusion of multimodal images to reconstruct three-dimensional tracheal tree CT images, and extract the vascular pathways and lesion location characteristics.

[0011] The path planning module is used to calculate the optimal path based on the information extracted by the image preprocessing module, and to realize navigation, dynamic replanning and risk warning during surgery.

[0012] The diagnostic decision module is used to predict the positive rate based on lesion characteristics and output operational strategy suggestions.

[0013] The treatment decision module is used to formulate operation methods and match operation tools according to the operation type and lesion characteristics, and to provide risk warnings and predict efficacy and benefits.

[0014] The learning optimization module is used to record the entire process data after each operation, retrain the prediction model, and correct its own decision rules through comparative learning.

[0015] Furthermore, the path planning module includes:

[0016] The preoperative planning submodule is used to calculate the optimal path from the main bronchus to the target lesion based on the segmented tracheal tree, vascular path and lesion location characteristics. The requirements are to be closest to the lesion, with the fewest turns and the smallest angle, and to be far away from blood vessels, airway weak areas and pleura, as well as the predetermined contact angle between the operating tool and the lesion.

[0017] The intraoperative navigation submodule is used to identify the characteristics of tracheal branches in the bronchoscopy image in real time to determine the current location, and to overlay navigation arrows on the screen to indicate the direction of travel. If it deviates from the planned route, it will issue a warning.

[0018] The dynamic replanning submodule is used to automatically calculate alternative routes when airway obstruction or changes in the actual exposure of lesions occur during surgery, provide voice prompts, and display navigation arrows and distance prompts on the display screen.

[0019] Furthermore, the path planning module also includes a risk warning submodule, which includes:

[0020] The bleeding risk assessment unit is used to calculate based on the vascular path extracted by the image preprocessing module: if the lesion is surrounded by blood vessels, it is marked as high risk (red); if the blood vessels are far from the lesion, it is marked as low risk (green); and a quantitative risk value or risk heatmap between 0 and 1 is given based on the surrounding vascular density, diameter, and blood supply richness.

[0021] The pneumothorax risk assessment unit is used to measure the distance between the lesion and the lung surface. If the distance is less than a set value, it indicates a risk of pneumothorax.

[0022] The airway injury risk assessment unit is used to assess airway wall thickness, cartilage integrity, and the presence of airway mucosal necrosis, erosion, or fragile areas.

[0023] Furthermore, the diagnostic decision module includes:

[0024] The operation strategy submodule is used to output parameter suggestions for the following operations based on the size, location, density, and blood supply characteristics of the lesion, through built-in decision rules: biopsy tool type, number of pieces, and angle; number of brushing operations and specific location; irrigation location, irrigation volume, and submitted items; and specific location, depth, and number of needle aspiration operations.

[0025] The positive rate prediction submodule is used to output a predicted value of 0-100% and factors affecting the positive rate based on the imaging parameters and operation parameters of the lesion using an AI model. If the predicted value is lower than the threshold of 50%, the operation method or the number of samplings will be changed.

[0026] Furthermore, the treatment decision module includes:

[0027] The operation method submodule is used to decide on the following treatment methods based on the characteristics of the lesion: tumor ablation, balloon dilation or stent placement for airway stenosis, forceps or cryoablation of airway foreign bodies, hemostasis with ice-cold saline, drugs or electrocoagulation;

[0028] The operation tool submodule is used to match the tool parameter library based on the operation type and lesion characteristics, and output the tool name, specific parameters and alternative solutions, including stent type, diameter, length, or ablation probe size, ablation time and power.

[0029] Furthermore, the treatment decision module also includes a risk warning submodule, which has the same function as the risk warning submodule in the path planning module and is used to assess the risk of bleeding, pneumothorax, and airway injury.

[0030] Furthermore, the treatment decision module also includes an efficacy and benefit prediction submodule, which includes:

[0031] The technology success rate prediction unit is used to predict the success rate of the operation as planned based on the patient's lung pathway conditions and the degree of tool compatibility.

[0032] Clinical efficacy prediction unit is used to predict the degree of improvement in symptoms or lung function based on the nature and severity of the patient's lesions, general condition, and imaging findings.

[0033] Complication rate prediction unit, used to predict the probability of adverse events based on lesion location, operation type, general condition and laboratory test results;

[0034] The overall profitability calculation unit is used to calculate the overall profitability based on the above three data points and an AI algorithm.

[0035] Furthermore, the learning optimization module includes:

[0036] The prediction model training unit is used to repeatedly retrain the positive rate prediction model, risk prediction model, and efficacy prediction model.

[0037] The decision rule correction unit is used to correct the system's decision rules through comparative learning when the system's suggestions are inconsistent with the doctor's actual choices and the clinical outcome of the doctor's actual operation plan is better than the system's recommended plan. This includes updating the matching logic of the tool recommendations or the priority of operation method selection.

[0038] The rare case collection unit is used to store rare cases or special autopsies as rare samples in the database for subsequent operation reference.

[0039] Furthermore, the intraoperative navigation submodule uses a reminder method analogous to the guidance method of navigation software; the distance prompts in the dynamic replanning submodule include prompts for the number of remaining tracheal branches.

[0040] Furthermore, the vascular pathways and lesion location characteristics extracted by the image preprocessing module are used by the path planning module to calculate the optimal path, the diagnostic decision module to match operation strategies, and the treatment decision module to formulate treatment plans and assess risks.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. This invention fuses chest CT images and real-time bronchoscopic images through an image preprocessing module. This allows for the reconstruction of the tracheal tree, extraction of vascular pathways, and identification of lesion locations within the same spatial coordinate system. This eliminates the need for surgeons to rely solely on experience to manually match preoperative CT images with intraoperative bronchoscopic images. The path planning module comprehensively considers factors such as distance to reach, number of airway branches, turning angles, vascular distance, pleural distance, and weak airway areas to calculate the optimal path for interventional procedures. During the procedure, navigation arrows, voice prompts, and information on the remaining branch count assist the surgeon, thereby improving lesion localization accuracy and operational efficiency.

[0043] 2. The risk warning, diagnostic decision-making, and treatment decision-making functions designed in this invention can transform information such as lesion size, location, density, blood supply characteristics, vascular distribution, pleural distance, and airway wall condition into quantifiable risk values ​​and decision-making basis. The system can provide early warnings of risks such as bleeding, pneumothorax, and airway injury, and recommend diagnostic strategies such as biopsy, brushing, irrigation, and needle aspiration based on different lesion characteristics, predicting the positive diagnostic rate. At the same time, it can match appropriate tools and parameters for treatment operations such as tumor ablation, balloon dilation, stent placement, foreign body removal, and hemostasis, predicting the technical success rate, clinical effectiveness, complication rate, and overall benefit rate, thereby improving the individualization and safety of the diagnosis and treatment plan.

[0044] 3. This invention also records the entire process data of each operation through a learning optimization module, including preoperative images, intraoperative pathways, system suggestions, actual doctor's operation, diagnostic results, efficacy results, and complication status. This data is then used to repeatedly train the positive rate prediction model, risk prediction model, and efficacy prediction model. When the doctor's actual choice differs from the system's suggestion, and the clinical outcome of the doctor's actual procedure is better than the system's recommended approach, the system can correct its decision-making rules through comparative learning and store rare cases or special anatomy in the database for future reference. Thus, the system can form a closed loop from image recognition, pathway planning, diagnostic and treatment decisions to postoperative feedback optimization, improving the accuracy and adaptability of auxiliary decision-making during long-term use. Attached Figure Description

[0045] Figure 1 This is a system block diagram of the airway interventional operation auxiliary decision-making system based on chest CT and bronchoscopic images in an embodiment of the present invention.

[0046] Figure 2 This is a diagram showing the composition of the risk warning submodule of the airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images in an embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with embodiments and appendices. Figure 1-2 The present invention will be further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] Example 1: This example proposes a decision support system for endotracheal interventional procedures based on chest CT and bronchoscopic images. This system, through the coordinated efforts of multiple modules including image preprocessing, pathway planning, diagnostic decision-making, treatment decision-making, and learning optimization, aims to provide physicians with real-time decision support, optimize pathway planning, operational strategies, and treatment plans for endotracheal interventional procedures, reduce operational risks, and improve the success rate. The functions and workflows of each module and unit of the system are as follows:

[0049] The image preprocessing module is one of the fundamental modules of the system. Its function is to process the input chest CT images and bronchoscopic images, extracting the tracheal tree, vascular pathways, and lesion location characteristics. Specifically, this module first segments the tracheal tree region using image segmentation technology and extracts the various branches of the trachea. Then, using multimodal image fusion technology, the CT images and bronchoscopic images are combined to generate high-quality 3D reconstructed images, thereby providing accurate image data for subsequent pathway planning and diagnostic decisions.

[0050] The path planning module calculates the operational path based on information extracted by the image preprocessing module. First, the preoperative planning submodule calculates the optimal path from the main bronchus to the target lesion based on the segmented tracheal tree, vascular pathways, and the specific location of the lesion. The standard for path planning requires the path to be closest to the lesion, with the fewest turns and the smallest angle, while avoiding passage through blood vessels, airway weak areas, and the pleural region, and maintaining the predetermined contact angle between the operating tool and the lesion. The intraoperative navigation submodule tracks the bronchoscope image in real time, determines the current position by identifying the characteristics of tracheal branches, and overlays navigation arrows on the display screen to indicate the current direction of travel. If the bronchoscope deviates from the planned path, the system will issue a warning and adjust the path. The dynamic replanning submodule automatically calculates alternative routes and provides voice prompts when airway obstruction or changes in the actual exposure of the lesion occur during surgery, while simultaneously displaying new path information and distance indicators.

[0051] The diagnostic decision module is responsible for predicting the positive rate based on lesion characteristics and providing operational strategy suggestions. This module, through its operational strategy sub-module, outputs corresponding operational strategy suggestions based on the lesion's size, location, density, blood supply characteristics, and imaging parameters, combined with built-in decision rules. These suggestions include the type, number, and angle of biopsy tools; the number and location of brushing; the irrigation location and volume; and the specific location and depth of needle aspiration. Simultaneously, the positive rate prediction sub-module, based on the lesion's imaging and operational parameters, uses an AI model to output a predicted positive rate and the factors influencing the positive rate. If the predicted value is below the 50% threshold, it will suggest changing the operational method or increasing the number of sampling attempts.

[0052] The treatment decision-making module is responsible for determining the appropriate procedure and matching the right tools based on the type of procedure and the characteristics of the lesion, as well as providing risk warnings and predicting treatment efficacy. The procedure method submodule automatically selects the most suitable treatment method based on the different characteristics of the lesion (e.g., tumors, airway stenosis), including tumor ablation, balloon dilation for airway stenosis, stent placement, and airway foreign body removal. The procedure tool submodule selects the most suitable tool from the tool parameter library based on the selected procedure and the specific condition of the lesion, providing the tool's name, specific parameters, and alternative options, including stent type, diameter, length, ablation probe size, ablation time, and power. This module also works in conjunction with the risk warning submodule of the pathway planning module to assess the risks of bleeding, pneumothorax, and airway injury, ensuring the safety of the treatment process.

[0053] The learning and optimization module continuously optimizes the system by recording data from each operation. This module includes a predictive model training unit, a decision rule correction unit, and a rare case collection unit. The predictive model training unit gradually improves the system's predictive accuracy by repeatedly training the positive rate prediction model, risk prediction model, and efficacy prediction model. The decision rule correction unit, through comparative learning, corrects its decision rules when discrepancies arise between the system's recommended treatment and the doctor's actual choice, ensuring that future decisions are more aligned with clinical practice. The rare case collection unit stores specific cases and anatomical structures in a database for subsequent system use, enabling better handling of rare cases and anatomical features.

[0054] Overall, through the collaborative work of its various modules, the system can provide precise operational pathways, optimal treatment plans, and comprehensive risk assessments in real time based on image data. By integrating modules for image preprocessing, pathway planning, diagnostic decision-making, treatment decision-making, and learning optimization, the system not only improves the success rate of endotracheal interventions but also reduces surgical risks. Furthermore, after each procedure, the system continuously optimizes itself based on actual operational data, gradually improving decision-making rules and predictive models to adapt to ever-changing clinical needs.

[0055] Example 2: This example provides an airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images. The system receives the patient's chest CT images before the intervention and receives real-time bronchoscopic images during the intervention. It integrates the tracheal tree structure, vascular pathways, and lesion location characteristics obtained from the chest CT images with the real-time bronchoscopic images, enabling the system to complete preoperative path planning, intraoperative navigation, risk warning, diagnostic operation suggestions, treatment operation suggestions, efficacy and benefit prediction, and postoperative learning optimization in the same spatial coordinate system.

[0056] The system does not rely on a single image parameter to make a judgment. Instead, it uses the basic data extracted by the image preprocessing module as a unified input and passes it to the path planning module, diagnosis decision module, treatment decision module, and learning optimization module respectively, so that a continuous data call relationship is formed between the modules.

[0057] In practice, the image preprocessing module first reads the patient's chest CT image data, denoted as... ,in This represents the patient's three-dimensional volumetric data from a chest CT scan. The system... After denoising, window width and level standardization, and slice thickness uniformity, the trained segmentation model was used to segment the tracheal tree, blood vessels, and lesions, yielding the following results:

[0058]

[0059] In the formula, This represents a 3D mask of the tracheal tree obtained by segmenting chest CT images. This represents a three-dimensional mask of blood vessels obtained from segmentation of chest CT images. This represents a three-dimensional mask of the lesion obtained by segmenting chest CT images; Represents an image segmentation model; These represent the training parameters of the image segmentation model; This represents the input 3D chest CT image data. The result is obtained through this formula. , , It serves as the foundational data for subsequent pathway planning, risk assessment, diagnostic decisions, and treatment decisions.

[0060] The system obtains a 3D mask of the tracheal tree. Next, the tracheal tree is extracted by centerline extraction and converted into an airway topology map:

[0061]

[0062] In the formula, Represents the tracheal tree topology; This represents the set of airway nodes in the tracheal tree, each of which... This indicates an airway bifurcation point, airway endpoint, or accessible sampling point. Represents the airway side assembly, each of which Represents a node With nodes A section of the airway is passable; This represents the total number of airway nodes. (Airway topology diagram) By tracheal tree mask The resulting topology map serves as the basis for path calculation in subsequent preoperative planning, intraoperative navigation, and dynamic replanning submodules.

[0063] At the same time, the system uses a three-dimensional mask of the lesion. Calculate the lesion center, lesion volume, maximum lesion diameter, average CT value of the lesion, and the spatial relationship between the lesion and airway vessels and pleura. The formula for calculating the lesion center is:

[0064]

[0065] In the formula, Indicates the coordinates of the center point of the lesion; Indicates lesion masking The set of all lesion voxels; express The three-dimensional coordinates of any voxel point in the lesion; This indicates the number of voxels in the lesion. The lesion center obtained from this formula... It will serve as the endpoint for path planning, the benchmark point for calculating lesion distance, the target point for diagnostic sampling, and the treatment target point.

[0066] The formula for calculating lesion volume is:

[0067]

[0068] In the formula, Indicates the volume of the lesion; Indicates the number of lesion foci; This represents the actual volume corresponding to a single CT voxel.

[0069] The formula for calculating the maximum diameter of the lesion is:

[0070]

[0071] In the formula, Indicates the maximum diameter of the lesion; and These represent the coordinates of any two voxel points within the lesion region; This represents the Euclidean distance. The formula for calculating the average CT value of a lesion is:

[0072]

[0073] In the formula, This represents the average CT value of the lesion; Voxel points representing lesions Grayscale values ​​or CT values ​​in chest CT images. (The above) , , , These features together constitute the basic imaging characteristics of the lesion. The subsequent diagnostic decision-making module will determine the appropriateness of biopsy, brushing, irrigation or needle aspiration based on these characteristics. The treatment decision-making module will also select ablation, balloon dilation, stent placement, foreign body removal or hemostasis methods based on these characteristics.

[0074] The system further calculates the distances between the lesion and the blood vessels, pleura, and target airway. The distance from the lesion to the nearest blood vessel is:

[0075]

[0076] In the formula, Indicates the shortest distance between the lesion and the nearest blood vessel; Represents the set of voxel points of the lesion; Indicates vascular mask The set of all vascular voxel points in the middle; Represents the coordinates of the lesion's voxel point; This represents the coordinates of the blood vessel voxel. The distance from the lesion to the pleura is:

[0077]

[0078] In the formula, This indicates the shortest distance between the lesion and the lung surface or pleura; This represents the set of voxel points on the pleura or lung surface extracted from CT images. This represents the coordinates of a voxel point on the pleura or lung surface. The distance from the lesion to the reachable endpoint of the airway is:

[0079]

[0080] In the formula, This represents the minimum distance from an airway topological node to the center of the lesion; Represents the first tracheal tree topology graph One airway node; Represents a node 3D coordinates; This represents the coordinates of the center point of the lesion. It is used to determine whether an intra-airway interventional tool can reach the lesion, and also for predicting the positive rate and technical success rate.

[0081] After the bronchoscope enters the airway, the system acquires real-time images under bronchoscopy, denoted as... ,in This indicates the current time. To ensure correspondence between 2D or localized images obtained during bronchoscopy and preoperative 3D CT structures, the system performs multimodal image registration based on airway bifurcation morphology, airway lumen contour, mucosal texture, airway diameter changes, and bronchoscopy pose information. The registration transformation is denoted as:

[0082]

[0084] In the formula, Indicates the current time Optimal registration transformation from lower bronchoscopic images to CT coordinate space; Represents the spatial transformation parameters to be solved; Represents the mutual information evaluation function; This indicates the current bronchoscopic image. According to the transformation parameters The image mapped to CT space; This represents the preoperative chest CT image. The formula is used to obtain... The real-time intraoperative images can be aligned with the reconstructed tracheal tree, vascular pathways, and lesion locations from the preoperative CT scan.

[0085] The fused real-time navigation imagery can be represented as:

[0086]

[0088] In the formula, This represents the fused image at the current moment; This represents a local CT reconstructed image captured based on the current position of the bronchoscopy. Indicates optimal registration transformation Mapped bronchoscopic images; This represents the CT image fusion weight, with a value ranging from 0 to 1; This indicates the weighting of the bronchoscopic image fusion. The fused image... It is used both for physician observation and for the intraoperative navigation submodule to determine the current position and degree of deviation. The path planning module obtains the airway topology map. Lesion center Distance between blood vessels pleural distance Airway reach distance Then, first calculate the edge of each airway segment. The cost of passage from node. To the node airway side Its cost function is:

[0089]

[0091] In the formula, Indicates airway side The overall cost of passage; Represents a node To node The length of the airway between them; This indicates the number of turns or bifurcation penalties incurred when entering the side of this airway from the previous airway. Indicates airway side The angle between the direction of the preceding airway; Represents a node Nearby bleeding risk value; Represents a node Nearby pneumothorax risk value; Represents a node Risk value for nearby airway damage; to These represent the weighting coefficients of length, turning angle, angle, bleeding risk, pneumothorax risk, and airway injury risk in pathway planning, respectively. This formula reflects that pathway planning not only pursues shorter distances but also considers fewer turns, smaller angles, distance from blood vessels and the pleura, and avoidance of airway weak areas.

[0092] Among them, bleeding risk value Calculated based on the distance to blood vessels around the lesion or nodule, blood vessel density, blood vessel diameter, and blood supply richness:

[0093]

[0095] In the formula, Represents a node The bleeding risk value of the area or its vicinity, ranging from 0 to 1; This represents the Sigmoidl normalization function; , , , These represent the weighting coefficients for vascular distance, vascular density, vascular diameter, and blood supply richness, respectively. Represents a node Or the distance from the target lesion to the nearest blood vessel; This indicates a very small positive number that prevents the denominator from being zero; Represents a node The number of blood vessel voxels per unit volume in the vicinity, i.e., blood vessel density; Represents a node Average radius or average diameter of nearby blood vessels; Represents a node Assessment of the richness of blood supply in the vicinity. If the lesion is surrounded by blood vessels, then... Smaller higher and Larger, calculated When the value is close to 1, the system marks it in red on the display interface; if the blood vessel is far from the lesion, the calculated value... Lower values ​​are indicated by the system in green and can be determined based on... Generate a risk heatmap from the 3D reconstructed image. (Pneumothorax risk value) Calculated based on the distance between the lesion or operative path and the pleura:

[0096]

[0098] In the formula, Represents a node Or the risk value of pneumothorax in the target operating area; , These represent the weighting coefficients for pleural distance and peripheral location factors, respectively. Represents a node Or the shortest distance from the target lesion to the pleura; This indicates a very small positive number that prevents the denominator from being zero; This score indicates the extent of the lesion's peripheral location; the closer the lesion is to the lung periphery, the higher the value. If... Less than the preset safety threshold The system will immediately warn of the risk of pneumothorax. This indicates the system's preset safe distance threshold from the lesion to the pleura.

[0099] airway injury risk value Calculations are based on airway wall thickness, cartilage integrity, necrotic or fragile areas, and the angle of manipulation:

[0100]

[0102] In the formula, Represents a node Risk value for nearby airway damage; , , , These represent the weighting coefficients for airway wall thickness, cartilage integrity, vulnerable areas, and operating angle, respectively. Represents a node Thickness of the nearby airway walls; Represents a node The score for the integrity of nearby airway cartilage is given; a higher score indicates more intact cartilage. The score indicates areas of necrosis, inflammation, erosion, or fragility. Indicates the angle at which the airway is entered; This represents a very small positive number. This risk value directly participates in the aforementioned path travel cost. The calculations automatically increase the cost of paths with thin airway walls, damaged cartilage, necrosis, or excessively large turns, thus reducing the probability of them being selected as the optimal path.

[0103] The preoperative planning submodule is based on the airway topology map. and the combined passage cost along each airway Calculate from the main bronchus inlet node To the target node The optimal path:

[0104]

[0106] In the formula, This represents the optimal path obtained from preoperative planning. Indicates the inlet node of the main bronchus To the target node The set of all candidate paths; Indicates the starting point of the path; Indicates distance from the center of the lesion The nearest target airway node that meets the conditions for medical device accessibility; Indicate candidate path A section of the airway; This represents the total passage cost along that airway. Because... This formula already incorporates factors such as distance, number of turns, angle, bleeding risk, pneumothorax risk, and airway injury risk. It also meets the requirements of being close to the lesion, having fewer turns, a smaller angle, being away from blood vessels, being away from the pleura, and avoiding areas with weak airways, as well as the predetermined contact angle between the operating tool and the lesion.

[0107] The intraoperative navigation submodule, after the bronchoscope enters the airway, uses real-time fused images to navigate. Current registration transformation And airway topology diagram To determine the current location of the bronchoscope, then:

[0108]

[0110] In the formula, This indicates the current location of the bronchoscopy or the nearest airway node; This represents a bronchoscopy positioning model. Indicates the currently fused image; Represents the tracheal tree topology; This indicates the current registration transformation. The system will... Compared with the optimal preoperative path Compare the current location with the planned path and calculate the degree of deviation.

[0111]

[0113] In the formula, This indicates the deviation of the current bronchoscope position from the optimal path; Indicates the current node 3D coordinates; Represents the optimal path Any node in; Represents a node The three-dimensional coordinates. If Greater than the preset deviation threshold The system determines that the bronchoscope has deviated from the planned route and overlays navigation arrows, turn prompts, voice prompts, and deviation warnings on the screen in a manner similar to navigation software. This indicates the maximum allowed path deviation distance.

[0114] To enable the operator to intuitively understand the remaining path, the system also considers the current node. In the optimal path Calculate the number of remaining tracheal branches based on their location:

[0115]

[0117] In the formula, This indicates the number of remaining tracheal branches that need to be traversed from the current position to the target node; Represents the optimal path Total number of nodes; Indicates the current node In the optimal path The system displays information such as "Which branch ahead is left turn?" and "How many tracheal branches remain to reach the target area?", making intraoperative navigation more similar to the experience of actual map navigation.

[0118] When situations arise during surgery, such as airway obstruction, secretion blockage, inability to pass through narrowed segments, or discrepancies between the actual exposed location of the lesion and the preoperative reconstruction location, the dynamic replanning submodule sets the passage cost of the obstructed airway to impassable or significantly increased.

[0119]

[0121] In the formula, This indicates the dynamically adjusted airway side passage cost. This represents the original passage cost calculated preoperatively. Indicates the obstacle penalty coefficient; Indicates airway side An obstruction occurs when the airway is blocked or impassable. ,otherwise The system then uses the current location node. Recalculate the alternative path as a new starting point:

[0122] ;

[0123] In the formula, This indicates the alternative path obtained through dynamic replanning during the operation. Indicates starting from the current node To the target node The set of all candidate paths; This represents the airway side passage cost after considering obstacles. The system obtains an alternative path. Afterwards, the operator is prompted by voice and on-screen arrows to detour around the affected area, while the remaining distance, the number of remaining tracheal branches, and changes in risk are displayed simultaneously.

[0124] Based on path planning and risk assessment, the diagnostic decision-making module further utilizes lesion characteristics, path accessibility, and operational parameters to generate a diagnostic operation strategy. The system will then consider the maximum diameter of the lesion. lesion volume Mean CT value of lesions Distance from lesion to blood vessel Distance from lesion to pleura Airway reach distance Bleeding risk risk of pneumothorax airway damage risk Combined into a diagnostic decision feature vector:

[0125]

[0127] In the formula, Represents the diagnostic decision feature vector; Indicates the maximum diameter of the lesion; Indicates the volume of the lesion; This represents the average CT value of the lesion; Indicates the distance from the lesion to the nearest blood vessel; Indicates the distance from the lesion to the pleura; Indicates the distance from the lesion to the nearest accessible airway node; Indicates the bleeding risk value of the target area; Indicates the risk value of pneumothorax in the target area; This indicates the airway damage risk value for the target area; Indicates the first The operating parameters for each diagnostic operation method, among which It can be used for biopsy, brushing, irrigation or needle aspiration.

[0128] For biopsy procedures This can include the type of biopsy tool, the number of sampling blocks, the sampling angle, and the sampling depth; for brushing operations, This can include the number of brushing operations and the brushing locations; for rinsing operations... This can include the irrigation location, irrigation volume, and items to be tested; for needle aspiration operations... This can include the aspiration location, aspiration depth, and number of aspirations. The system calculates a predicted positive rate for each candidate diagnostic procedure.

[0129]

[0131] In the formula, Indicates the first The positive rate predictive value for each diagnostic procedure, expressed as a percentage; Represents the Sigmoid function; This represents the bias parameter of the positivity rate prediction model; This represents the weight vector of the positivity rate prediction model; Represents the weight vector With diagnostic decision feature vector The product of these factors. As can be seen from this formula, the positive rate prediction is not calculated independently, but rather directly utilizes the aforementioned image preprocessing and risk assessment results, including... , , , , , , , , Parameters such as these.

[0132] The system then calculates a comprehensive recommendation score for different diagnostic procedures:

[0133]

[0135] In the formula, Indicates the first A comprehensive recommendation score for each diagnostic procedure; This indicates the predicted positive rate for this diagnostic procedure. , These represent the risks of bleeding, pneumothorax, and airway injury, respectively. Indicates the first Operational complexity score for each operation method; to These represent the weighting coefficients for positive rate, bleeding risk, pneumothorax risk, airway injury risk, and procedural complexity, respectively. System selection. The most popular operating method is prioritized and recommended, along with specific operating parameters. For example, when the overall recommendation score for biopsy is the highest, the system outputs the biopsy forceps type, suggested number of samples, sampling angle, and direction to avoid blood vessels; when the overall recommendation score for needle aspiration is the highest, the system outputs the needle aspiration position, needle depth, number of needle insertions, and safe angle range. If the highest overall recommendation score among all candidate methods is selected... Still below 50%, that is The system indicated that the positive rate was low and suggested changing the operation method, increasing the number of samplings, or combining brushing, irrigation, needle aspiration, and other methods.

[0136] The treatment decision module, based on the diagnostic decision module, selects the treatment method according to the nature of the lesion, the degree of airway narrowing, the location of the foreign body, the bleeding status, and airway patency. The system represents treatment-related characteristics as follows:

[0137]

[0139] In the formula, Represents the feature vector of treatment decisions; , , , , , , The meaning is consistent with the above; A score indicating the degree of airway narrowing; Indicates the degree of foreign object obstruction; This indicates a score representing the severity of bleeding. The treatment decision module is based on... Compatibility scores were calculated for candidate treatment modalities such as tumor ablation, balloon dilation, stent placement, foreign body removal, and hemostasis.

[0140]

[0142] In the formula, Indicates the first The suitability score for each treatment method; Indicates the candidate treatment method number; Indicates the first The degree to which the treatment method matches the indications for the current lesion or disease type; Indicates the first The expected treatment effect score for each treatment method; , , These represent the risks of bleeding, pneumothorax, and airway injury, respectively. to These represent the weighting coefficients for indication matching, expected outcome, and various risks, respectively. System Selection The treatment method that meets the highest safety threshold requirements is recommended. When the system determines that tumor ablation is appropriate, the operation tool submodule matches the ablation probe size, ablation power, and ablation time according to the lesion volume, maximum lesion diameter, and risk value.

[0143] The ablation power can be determined by the following formula:

[0144]

[0146] In the formula, Indicates the recommended ablation power; This represents a limiting function, used to restrict the calculation results to an allowable range; Indicates the basic ablation power; This represents the adjustment coefficient of the maximum diameter of the lesion to the ablation power; This represents the inhibition coefficient of bleeding risk on ablation power. This represents the inhibition coefficient of airway damage risk on ablation power; Indicates the maximum diameter of the lesion; Indicates the risk of bleeding; Indicates the risk of airway damage; and These represent the minimum and maximum permissible ablation power, respectively. The ablation time can be further determined based on the ablation power and lesion volume.

[0147]

[0149] In the formula, Indicates the recommended ablation time; Indicates the basic ablation time; This represents the adjustment coefficient between lesion volume and ablation time. Indicates the volume of the lesion; This represents the recommended ablation power obtained from the previous formula; Represents a very small positive number; and These represent the minimum and maximum permissible ablation times, respectively.

[0150] Therefore, the ablation time Directly call the ablation power obtained in the previous step The formulas have clear input-output relationships. When the system determines that airway stenosis requires stent placement, the operating tool submodule determines the stent diameter and length based on the diameter and length of the stenotic segment.

[0151]

[0153]

[0155] In the formula, Indicates the recommended bracket diameter; Indicates the magnification factor of the support diameter; Indicates the diameter of the normal airway adjacent to the narrowed segment; Indicates the recommended bracket length; Indicates the length of the narrowed segment of the airway; This indicates that the length of the support extends beyond the safe coverage length of the narrow segment at both ends. The system is based on... and Match the specific stent type, diameter, length, and alternative models in the tool parameter library. When the system determines that balloon dilation is appropriate, it can be adjusted according to the degree of stenosis. and airway diameter Match the balloon diameter and inflation pressure; when it is determined to be an airway foreign body, match the foreign body forceps or cryoprobe according to the size, location, and degree of adhesion of the foreign body; when it is determined to be bleeding, manage according to... and Iced saline solution, medication for hemostasis, or electrocoagulation are recommended.

[0156] The risk warning submodule in the treatment decision-making module and the risk warning submodule in the path planning module use the same set of risk calculation results, namely... , , This approach is then applied to treatment selection, tool parameter matching, and efficacy prediction. The advantage of this setup is that the same risk value affects not only the feasibility of the treatment path and the safety of the diagnostic procedure, but also whether the treatment tool parameters need to be reduced in power, shortened in time, or a more conservative alternative should be chosen, thus avoiding inconsistencies in judgment criteria between modules.

[0157] The efficacy and benefit prediction submodule predicts the success rate, clinical effectiveness, complication rate, and overall benefit rate of the technique based on pathway conditions, tool matching degree, lesion nature, patient's general condition, imaging findings, and laboratory test results. The system first calculates the tool matching degree:

[0158]

[0160] In the formula, This indicates the degree of matching by the tool; the closer the value is to 1, the better the match. Indicate the key dimensions of the selected tool, such as the diameter of the stent, balloon, ablation probe, or sampling tool; Indicates the recommended dimensions for the target airway, target lesion, or target stenosis; This represents a very small positive number. The system then calculates the path operability score based on path length, turning angle, and tool matching degree:

[0161]

[0163] In the formula, Indicates the path feasibility score; Represents the optimal path The total length; Represents the optimal path The average rotation angle; Indicates the degree of tool matching; , , These represent the weighting coefficients for path length, average turning angle, and tool matching degree, respectively. This represents a very small positive number. The shorter the path, the smaller the number of turns, and the better the tool matching, the better. The higher.

[0164] The formula for predicting the success rate of a technology is:

[0165]

[0167] In the formula, This represents the predicted success rate of the technology, expressed as a percentage. This represents the bias parameters of the technology success rate prediction model; to These represent the weights of the impact of pathway feasibility, tool compatibility, bleeding risk, pneumothorax risk, and airway injury risk on the success rate of the technique, respectively. This represents the path feasibility score obtained from the previous formula; Indicates the degree of tool matching; , , These represent the risks of bleeding, pneumothorax, and airway injury, respectively.

[0168] The formula for predicting clinical efficacy is:

[0169]

[0171] In the formula, This indicates the predicted clinical efficacy rate, expressed as a percentage. This represents the bias parameter of the clinical efficacy prediction model; to These represent the weighting coefficients for technical success rate, lesion nature, patient general condition, and potential for improvement in imaging, respectively. This represents the predicted success rate of the technology obtained from the previous formula; Indicates the nature of the lesion; This indicates the patient's general condition score; This represents a score indicating the potential for improvement in imaging. The formula builds clinical effectiveness prediction on the prediction of technical success rate, rather than calculating it in isolation.

[0172] The formula for predicting the incidence of complications is:

[0173]

[0175] In the formula, This indicates the predicted incidence of complications, expressed as a percentage. Indicates the risk of bleeding; Indicates the risk of pneumothorax; Indicates the risk of airway damage; This indicates the risk of systemic complications derived from laboratory test results, such as coagulation disorders, infection risk, or poor baseline condition. The formula states that if any one of these risk factors increases, the overall probability of complications also increases.

[0176] The formula for calculating the overall rate of return is:

[0177]

[0179] In the formula, This indicates the overall benefit rate score; This represents the predicted success rate of the technology. This represents the predicted clinical efficacy rate. This indicates the predicted incidence of complications; , , These represent the weighting coefficients of technical success rate, clinical effectiveness rate, and complication rate in the overall benefit rate, respectively. This overall benefit rate... As an important basis for the final output of the treatment decision module, when High and When the level is below the safety threshold, the system suggests that the current treatment plan can be given priority; when low or When the temperature is high, the system will prompt you to adjust the treatment method, reduce the intensity of the operation, or select an alternative tool.

[0180] The learning optimization module automatically records all data after each airway intervention procedure, including preoperative CT images. Intraoperative bronchoscopic images tracheal tree mask Vascular mask Lesion masking Optimal path Actual operation path and risk value , , The system-recommended treatment plan, the doctor's actual selected treatment plan, positive sampling results, successful technical results, clinically effective results, and complication results. Each operation record can be represented as:

[0181]

[0183] In the formula, Indicates the first Complete record data of this operation; Represents the diagnostic decision feature vector; Represents the feature vector of treatment decisions; This represents the optimal path in the system planning; This indicates the system's recommended operating procedure; This indicates the actual treatment plan chosen by the doctor; This indicates an actual positive diagnostic result; Indicates a successful actual technical result; Indicates actual clinical effectiveness; This indicates the actual outcome of complications. This data record... The data is stored in the case database and used to retrain the positive rate prediction model, risk prediction model, and efficacy prediction model. During model training, the system uses the difference between the predicted value and the actual result as the loss function.

[0184]

[0186] In the formula, Indicates the total predicted loss; This indicates the loss in predicting the positivity rate; Indicates the predicted loss due to risk; This indicates the loss in predicting the effectiveness of treatment.

[0187] For predicting the positivity rate, cross-entropy loss can be used:

[0188]

[0190] In the formula, This indicates the loss in predicting the positivity rate; This represents the actual diagnostic result, with 1 for a positive result and 0 for a negative result. This represents the percentage of positive rate predicted by the system. If the predicted positive rate differs significantly from the actual result using this loss function, the model parameters will be updated.

[0191] When the system suggests With the doctor's actual choice When there is inconsistency, and postoperative evaluation indicates that the physician's proposed treatment plan has a better clinical outcome, the decision rule correction unit uses a contrastive learning approach to correct the recommendation logic. The system uses the physician's proposed treatment plan as a positive sample and the original system recommendation as a negative sample. The contrastive learning loss is:

[0192]

[0194] In the formula, Indicates the contrast learning loss; It represents a comprehensive case representation composed of current case imaging features, path features, risk features, and patient status features; This indicates that the doctor's treatment plan resulted in better clinical outcomes; This indicates the system's original recommended operation plan, but the effect was poor; This represents the similarity calculation function; This represents a temperature coefficient used to adjust the discriminative strength of contrastive learning. Through this formula, the system gradually improves the matching accuracy of the doctor's optimal treatment plan in similar cases and reduces the priority of the original incorrect recommendation plan.

[0195] The overall training loss is:

[0196]

[0198] In the formula, This represents the total loss of the model; Indicates the predicted loss; Indicates the contrast learning loss; Indicates the weights of the contrastive learning loss; Indicates the regularization weight; Represents model parameters The L2 norm regularization term is used to prevent overfitting. The model parameter update formula is:

[0199]

[0201] In the formula, This represents the updated model parameters; This represents the model parameters before the update; Indicates the learning rate; This represents the total loss function with respect to the model parameters. The gradient is then used. The updated model parameters are then used again for image segmentation, positive rate prediction, risk prediction, and efficacy prediction, allowing the system to gradually optimize its decision rules after multiple operations.

[0202] For rare cases or special anatomical structures, the system identifies them based on differences between case characteristics and historical case distribution:

[0203]

[0205] In the formula, Indicates the degree of rarity; This represents the comprehensive case profile of the current case; This represents the mean of the overall characteristics of historical cases;

[0206] The matrix representing the inverse of the covariance matrix of the comprehensive historical case representation;

[0207] This represents the transpose of the difference between the current case characteristics and the historical mean. If Greater than the preset threshold ,in If the threshold for rare cases is indicated, the system will mark the case as a rare case or a special anatomical case and store its images, pathways, procedures, and outcome data in the rare sample database for reference in subsequent similar cases.

[0208] As can be seen from the above implementation method, the technical process of this system is not simply about setting up image recognition, path planning, risk warning, and operation suggestions separately. Instead, it is based on the fusion of chest CT and bronchoscopic images. First, it obtains the tracheal tree, vascular pathways, and lesion locations. Then, it calculates risk values ​​from this basic information and embeds these risk values ​​into the path travel cost to obtain the optimal path. Furthermore, it inputs the optimal path, lesion characteristics, and risk values ​​into the diagnostic decision module and the treatment decision module, ultimately obtaining the diagnostic positive rate, operation strategy, treatment tool parameters, technical success rate, clinical effectiveness, complication rate, and overall benefit rate. After each operation, the system feeds back the actual results to the learning and optimization module. Through predictive model training and comparative learning, it corrects its own rules, making the path planning, risk assessment, and operation suggestions for subsequent cases more accurate.

[0209] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A decision-making support system for endobronchial interventional procedures based on chest CT and bronchoscopic images, characterized in that, The system includes: The image preprocessing module is used to segment the tracheal tree and perform real-time fusion of multimodal images to reconstruct three-dimensional tracheal tree CT images, and extract the vascular pathways and lesion location characteristics. The path planning module is used to calculate the optimal path based on the information extracted by the image preprocessing module, and to realize navigation, dynamic replanning and risk warning during surgery. The diagnostic decision module is used to predict the positive rate based on lesion characteristics and output operational strategy suggestions. The treatment decision module is used to formulate operation methods and match operation tools according to the operation type and lesion characteristics, and to provide risk warnings and predict efficacy and benefits. The learning optimization module is used to record the entire process data after each operation, retrain the prediction model, and correct its own decision rules through comparative learning.

2. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images according to claim 1, characterized in that, The path planning module includes: The preoperative planning submodule is used to calculate the optimal path from the main bronchus to the target lesion based on the segmented tracheal tree, vascular path and lesion location characteristics. The requirements are to be closest to the lesion, with the fewest turns and the smallest angle, and to be far away from blood vessels, airway weak areas and pleura, as well as the predetermined contact angle between the operating tool and the lesion. The intraoperative navigation submodule is used to identify the characteristics of tracheal branches in the bronchoscopy image in real time to determine the current location, and to overlay navigation arrows on the screen to indicate the direction of travel. If it deviates from the planned route, it will issue a warning. The dynamic replanning submodule is used to automatically calculate alternative routes when airway obstruction or changes in the actual exposure of lesions occur during surgery, provide voice prompts, and display navigation arrows and distance prompts on the display screen.

3. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic imaging according to claim 1, characterized in that, The path planning module also includes a risk warning submodule, which includes: The bleeding risk assessment unit is used to calculate based on the vascular path extracted by the image preprocessing module: if the lesion is surrounded by blood vessels, it is marked as high risk (red); if the blood vessels are far from the lesion, it is marked as low risk (green); and a quantitative risk value or risk heatmap between 0 and 1 is given based on the surrounding vascular density, diameter, and blood supply richness. The pneumothorax risk assessment unit is used to measure the distance between the lesion and the lung surface. If the distance is less than a set value, it indicates a risk of pneumothorax. The airway injury risk assessment unit is used to assess airway wall thickness, cartilage integrity, and the presence of necrotic or vulnerable areas.

4. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic imaging according to claim 1, characterized in that, The diagnostic decision module includes: The operation strategy submodule is used to output parameter suggestions for the following operations based on the size, location, density, and blood supply characteristics of the lesion, through built-in decision rules: biopsy tool type, number of pieces, and angle; number of brushing operations and specific location; irrigation location, irrigation volume, and submitted items; and specific location, depth, and number of needle aspiration operations. The positive rate prediction submodule is used to output a 0-100% prediction value and factors affecting the positive rate based on the imaging parameters and operation parameters of the lesion using an AI model. If the predicted value is lower than the threshold of 50%, the operation method or the number of samplings will be changed. This threshold can be dynamically adjusted according to different lesion types, different lesion sizes and clinical purposes.

5. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images according to claim 1, characterized in that, The treatment decision module includes: The operation method submodule is used to decide on the following treatment methods based on the characteristics of the lesion: tumor ablation, balloon dilation or stent placement for airway stenosis, forceps or cryoablation of airway foreign bodies, hemostasis with ice-cold saline, drugs or electrocoagulation; The operation tool submodule is used to match the tool parameter library based on the operation type and lesion characteristics, and output the tool name, specific parameters and alternative solutions, including stent type, diameter, length, or ablation probe size, ablation time and power.

6. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images according to claim 1, characterized in that, The treatment decision module also includes a risk warning submodule, which has the same function as the risk warning submodule in the path planning module and is used to assess the risk of bleeding, pneumothorax, and airway injury.

7. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images according to claim 1, characterized in that, The treatment decision module also includes an efficacy and benefit prediction submodule, which includes: The technology success rate prediction unit is used to predict the success rate of the operation as planned based on the patient's lung pathway conditions and the degree of tool compatibility. Clinical efficacy prediction unit is used to predict the degree of improvement in symptoms or lung function based on the nature and severity of the patient's lesions, general condition, and imaging findings. Complication rate prediction unit, used to predict the probability of adverse events based on lesion location, operation type, general condition and laboratory test results; The overall profitability calculation unit is used to calculate the overall profitability based on the above three data points and an AI algorithm.

8. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic imaging according to claim 1, characterized in that, The learning optimization module includes: The prediction model training unit is used to repeatedly retrain the positive rate prediction model, risk prediction model, and efficacy prediction model. The decision rule correction unit is used when the clinical outcome of the doctor's actual operation plan is better than the system's recommended plan. The system corrects its own decision rules through comparative learning, including updating the matching logic of the tool recommendation or the priority of operation method selection. The rare case collection unit is used to store rare cases or special autopsies as rare samples in the database for subsequent operation reference.

9. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images according to claim 2, characterized in that, The intraoperative navigation submodule uses a reminder method analogous to the guidance method of navigation software; the distance prompts in the dynamic replanning submodule include prompts for the number of remaining tracheal branches.

10. The airway interventional procedure auxiliary decision-making system based on chest CT and bronchoscopic images according to claim 1, characterized in that, The image preprocessing module extracts the vascular pathways and lesion location characteristics, which are then used by the path planning module to calculate the optimal path, the diagnostic decision module to match operation strategies, and the treatment decision module to formulate treatment plans and assess risks.

Images