Liver surgery planning system based on data fusion

Abstract

The application relates to the technical field of liver surgery, and discloses a liver surgery planning system based on data fusion. The system comprises an image data fusion module, a three-dimensional model construction module, a surgery path planning module, a risk assessment module and a scheme optimization module. The image data fusion module integrates CT and MRI images to generate a standardized image fusion model; the three-dimensional model construction module generates a liver three-dimensional visual model based on the image fusion model; the surgery path planning module plans an optimal resection path to form a preliminary surgery path scheme; the risk assessment module assesses the surgery risk and generates a report; and the scheme optimization module adjusts parameters based on the report to generate a final surgery planning scheme. The system accurately presents the liver structure and the tumor position, scientifically plans the surgery path, assesses and avoids the surgery risk, optimizes the surgery scheme, and helps precise implementation of liver surgery through multi-source data fusion and three-dimensional visualization technology.

Description

Technical Field

[0001] This invention relates to the field of liver surgery technology, specifically a liver surgery planning system based on data fusion. Background Technology

[0002] Liver surgery, due to its complex anatomy and dense vascular network, has always been one of the most challenging areas in surgery. Traditional liver surgery planning relies primarily on the surgeon's interpretation of two-dimensional images and personal experience, a method with many limitations. While two-dimensional CT and MRI images can provide structural details and soft tissue information about the liver, they are insufficient to fully reflect the spatial relationships within the liver when used alone. Surgeons need to integrate multiple images in their minds, which can easily lead to misjudgments of tumor location and vascular distribution due to subjective biases.

[0003] In tumor localization, two-dimensional imaging cannot accurately label the three-dimensional spatial location parameters of the tumor and surrounding blood vessels, making it difficult for surgeons to accurately assess the extent of tumor invasion. For large or deep-seated tumors, relying solely on two-dimensional imaging may miss subtle connections between the tumor and important blood vessels, thus affecting the rationality of the surgical plan. Furthermore, the intrahepatic vascular network has a complex branching distribution, and traditional methods struggle to clearly visualize its topology. This can lead to the oversight of small but crucial vascular branches when planning the surgical path, increasing the risk of intraoperative bleeding.

[0004] During surgical path planning, surgeons often choose surgical approaches based on experience, lacking quantitative analysis of path accessibility. This reliance on experience may result in planned surgical paths that fail to avoid important blood vessels, or paths that are too long or have unreasonable angles, increasing the difficulty of the surgery. Furthermore, traditional methods for assessing liver resection volume and residual liver function are mostly based on formula estimations, making it difficult to accurately calculate considering individual differences in liver function among patients. This can lead to situations where the resection volume is too large, resulting in insufficient residual liver function, or incomplete resection leading to tumor recurrence.

[0005] In the risk assessment phase, traditional methods rely primarily on the doctor's clinical experience to judge potential risks, lacking a systematic identification of bleeding risk areas and a quantitative analysis of the probability of complications. For potential bleeding points, bile leakage, and other complications, it is difficult to develop targeted avoidance strategies in advance, potentially leading to unexpected situations during surgery, prolonged operation time, and increased difficulty in postoperative recovery for the patient.

[0006] With the development of medical imaging technology, 3D visualization technology has been gradually applied to liver surgery planning. However, existing systems still have shortcomings in terms of the accuracy of multi-source image fusion, the completeness of vascular network reconstruction, and the dynamic adjustment of surgical path optimization. Some systems can only achieve simple image overlay and cannot construct accurate feature mapping relationships; during vascular network reconstruction, small vascular branches are easily missed, affecting the judgment of the relative position of tumors and blood vessels; after surgical path planning, it is difficult to optimize in real time based on risk assessment results, resulting in poor adaptability of the final surgical plan. These problems make it difficult for existing technologies to meet the needs of precise planning in complex liver surgeries, and a system that can integrate multi-source data, achieve 3D visualization planning, dynamically assess risks, and optimize the plan is needed. Summary of the Invention

[0007] The purpose of this invention is to provide a liver surgery planning system based on data fusion to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides a liver surgery planning system based on data fusion, the system comprising:

[0009] The image data fusion module acquires CT and MRI images, integrates the spatial coordinates of CT and MRI images, constructs image feature mapping relationships, matches anatomical structure identifiers of CT and MRI images, and generates a standardized image fusion model.

[0010] The 3D model construction module extracts the contour features of the liver organ based on the standardized image fusion model, reconstructs the topology of the intrahepatic vascular network, labels the spatial location parameters of the tumor region, and generates a 3D visualization model of the liver.

[0011] Based on the three-dimensional visualization model of the liver, the surgical path planning module analyzes the relative distance between the tumor and blood vessels, evaluates the accessibility index of the surgical approach, plans the optimal resection path that avoids important blood vessels, and generates a preliminary surgical path plan.

[0012] Based on the preliminary surgical approach, the risk assessment module calculates the ratio of the expected resection volume to the residual liver function, identifies potential bleeding risk areas, assesses the probability of surgical complications, and generates a surgical risk assessment report.

[0013] Based on the surgical risk assessment report, the scheme optimization module adjusts the resection boundary parameters, optimizes the operating angle of surgical instruments, updates the risk area avoidance strategy, and generates the final surgical plan.

[0014] Preferably, the standardized image fusion model includes modal registration parameters, an anatomical structure mapping table, and image resolution adaptation values; the liver three-dimensional visualization model includes an organ contour coordinate set, a vascular branch topology map, and tumor region annotation data; the preliminary surgical approach plan includes approach angle parameters, resection path coordinate sequence, and important structure avoidance markers; the surgical risk assessment report includes liver volume ratio, bleeding risk level, and complication probability prediction value; and the final surgical planning scheme includes optimized resection boundary data, instrument operation angle range, and risk area avoidance update strategy.

[0015] Preferably, the image data fusion module includes:

[0016] The multimodal image acquisition submodule acquires CT and MRI image data, obtains image slice thickness and pixel spacing parameters, extracts DICOM metadata information of the images, identifies the scanning coordinate system of different modal images, and generates the original image dataset.

[0017] The spatial coordinate registration submodule, based on the original image dataset, calls the image rigid registration algorithm to align the spatial coordinate origins of different modal images, calculates the coordinate system transformation matrix, corrects the spatial position deviations between images, and generates coordinate registration results.

[0018] Based on the coordinate registration results, the feature fusion processing submodule extracts liver parenchyma and blood vessel feature points from the image, establishes the mapping relationship between feature points, fuses grayscale information from different modal images, and generates a standardized image fusion model.

[0019] Preferably, the 3D model construction module includes:

[0020] The organ contour extraction submodule, based on the standardized image fusion model, uses a region growing algorithm to segment the liver parenchyma region, extracts the three-dimensional coordinate points of the liver surface contour, fits the contour curve to generate a closed surface, and obtains the liver organ contour model.

[0021] The vascular network reconstruction submodule, based on the liver organ contour model, identifies the cross-sectional area of ​​blood vessels in the image, tracks the direction of blood vessels and connects branch structures, constructs a topological relationship diagram of the vascular network, labels the parameters of blood vessel diameter and branch angle, and generates a three-dimensional model of intrahepatic blood vessels.

[0022] The model accuracy verification submodule calls the intrahepatic vascular 3D model, compares it with the anatomical structure position in the actual image, calculates the spatial error value between the model and the image, adjusts the model's mesh density parameters, optimizes the model's surface smoothness, and generates a 3D visualization model of the liver.

[0023] Preferably, the surgical path planning module includes:

[0024] The anatomical structure analysis submodule, based on the three-dimensional visualization model of the liver, locates the coordinate position of the tumor center, measures the shortest distance between the tumor and the portal vein and hepatic vein, counts the distribution density of blood vessels around the tumor, and generates anatomical structure distance parameters.

[0025] The accessibility assessment submodule simulates the instrument operation space for different surgical access directions based on the anatomical structure distance parameters, calculates the probability of collision between surgical instruments and important blood vessels, evaluates the feasibility score of the access path, and screens highly feasible access directions.

[0026] The resection path generation submodule plans a resection path coordinate sequence from the liver surface to the tumor region based on the highly feasible approach direction, sets key turning points on the path to ensure that the path avoids blood vessels with a diameter greater than three millimeters, and generates a preliminary surgical path plan.

[0027] Preferably, the risk assessment module includes:

[0028] The liver volume calculation submodule, based on the preliminary surgical path plan, divides the expected liver resection area and the residual liver area, calculates the three-dimensional volume values ​​of the two areas, obtains the ratio of the residual liver volume to the standard liver volume, and generates the liver volume ratio parameter.

[0029] The bleeding risk analysis submodule calls the liver volume ratio parameter to identify the number and diameter of blood vessels passing through the resection path, assesses the probability of blood vessel damage, and calculates the potential bleeding volume by combining the patient's coagulation function indicators, generating a bleeding risk level.

[0030] The complication prediction submodule analyzes whether the residual liver volume meets functional requirements based on the bleeding risk level, assesses the probability of liver failure, and comprehensively predicts the probability of surgical complications by combining the patient's age and underlying disease information, generating a surgical risk assessment report.

[0031] Preferably, the scheme optimization module includes:

[0032] The resection boundary adjustment submodule, based on the surgical risk assessment report, expands the preservation area if the residual liver volume ratio is lower than the safety threshold, and shrinks the resection boundary if the bleeding risk level is too high, redetermines the three-dimensional coordinate range of the resection area, and generates optimized resection boundary data.

[0033] The operation angle optimization submodule calls the optimized resection boundary data, simulates the operation field of surgical instruments at different angles, calculates the range of motion of the instrument end, selects an angle with a clear field of view and flexible operation, and generates the instrument operation angle range.

[0034] The avoidance strategy update submodule, based on the instrument operation angle range, increases the avoidance buffer distance for high-risk blood vessel areas in the path, adjusts the position parameters of the path turning points, ensures that all critical structures are outside the safe distance, and generates the final surgical plan.

[0035] Preferably, the system further includes:

[0036] Based on the final surgical plan, the surgical simulation module constructs a virtual surgical environment, simulates the cutting mechanical properties of liver tissue, calculates the tissue deformation at different resection speeds, verifies the feasibility of the plan, and generates surgical simulation results.

[0037] The surgical simulation results specifically include tissue deformation parameters, simulated blood loss data, and estimated operation time.

[0038] Preferably, the surgical simulation module includes:

[0039] The virtual environment construction submodule, based on the final surgical plan, imports a three-dimensional visualization model of the liver and a surgical instrument model, sets the elastic modulus and friction coefficient parameters of the tissue, and constructs a virtual surgical scene that conforms to real physical properties.

[0040] The cutting process simulation submodule calls the virtual surgical scene, performs virtual cutting operations according to the planned path, calculates the tissue stress distribution on the cutting surface, simulates the tearing and tensile deformation of liver tissue, and records tissue deformation parameters;

[0041] The simulation result evaluation submodule, based on the tissue deformation parameters, statistically analyzes the simulated bleeding during the virtual cutting process, analyzes the distribution of surgical operation time, compares it with the average operation time of the actual surgery, and generates surgical simulation results.

[0042] Preferably, the resection path generation submodule further includes a dynamic adjustment unit, which receives intraoperative image feedback data in real time when planning the path, compares the deviation between the preset path and the actual anatomical structure, automatically corrects the coordinates of the path turning points, and generates a resection path plan that is updated in real time.

[0043] Compared with the prior art, the beneficial effects of the present invention are:

[0044] The image data fusion module integrates CT and MRI images in terms of spatial coordinates and anatomical structure matching. The resulting standardized image fusion model can simultaneously reflect the advantages of both types of images, freeing doctors from relying on subjective integration of two-dimensional images. This reduces information bias caused by interpreting different images separately, making the presentation of liver internal structure, tumor location, and blood vessel distribution more comprehensive and accurate.

[0045] Based on a standardized image fusion model, the 3D model construction module extracts the contour features of the liver and reconstructs the topology of the intrahepatic vascular network. The generated 3D visualization model transforms spatial relationships that are difficult to intuitively represent in 2D images into stereoscopic images. The annotation of spatial location parameters of the tumor region allows doctors to clearly observe the three-dimensional distribution of the tumor and surrounding blood vessels, avoiding misjudgments of the tumor invasion range under traditional 2D imaging, and providing a more accurate spatial reference for subsequent surgical path planning.

[0046] The surgical path planning module analyzes the relative distance between the tumor and blood vessels and the accessibility of the surgical approach to plan the optimal resection path that avoids important blood vessels. This quantitative analysis based on a three-dimensional model changes the traditional experience-based approach to selecting surgical approaches, reduces the risk of vascular damage caused by improper path selection, and makes the initial surgical path plan more scientific and reasonable.

[0047] The risk assessment module calculates the ratio of the expected resection volume to residual liver function for the initial surgical approach plan, identifies potential bleeding risk areas, and assesses the probability of complications. This process no longer relies on the doctor's experience-based judgment, but rather uses quantitative data to systematically assess surgical risks, allowing doctors to know in advance what risks may occur during surgery and providing specific risk basis for adjusting the surgical plan.

[0048] The surgical plan optimization module adjusts the resection boundary parameters, optimizes the surgical instrument operation angles, and updates the risk area avoidance strategy based on the risk assessment report. The resulting final surgical plan can dynamically adapt to the risk assessment results. This dynamic optimization mechanism solves the problem of traditional surgical plans being difficult to adjust flexibly once determined. It ensures effective tumor resection while minimizing damage to normal liver tissue and blood vessels, reducing the possibility of complications such as intraoperative bleeding and insufficient residual liver function. This improves the safety and precision of liver surgery, helps shorten postoperative recovery time, and improves surgical prognosis. Attached Figure Description

[0049] Figure 1 This is a schematic diagram illustrating the working principle of the liver surgery planning system based on data fusion as described in this invention.

[0050] Figure 2 A flowchart for the image data fusion module;

[0051] Figure 3 Flowchart for building modules for 3D models;

[0052] Figure 4 A flowchart for the risk assessment module;

[0053] Figure 5 This is a flowchart for the solution optimization module. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Please see Figure 1 This invention provides a liver surgery planning system based on data fusion, the system comprising:

[0056] The image data fusion module acquires CT and MRI images, integrates their spatial coordinates, constructs image feature mapping relationships, matches anatomical structure markers, and generates a standardized image fusion model. The 3D model construction module, based on this standardized image fusion model, extracts liver organ contour features, reconstructs the intrahepatic vascular network topology, and labels the spatial location parameters of the tumor region, thereby generating a 3D visualization model of the liver. The surgical path planning module, based on the 3D visualization model of the liver, analyzes the relative distance between the tumor and blood vessels, assesses the accessibility indicators of the surgical approach, plans the optimal resection path avoiding important blood vessels, and forms a preliminary surgical path plan. The risk assessment module, based on the preliminary surgical path plan, calculates the expected resection volume to the ratio of residual liver function, identifies potential bleeding risk areas, assesses the probability of surgical complications, and generates a surgical risk assessment report. The plan optimization module, referring to the surgical risk assessment report, adjusts the resection boundary parameters, optimizes the surgical instrument operation angles, updates the risk area avoidance strategy, and finally generates the final surgical plan.

[0057] Example 1: See Figure 2 The image data fusion module completes the entire process from raw image acquisition to standardized image fusion model generation through the coordinated operation of the multimodal image acquisition submodule, spatial coordinate registration submodule, and feature fusion processing submodule.

[0058] The multimodal image acquisition submodule first initiates the image acquisition process, simultaneously acquiring CT and MRI image data. During acquisition, this submodule automatically records the slice thickness parameters of the images. For example, CT images may be scanned using a slice thickness of 0.625 mm, while MRI images may use a slice thickness of 1 mm. Different slice thickness parameters directly affect the vertical resolution of the images. Simultaneously, the submodule extracts pixel pitch information, clarifying the physical size of each pixel in the actual anatomical space. For instance, the pixel pitch of a CT image may be 0.488 mm × 0.488 mm, while the pixel pitch of an MRI image may be 0.5 mm × 0.5 mm. These parameters provide the basic data for subsequent spatial coordinate integration. Furthermore, the submodule deeply analyzes the DICOM metadata of the images, extracting detailed information such as patient information, scanning equipment model, scanning sequence parameters, and image generation time. This metadata not only includes the technical parameters of the images but also involves the patient's basic information, serving as an important reference for image data management and subsequent processing. While acquiring the above data, the submodule identifies the scanning coordinate system of different modalities. Due to differences in scanning equipment and methods, CT and MRI images may have different origin settings and coordinate axis directions. For example, CT images may use the patient's midaxillary line as a reference for a certain coordinate axis, while MRI images may use the patient's glabella as the origin reference. The submodule records the characteristic information of these coordinate systems and finally integrates CT image data, MRI image data, slice thickness, pixel spacing, DICOM metadata, and scanning coordinate system information to form the original image dataset. This dataset completely preserves the original information of the two modalities, providing a data foundation for subsequent coordinate registration and feature fusion.

[0059] After receiving the original image dataset, the spatial coordinate registration submodule begins the spatial coordinate alignment process. This submodule invokes a preset rigid image registration algorithm. This algorithm assumes that there is no nonlinear deformation of the anatomical structures between different modalities, and spatial alignment can be achieved solely through rigid transformations such as translation, rotation, and scaling. First, the algorithm selects a set of common anatomical landmarks in both CT and MRI images, such as the top of the diaphragm and the hilum of the liver, which are clearly visible in both images. These landmarks serve as references for coordinate alignment. Next, using the coordinate system of one image as a reference (usually the CT image's coordinate system), the algorithm calculates the required translation, rotation angle, and scaling for the other image (MRI image), thereby determining the coordinate transformation matrix. This transformation matrix contains all the parameters for converting the MRI image coordinates to the CT image coordinates. By applying this transformation matrix, every pixel in the MRI image can be mapped to the CT image coordinate system, achieving alignment of the spatial coordinate origins of the two images. During the conversion process, the submodule calculates the spatial position deviation of the corresponding anatomical structures in the two images after conversion. For example, the cross-section of the right hepatic vein, which is located at a certain coordinate in the MRI image, may have a deviation of 0.3mm from the actual position of the vein in the CT image after conversion to the CT coordinate system. The submodule will fine-tune the conversion matrix according to this deviation to correct the spatial position deviation between the images and finally generate the coordinate registration result. This result ensures the consistency of the spatial position between the CT image and the MRI image.

[0060] The feature fusion processing submodule further fuses the features of the two images based on the coordinate registration results. First, the submodule extracts feature points of the liver parenchyma and blood vessels from the registered CT and MRI images. For the liver parenchyma, it selects landmark points such as inflection points of the edges and hepatic fissures; for blood vessels, it selects points with obvious features such as bifurcation points, start points, and termination points. Then, by comparing the position and attributes of these feature points in the two images, a mapping relationship is established between the feature points. For example, a bifurcation point of the portal vein in the CT image and its corresponding bifurcation point in the MRI image are marked as a pair of mapping points. This mapping relationship clarifies the correspondence of features in the two images. Based on the established mapping relationship, the submodule fuses the grayscale information of the two images. CT images have advantages in displaying liver parenchyma and high-density structures (such as calcifications), and their grayscale values ​​can reflect tissue density differences; MRI images are more effective in distinguishing soft tissues and displaying tumor boundaries, and the grayscale values ​​of different MRI sequences can reflect characteristics such as tissue relaxation time. During the fusion process, the submodule weights the grayscale values ​​at corresponding locations based on the mapping relationship of feature points. For example, in the liver parenchyma, CT image grayscale values ​​may be assigned a higher weight, while in suspected tumor areas, MRI image grayscale values ​​may be assigned a higher weight. This integrates the advantages of both images in this way. Finally, a standardized image fusion model is generated, which includes modal registration parameters, an anatomical structure mapping table, and image resolution adaptation values. The modal registration parameters record in detail the coordinates of anatomical landmarks used in rigid registration, the coordinate system transformation matrix, and the final spatial deviation value. The anatomical structure mapping table lists the correspondence between all major anatomical structures in CT and MRI images, such as the correspondence between the left lobe of the liver in CT and MRI images, and the correspondence of the proper hepatic artery in the two images. The image resolution adaptation values ​​are achieved at the data level by interpolation or downsampling, for example, adjusting the resolution of the MRI image to match that of the CT image, ensuring that each pixel of the fused image contains information from both modalities, thus generating a standardized image fusion model that combines the advantages of both CT and MRI.

[0061] Example 2: See Figure 3After the 3D model construction module is started, the organ contour extraction submodule first reads the data from the standardized image fusion model. This submodule uses a region growing algorithm, starting from pre-selected seed points inside the liver in the image, and gradually incorporating surrounding pixels into the growing region according to a set grayscale similarity threshold, thereby segmenting the liver parenchyma and surrounding tissues. During the segmentation process, the algorithm automatically ignores non-parenchymal structures such as blood vessels and tumors in the image, ensuring that the segmented region contains only the liver parenchyma. Subsequently, the submodule extracts 3D coordinate points from the edges of the segmented liver parenchyma region. These coordinate points are evenly distributed across various parts of the liver surface, covering key areas such as the diaphragmatic surface, visceral surface, and hepatic hilum. By performing curve fitting on these coordinate points, a series of continuous contour curves are generated, and these curves are then connected to form a closed surface, ultimately obtaining the liver organ contour model. This model can intuitively display the overall shape and external boundaries of the liver.

[0062] The vascular network reconstruction submodule uses the liver organ contour model as a reference framework to begin reconstructing intrahepatic vessels. This submodule first identifies vascular cross-sectional regions in a standardized image fusion model. By analyzing the grayscale value changes of pixels in the images, it distinguishes vessels from the surrounding liver parenchyma. For example, vessels typically appear as low-density areas in CT images, while in specific sequences of MRI images, they exhibit flow void signals. After identifying the vascular cross-sections, the submodule traces along the vessel's direction, starting from the vessel's origin and sequentially connecting adjacent cross-sectional regions. Simultaneously, it records the branching of the vessels. When encountering a bifurcation, it traces the direction of each branch and establishes the connections between them, thus constructing a topological map of the vascular network. During reconstruction, the submodule also measures and labels the diameter and branch angle parameters of each vessel. The vessel diameter is obtained by measuring the maximum distance of the cross-sectional region, and the branch angle is determined by calculating the angle between the branch vessel and the main vessel. These parameters are integrated into the vascular network model, ultimately generating a three-dimensional model of the intrahepatic vessels.

[0063] The model accuracy verification submodule calls the 3D model of intrahepatic vessels and compares the location and direction of the vessels in the model with the actual image data in the standardized image fusion model. By calculating the spatial distance between the vessel feature points in the model and the corresponding vessel feature points in the image, the spatial error value between the model and the image is obtained. Based on the magnitude of the error value, the submodule adjusts the model's mesh density parameter. If the error is large, the number of meshes is increased to improve the model's precision; if the error is within acceptable limits, the number of meshes is appropriately reduced to simplify the model. Simultaneously, the model surface is smoothed to remove burrs and sharp edges, making the model's shape closer to the actual anatomical structure of the liver. After these processes, a 3D visualization model of the liver is generated. This model includes an organ contour coordinate set, a vascular branch topology map, and tumor region annotation data. The organ contour coordinate set records the 3D coordinates of all contour points on the liver surface; the vascular branch topology map clearly shows the connection methods and distribution of vessels at various levels within the liver; and the tumor region annotation data clarifies the specific location and size of the tumor within the liver by marking the tumor's boundary coordinates in the model.

[0064] After the surgical path planning module is activated, the anatomical structure analysis submodule operates based on a three-dimensional visualization model of the liver. This submodule determines the three-dimensional coordinates of the tumor center by identifying the labeled data of the tumor region in the model. Subsequently, it measures the shortest straight-line distance between the tumor center and important blood vessels such as the portal vein and hepatic vein, and simultaneously counts the distribution and number of blood vessels within a certain range around the tumor (e.g., a 2 cm radius area centered on the tumor). This generates anatomical distance parameters, which reflect the spatial relationship between the tumor and surrounding important blood vessels.

[0065] After receiving anatomical distance parameters, the accessibility assessment submodule begins simulating the instrument operation space for different surgical approach directions. This submodule presets multiple possible surgical approach directions, such as those from the anterior or lateral abdomen. For each approach direction, it simulates the trajectory of surgical instruments as they enter the liver from outside the body and reach the tumor region, analyzing the spatial size within the trajectory range to determine if the instruments have sufficient maneuverability. Simultaneously, it calculates the probability of the surgical instruments colliding with important blood vessels during movement. By evaluating the feasibility score of the approach path, it ranks the various approach directions and selects the most feasible ones as the basis for subsequent planning.

[0066] The resection path generation submodule plans a resection path from the liver surface to the tumor region based on a highly feasible approach direction. This submodule starts at a suitable point on the liver surface and ends at the boundary of the tumor region, generating a series of continuous coordinate points that form the resection path coordinate sequence. During the planning process, the submodule sets key turning points along the path to allow for flexible avoidance of obstructive structures. Simultaneously, referencing vascular data from a 3D visualization model of the liver, it ensures that the planned path does not pass through blood vessels larger than three millimeters in diameter, ultimately generating a preliminary surgical path plan. This plan includes approach angle parameters, specifying the angle at which surgical instruments enter the body; a resection path coordinate sequence, precisely describing the surgical resection trajectory; and important structure avoidance markers, indicating important structures such as blood vessels that need to be avoided along the path.

[0067] The dynamic adjustment unit within the resection path generation submodule operates continuously throughout the path planning process, receiving real-time intraoperative image feedback data. This feedback data, sourced from real-time image acquisition equipment during the surgery, reflects the actual location and morphology of anatomical structures within the body. The dynamic adjustment unit compares the preset resection path with the actual anatomical structures in the real-time images, calculating the deviation between the two, such as the distance deviation between a point on the path and the actual blood vessel. Based on the deviation value, it automatically corrects the coordinates of the path's turning points, ensuring the resection path adapts to the actual anatomical structure and generating a real-time updated resection path plan.

[0068] Example 3: See Figure 4 After the risk assessment module is activated, the liver volume calculation submodule first obtains the resection boundary parameters from the preliminary surgical path plan. Based on these parameters, it performs region segmentation in the 3D visualization model of the liver. During segmentation, the submodule divides the 3D liver model into two parts: the expected resection area and the residual liver area, according to the resection range coordinates set in the path plan. The boundary of the expected resection area is determined by the resection path coordinate sequence in the preliminary surgical path plan, covering all tissue to be resected from the liver surface to the tumor area; the residual liver area is the remaining liver tissue after resection, containing key structures that maintain the patient's postoperative liver function.

[0069] After completing the region segmentation, the liver volume calculation submodule uses a three-dimensional volume calculation method to measure the volume of the two regions. This method decomposes the three-dimensional model into several tiny cubic units, counts the number of cubic units contained in each region, and then calculates the three-dimensional volume values ​​of the two regions by combining the volume of an individual cubic unit. The volume of an individual cubic unit is determined by the spatial resolution of the liver three-dimensional visualization model. For example, when the spatial resolution of the model is 0.5mm × 0.5mm × 0.5mm, the volume of an individual cubic unit is 0.125mm³.

[0070] Subsequently, the submodule obtains the patient's standard liver volume, which is typically estimated using a specific formula based on the patient's height, weight, and other physical indicators. By calculating the ratio of the residual liver volume to the standard liver volume, a liver volume ratio parameter is generated. This parameter directly reflects the proportional relationship between the postoperative residual liver and the patient's individual standard liver volume.

[0071] After receiving the liver volume ratio parameter, the bleeding risk analysis submodule begins to assess the bleeding risk during the operation. First, the submodule accesses the vascular data from the liver's 3D visualization model to identify all vessels traversed by the resection path in the preliminary surgical plan, recording the number of these vessels and the diameter of each. The vessel diameter is measured based on the 3D coordinate data of the vessel cross-section in the model, obtained by calculating the maximum distance between two points on the cross-sectional boundary.

[0072] Based on the number and diameter of the blood vessels traversed, the submodule assesses the probability of vascular injury. Generally, the more blood vessels traversed and the larger their diameter, the higher the probability of vascular injury. Simultaneously, the submodule imports the patient's coagulation function indicators, including clinical laboratory data such as prothrombin time, activated partial thromboplastin time, and platelet count. These indicators reflect the patient's own coagulation ability. Combining the probability of vascular injury and coagulation function indicators, the submodule calculates the potential bleeding volume during the procedure, using the following formula:

[0073]

[0074] in, Indicates potential bleeding volume. This is a coefficient related to the probability of vascular injury. For the first The diameter of the strip that passes through the blood vessel. For the resection path to pass through the first The length of a blood vessel The coefficient is related to the patient's coagulation function indicators.

[0075] Based on the potential amount of bleeding, the submodule divides the bleeding risk into different levels and generates bleeding risk level parameters. For example, it can be divided into three levels: low risk, medium risk, and high risk. Each level corresponds to a certain range of potential bleeding.

[0076] The complication prediction submodule further predicts surgical complications based on bleeding risk levels. First, the submodule analyzes whether the residual liver volume is sufficient to meet the patient's postoperative liver function needs, and uses the liver volume ratio parameter to assess the residual liver's reserve capacity in metabolic and detoxification functions. If the residual liver volume is insufficient, the probability of liver failure is assessed; the smaller the residual liver volume, the higher the probability of liver failure.

[0077] Simultaneously, the submodule collects the patient's age information and underlying medical conditions, such as whether they suffer from chronic diseases like hypertension, diabetes, and heart disease. Older patients or those with multiple underlying conditions have a relatively weaker tolerance to surgical trauma, and the probability of postoperative complications will increase accordingly.

[0078] Taking into account factors such as the probability of liver failure, patient age, underlying medical conditions, and bleeding risk level, this submodule employs multivariate analysis to predict the overall probability of various complications that may occur during surgery (such as infection, bile leakage, and liver failure). Finally, it integrates the predicted liver volume ratio, bleeding risk level, and complication probability to generate a surgical risk assessment report. This report comprehensively presents the potential risks during surgery, providing detailed risk information for subsequent surgical plan optimization.

[0079] Example 4: See Figure 5 After the scheme optimization module is activated, the resection boundary adjustment submodule first receives the surgical risk assessment report and extracts the liver volume ratio parameter and bleeding risk level information from it. When the report shows that the residual liver volume ratio is lower than the preset safety threshold, the submodule will redefine the boundary of the resection area, reducing the three-dimensional spatial proportion of the resection area based on the resection range determined by the original preliminary surgical path plan. For example, if the original plan was to resect a wedge-shaped area of ​​the right lobe of the liver, the adjustment will move the apex of the wedge towards the tumor, reducing the amount of normal liver tissue resected, thereby expanding the area to be preserved. Conversely, if the bleeding risk level is too high, indicating that the resection path may pass through many or large blood vessels, the submodule will shrink the resection boundary, shifting the edge of the resection area a certain distance towards the tumor to avoid densely vascularized areas that may cause massive bleeding. During the adjustment process, the submodule recalculates the three-dimensional coordinate range of the resection area and records the adjusted boundary parameters as optimized resection boundary data. This data is accurate to the millimeter level and includes the maximum and minimum coordinate values ​​of the resection area in three-dimensional space, as well as the coordinate information of key control points on the boundary.

[0080] After acquiring the optimized resection boundary data, the operation angle optimization submodule begins to optimize the operation angle of the surgical instruments. This submodule calls a 3D visualization model of the liver to simulate the operation of different types of surgical instruments, such as the ultrasonic scalpel and laparoscope, during the resection process. The submodule presets multiple possible operation angles, covering different incident directions from the liver surface to the resection area, such as a 45-degree angle above the diaphragmatic surface of the liver and a 30-degree angle below the visceral surface of the liver. For each preset angle, the submodule analyzes the field of view of the instrument tip to determine whether the boundary of the resection area and the distribution of surrounding blood vessels can be clearly observed. At the same time, it calculates the range of motion of the instrument tip at that angle, including the range of motion in the vertical, horizontal, and forward / backward directions, to ensure that the instrument is not obstructed by ribs, other organs, or other surrounding structures during operation. By comparing the clarity of the field of view and the flexibility of movement at different angles, the submodule selects the angle range that allows the surgeon to clearly observe the surgical area while allowing the instrument to move freely. The maximum and minimum values ​​of these angles are recorded as the instrument operation angle range. For example, the effective operation angle of the ultrasonic scalpel is determined to be between 30 and 60 degrees.

[0081] The avoidance strategy update submodule further optimizes the avoidance strategy for risk areas based on the instrument operation angle range. This submodule first marks high-risk vascular areas in the 3D visualization model of the liver. These areas are typically densely distributed areas of vessels with a diameter exceeding three millimeters, or the routes of important vessels such as the main portal vein and main branches of the hepatic vein. Based on the instrument operation angle, the submodule calculates the spatial range that the surgical instrument may touch during movement and increases the avoidance buffer distance around high-risk vascular areas within this range. For example, for a five-millimeter-diameter branch of the portal vein, the initial avoidance distance was set at two millimeters; after the update, the buffer distance will be increased to four millimeters based on the slight deviations that may occur due to the instrument operation angle. Simultaneously, the submodule readjusts the positional parameters of key turning points on the resection path. If a turning point was originally close to an important vessel, it will be moved a certain distance away from the vessel to ensure that the path segments around the turning point are all outside the safe distance. During the adjustment process, the submodule repeatedly verifies the distance between the path and all critical structures, including important ligaments on the liver surface and adjacent organs such as the gallbladder, until it is confirmed that all critical structures are outside the safe distance. The final surgical plan includes optimized resection boundary data, instrument manipulation angle ranges, and risk area avoidance update strategies. The optimized resection boundary data clarifies the final determined resection area; the instrument manipulation angle ranges provide the surgeon with reference angle intervals for instrument manipulation; and the risk area avoidance update strategies detail the specific avoidance methods for each high-risk area, such as the required inward bend angle for a particular segment of hepatic vein, or the appropriate turning angle when approaching a vessel bifurcation. This information collectively constitutes a complete and optimized surgical plan, providing concrete execution guidelines for the actual surgical procedure.

[0082] Example 5: After the surgical simulation module in the system is started, various simulation operations are carried out based on the final surgical plan. The virtual environment construction submodule first reads relevant data from the final surgical plan, including optimized resection boundary data, instrument operation angle range, and risk area avoidance update strategies, and simultaneously imports the liver 3D visualization model and surgical instrument model. The liver 3D visualization model contains detailed information such as the overall morphology of the liver, internal blood vessel distribution, and tumor location, while the surgical instrument model covers the 3D structure of commonly used surgical instruments, such as scalpels, hemostatic forceps, and ultrasonic scalpels. The model of each instrument accurately reproduces the size, shape, and operating characteristics of the actual instrument.

[0083] After importing the model, the virtual environment construction submodule begins setting the physical parameters of the tissue. The elastic modulus parameter simulates the deformation of liver tissue under external force; different regions of the liver tissue have different elastic modulus values. For example, tissue near the liver surface has a lower elastic modulus, while tissue in the hepatic hilum, containing more connective tissue, has a relatively higher elastic modulus. The friction coefficient parameter simulates the frictional characteristics between surgical instruments and liver tissue. Different friction coefficients are set according to the surface material of the instruments; for example, metal instruments have a lower friction coefficient, while textured instruments have a relatively higher friction coefficient. By setting these parameters, a virtual surgical scene that conforms to real-world physical characteristics is constructed, realistically reflecting the physical phenomena that may occur during surgery.

[0084] The cutting process simulation submodule invokes the constructed virtual surgical scene and executes virtual cutting operations according to the resection path coordinate sequence in the final surgical plan. During the simulation, the submodule calculates the tissue stress distribution on the cutting surface in real time, and determines the stress transmission path and distribution range in the tissue by analyzing the force at the contact point between the cutting tool and the liver tissue. Simultaneously, based on the physical characteristics of the liver tissue and the cutting speed, it simulates tearing and tensile deformation of the liver tissue. When the cutting speed is high, the tissue may exhibit more obvious tearing; while at a slower cutting speed, the tissue exhibits more slow tensile deformation. The submodule records tissue deformation parameters at different cutting stages, including the degree, range, and speed of deformation. These parameters can reflect the morphological changes of the liver tissue in detail during the cutting process.

[0085] After receiving the tissue deformation parameters generated by the cutting process simulation submodule, the simulation result evaluation submodule begins to evaluate the simulation results. First, based on the tissue deformation parameters and the distribution of blood vessels involved in the cutting path, the simulated bleeding volume during the virtual cutting process is calculated. The calculation of the simulated bleeding volume is based on factors such as the number and diameter of blood vessels crossed by the cutting path, and the degree of tissue damage. When the cutting path crosses larger blood vessels and the tissue damage is more severe, the simulated bleeding volume increases accordingly.

[0086] The submodule analyzes the distribution of surgical operation time, recording the time spent at each stage from the start of the incision to the completion of the entire resection, such as the time for cutting the liver surface tissue, separating the tissue surrounding the tumor, and handling blood vessels. Simultaneously, it compares this time data with the average operation time of actual surgeries, analyzes the differences between the two, and identifies potential time-consuming points in the simulated operation.

[0087] The submodule integrates tissue deformation parameters, simulated blood loss data, and operation time estimates to generate a complete surgical simulation result. This result comprehensively reflects various situations during the virtual surgery process, providing surgeons with reference information on surgical difficulty, possible tissue reactions, and operation time, helping them better understand the surgical procedure and prepare for various situations in advance.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A liver surgery planning system based on data fusion, characterized in that, The system comprises: The image data fusion module collects CT images and MRI images, integrates the spatial coordinates of the CT images and the MRI images, constructs an image feature mapping relationship, matches the anatomical structure identifiers of the CT images and the MRI images, and generates a standardized image fusion model; The three-dimensional model construction module extracts liver organ contour features, reconstructs intrahepatic blood vessel network topology, labels the spatial position parameters of the tumor region, and generates a liver three-dimensional visualization model based on the standardized image fusion model; The surgical path planning module analyzes the relative distance between the tumor and the blood vessels, evaluates the accessibility index of the surgical approach, plans the optimal resection path that avoids important blood vessels, and generates a preliminary surgical path scheme based on the liver three-dimensional visualization model; The risk assessment module calculates the predicted resection volume and residual liver function ratio, identifies potential bleeding risk areas, evaluates the probability of surgical complications, and generates a surgical risk assessment report based on the preliminary surgical path scheme; The scheme optimization module adjusts the resection boundary parameters, optimizes the operation angle of the surgical instrument, updates the risk area avoidance strategy, and generates a final surgical planning scheme based on the surgical risk assessment report; The scheme optimization module comprises: The resection boundary adjustment submodule expands the reserved area if the residual liver volume ratio is below the safety threshold, shrinks the resection boundary if the bleeding risk level is too high, re-determines the three-dimensional coordinate range of the resection area, and generates optimized resection boundary data based on the surgical risk assessment report; The operation angle optimization submodule calls the optimized resection boundary data, simulates the operation field of the surgical instrument at different angles, calculates the movement range of the instrument tip, selects an angle with clear vision and flexible operation, and generates an instrument operation angle range; The avoidance strategy update submodule increases the avoidance buffer distance for high-risk blood vessel areas in the path, adjusts the position parameters of the path turning points, ensures that all key structures are outside the safety distance, and generates a final surgical planning scheme based on the instrument operation angle range.

2. The data fusion based liver surgery planning system of claim 1, wherein, The standardized image fusion model includes modality registration parameters, an anatomical structure mapping table, and image resolution adaptation values. The liver three-dimensional visualization model includes organ contour coordinate sets, blood vessel branch topology maps, and tumor region annotation data. The preliminary surgical path scheme includes approach angle parameters, resection path coordinate sequences, and important structure avoidance markers. The surgical risk assessment report includes liver volume ratio, bleeding risk level, and complication probability prediction values. The final surgical planning scheme includes optimized resection boundary data, instrument operation angle range, and risk area avoidance update strategy.

3. The data fusion based liver surgery planning system of claim 1, wherein, The image data fusion module comprises: The multi-modal image acquisition submodule collects CT image data and MRI image data, obtains image layer thickness and pixel spacing parameters, extracts DICOM metadata information of the images, identifies the scanning coordinate systems of different modal images, and generates an original image data set; The spatial coordinate registration submodule calls an image rigid registration algorithm based on the original image data set, aligns the spatial coordinate origins of different modal images, calculates the coordinate system conversion matrix, corrects the spatial position deviation between the images, and generates a coordinate registration result; The feature fusion processing sub-module extracts liver parenchyma and blood vessel feature points in the image based on the coordinate registration result, establishes a mapping relationship between the feature points, fuses the gray value information of different modal images, and generates a standardized image fusion model.

4. The data fusion based liver surgery planning system of claim 1, wherein, The three-dimensional model construction module comprises: The organ contour extraction sub-module extracts three-dimensional coordinate points of the liver surface contour based on the standardized image fusion model, fits a contour curve to generate a closed surface, and obtains a liver organ contour model by using a region growing algorithm to segment the liver parenchyma region. The blood vessel network reconstruction sub-module identifies a blood vessel cross-sectional region in the image, traces the blood vessel direction and connects the branch structure, constructs a topological relationship diagram of the blood vessel network, labels the blood vessel diameter and branch angle parameters, and generates a three-dimensional model of the intrahepatic blood vessels based on the liver organ contour model. The model precision verification sub-module calls the three-dimensional model of the intrahepatic blood vessels, compares the position of the anatomical structure in the actual image, calculates the spatial error value of the model and the image, adjusts the grid density parameters of the model, optimizes the surface smoothness of the model, and generates a three-dimensional visual model of the liver.

5. The data fusion based liver surgery planning system of claim 1, wherein, The surgical path planning module comprises: The anatomical structure analysis sub-module locates the tumor center coordinate position, measures the shortest distance between the tumor and the hepatic portal vein and the hepatic vein, and counts the distribution density of the blood vessels around the tumor based on the three-dimensional visual model of the liver, and generates anatomical structure distance parameters. The approach accessibility evaluation sub-module simulates the instrument operation space in different surgical approach directions based on the anatomical structure distance parameters, calculates the collision probability of the surgical instrument and the important blood vessels, evaluates the feasibility score of the approach path, and selects a high feasibility approach direction. The resection path generation sub-module plans a resection path coordinate sequence from the surface of the liver to the tumor region based on the high feasibility approach direction, sets key turning points on the path, ensures that the path avoids blood vessels with a diameter greater than three millimeters, and generates a preliminary surgical path scheme.

6. The data fusion based liver surgery planning system of claim 1, wherein, The risk assessment module comprises: The liver volume calculation sub-module segments the liver region to be resected and the residual liver region based on the preliminary surgical path scheme, calculates the three-dimensional volume values of the two regions, obtains the ratio of the residual liver volume to the standard liver volume, and generates a liver volume ratio parameter. The bleeding risk analysis sub-module identifies the number and diameter of blood vessels passing through the resection path based on the liver volume ratio parameter, evaluates the blood vessel damage probability, combines the patient's coagulation function indicators, calculates the potential bleeding volume, and generates a bleeding risk grade. The complication prediction sub-module analyzes whether the residual liver volume meets the functional requirements based on the bleeding risk grade, evaluates the probability of liver failure, combines the patient's age and underlying disease information, comprehensively predicts the probability of surgical complications, and generates a surgical risk assessment report.

7. The data fusion based liver surgery planning system of claim 1, wherein, The system further comprises: The surgical simulation module constructs a virtual surgical environment based on the final surgical planning scheme, simulates the cutting mechanical properties of liver tissue, calculates the tissue deformation amount under different resection speeds, verifies the executability of the planning scheme, and generates a surgical simulation result. The surgical simulation result specifically includes tissue deformation parameters, simulated bleeding volume data, and operation time estimation results.

8. The data fusion based liver surgery planning system of claim 7, wherein, The surgical simulation module comprises: The virtual environment construction submodule imports a liver three-dimensional visualization model and a surgical instrument model based on the final surgical planning scheme, sets elastic modulus and friction coefficient parameters of tissues, and constructs a virtual surgical scene conforming to real physical characteristics; The cutting process simulation submodule calls the virtual surgical scene, performs virtual cutting operations according to the planning path, calculates tissue stress distribution of a cutting surface, simulates tearing and tensile deformation of liver tissues, and records tissue deformation parameters; The simulation result evaluation submodule statistically analyzes simulated bleeding volume in the virtual cutting process based on the tissue deformation parameters, analyzes distribution of a surgical operation time, compares the average operation time of an actual surgery, and generates a surgical simulation result.

9. The data fusion based liver surgery planning system of claim 5, wherein, The resection path generation submodule further comprises a dynamic adjustment unit which receives intraoperative image feedback data in real time when planning a path, compares deviations between a preset path and actual anatomic structures, automatically corrects path turning point coordinates, and generates a real-time updated resection path scheme.

Images