Soft landmark weighted hepato-biliary navigation registration method and device, computer equipment and readable storage medium

Abstract

The application relates to a liver and gallbladder navigation registration method and device based on soft landmark point weighting, computer equipment and a readable storage medium. The method comprises the following steps: acquiring each first landmark point in a three-dimensional model coordinate system and a landmark point set corresponding to each first landmark point; the landmark point set comprises a plurality of second landmark points, the second landmark points are a plurality of landmark points collected in an organ tissue that is deformed under a physical coordinate system and is aimed at a single first landmark point; for each first landmark point, a landmark point is predicted according to a landmark point prediction strategy and each second landmark point, and a soft landmark point corresponding to the first landmark point is determined; based on a preset weighted registration optimization strategy, each first landmark point and the soft landmark point corresponding to each first landmark point, registration calculation is performed on the three-dimensional model coordinate system and the physical coordinate system to obtain a registration result; and the registration result is used for liver and gallbladder navigation. The method can improve registration accuracy.

Description

Technical Field

[0001] This application relates to the fields of medical image registration, intraoperative navigation, and hepatobiliary navigation technology, and in particular to a soft marker-weighted hepatobiliary navigation registration method, apparatus, computer equipment, and computer-readable storage medium. Background Technology

[0002] With the development of hepatobiliary navigation diagnostic technology, electromagnetic sensors are widely used in intraoperative instrument pose estimation and navigation presentation. Electromagnetic sensors can determine the coordinates of anatomical landmarks in organs and tissues within the intraoperative physical coordinate system. However, before performing endoscopic navigation, it is necessary to spatially register the preoperative 3D model coordinate system with the intraoperative physical coordinate system.

[0003] In traditional techniques, for each registration marker in the three-dimensional model coordinate system, the anatomical marker corresponding to each registration marker is determined in the organ tissue by the acquisition device. Then, the anatomical marker (hard marker) is taken as the real position of the registration marker in the intraoperative coordinate system. The mapping relationship between the anatomical marker (in the physical coordinate system) and the registration marker (in the three-dimensional model coordinate system) is calculated according to the registration algorithm. This mapping relationship is used as the registration result to achieve the registration between the three-dimensional model coordinate system and the physical coordinate system.

[0004] However, in current traditional technologies, due to the slight tremors when the user operates the probe or the deformation of organs and tissues caused by respiration, there is a positional error between the anatomical landmark and the landmark to be registered. The position coordinates of the anatomical landmark cannot reflect the true position of the landmark to be registered, which in turn leads to poor registration accuracy between the 3D model coordinate system and the physical coordinate system. Summary of the Invention

[0005] Therefore, it is necessary to provide a soft-marker-weighted hepatobiliary navigation registration method, apparatus, computer equipment, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0006] Firstly, this application provides a soft-marker-weighted hepatobiliary navigation registration method, including:

[0007] Obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0008] For each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point.

[0009] Based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, the registration calculation is performed between the three-dimensional model coordinate system and the physical coordinate system to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0010] In one embodiment, the step of performing marker prediction on the first marker point according to the marker prediction strategy and each of the second marker points for each first marker point, and determining the soft marker point corresponding to the first marker point, includes:

[0011] For each first marker point, the central trend position of each second marker point is predicted according to the marker point prediction strategy and each second marker point, and the central trend position is determined as a soft marker point.

[0012] In one embodiment, predicting the central trend position of each of the second markers for each of the first markers, based on the marker prediction strategy and each of the second markers, includes:

[0013] For each of the first marker points, the spatial mean of the corresponding second marker points is calculated to obtain the central trend position of the set of marker points.

[0014] In one embodiment, the registration result is the rigid body transformation parameters from the 3D model coordinate system to the physical coordinate system; the registration calculation of the 3D model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each first marker point, and the soft marker points corresponding to each first marker point, to obtain the registration result, includes:

[0015] Each first marker is evaluated according to a preset evaluation strategy and each second marker, and the confidence result corresponding to each first marker is obtained.

[0016] Based on the preset weighted registration optimization strategy and the confidence results of each of the first marker points, the registration weights corresponding to each of the soft marker points are determined.

[0017] The spatial deviations between each of the first marker points and the soft marker points are weighted according to the registration weights, and the rigid body transformation parameters from the three-dimensional model coordinate system to the physical coordinate system are determined based on the weighting results and the preset optimization target.

[0018] In one embodiment, the step of evaluating each first marker point according to a preset evaluation strategy and each second marker point to obtain the confidence result corresponding to each first marker point includes:

[0019] The distribution dispersion of the second marker points is determined according to the preset evaluation strategy and each second marker point, and the confidence result corresponding to the first marker point is obtained according to the distribution dispersion.

[0020] In one embodiment, after performing registration calculations on the 3D model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each first marker point, and the soft marker points corresponding to each first marker point, and obtaining the registration result, the method further includes:

[0021] The evaluation index of the rigid body transformation parameters is determined based on the registration weight and / or the registration result.

[0022] Visual feedback information is generated based on the evaluation indicators, and the visual feedback information is displayed.

[0023] Secondly, this application also provides a soft-marker-weighted hepatobiliary navigation registration system, the system comprising a positioning acquisition module and a registration calculation module, wherein:

[0024] The registration calculation module is used to obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the deformed organ tissue in the physical coordinate system for a single first marker point; for each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point; based on the preset weighted registration optimization strategy, each first marker point and the soft marker point corresponding to each first marker point, the three-dimensional model coordinate system and the physical coordinate system are registered to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0025] The positioning and acquisition module is used to acquire the second marker points corresponding to each of the first marker points in the deformed organ tissue under the physical coordinate system.

[0026] Thirdly, this application also provides a soft-marker-weighted hepatobiliary navigation registration device, comprising:

[0027] The acquisition module is used to acquire each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0028] The prediction module is used to predict the first marker point for each first marker point according to the marker point prediction strategy and each second marker point, and determine the soft marker point corresponding to the first marker point.

[0029] The registration module is used to perform registration calculations between the three-dimensional model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0030] In one embodiment, the prediction module is specifically used to predict the central trend position of each of the second markers for each of the first markers according to the marker prediction strategy and each of the second markers, and to determine the central trend position as a soft marker.

[0031] In one embodiment, the prediction module is specifically used to calculate the spatial mean of the second marker corresponding to each first marker to obtain the central trend position of the marker set.

[0032] In one embodiment, the registration result is a rigid body transformation parameter from the three-dimensional model coordinate system to the physical coordinate system; the registration module is specifically used to evaluate each of the first marker points according to a preset evaluation strategy and each of the second marker points, and obtain the confidence result corresponding to each of the first marker points;

[0033] Based on the preset weighted registration optimization strategy and the confidence results of each of the first marker points, the registration weights corresponding to each of the soft marker points are determined.

[0034] The spatial deviations between each of the first marker points and the soft marker points are weighted according to the registration weights, and the rigid body transformation parameters from the three-dimensional model coordinate system to the physical coordinate system are determined based on the weighting results and the preset optimization target.

[0035] In one embodiment, the registration module is specifically used to determine the distribution dispersion of the second marker points according to a preset evaluation strategy and each second marker point, and to obtain the confidence result corresponding to the first marker point according to the distribution dispersion.

[0036] In one embodiment, the device further includes:

[0037] An evaluation module is used to determine the evaluation index of the rigid body transformation parameters based on the registration weights and / or the registration results.

[0038] The generation module is used to generate visual feedback information based on the evaluation indicators and to display the visual feedback information.

[0039] Fourthly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0040] Obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0041] For each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point.

[0042] Based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, the registration calculation is performed between the three-dimensional model coordinate system and the physical coordinate system to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0043] Fifthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0044] Obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0045] For each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point.

[0046] Based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, the registration calculation is performed between the three-dimensional model coordinate system and the physical coordinate system to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0047] Sixthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0048] Obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0049] For each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point.

[0050] Based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, the registration calculation is performed between the three-dimensional model coordinate system and the physical coordinate system to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0051] The aforementioned soft-marker-weighted hepatobiliary navigation registration method, apparatus, computer equipment, computer-readable storage medium, and computer program product, by acquiring multiple second markers in the physical coordinate system to form a marker set for each first marker point used for registration calculation in the 3D model coordinate system, and predicting soft markers that can represent the true position of the first marker points according to the marker point prediction strategy, can reduce the influence of position errors caused by random jitter and respiratory deformation during the acquisition of second marker points. Furthermore, based on the confidence results, the registration weight of each first marker point is determined, and the influence of using the first marker point and its corresponding soft marker point for registration calculation is controlled by the registration weight. The registration solution is then performed using a weighted registration optimization strategy, thereby improving the registration accuracy between the 3D model coordinate system and the physical coordinate system. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart illustrating a soft-marker-weighted hepatobiliary navigation registration method in one embodiment.

[0054] Figure 2 This is a schematic diagram of the registration calculation based on the first marker point and the soft marker point in one embodiment;

[0055] Figure 3 This is a flowchart illustrating the process of determining evaluation indicators and displaying visual feedback information in one embodiment.

[0056] Figure 4 This is a schematic diagram of a soft-marker-weighted hepatobiliary navigation registration system in one embodiment;

[0057] Figure 5 This is a flowchart illustrating an example of a soft-marker-weighted hepatobiliary navigation registration method in one embodiment.

[0058] Figure 6 This is a structural block diagram of a soft-marker-weighted hepatobiliary navigation registration device in one embodiment;

[0059] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0061] In one embodiment, such as Figure 1 As shown, a soft-marker-weighted hepatobiliary navigation registration method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and furthermore, to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0062] Step 102: Obtain each first marker point and the set of marker points corresponding to each first marker point in the 3D model coordinate system.

[0063] The set of landmarks includes multiple second landmarks. The second landmarks are multiple landmarks collected in the deformed organ tissue in the physical coordinate system in relation to a single first landmark. The physical coordinate system is the intraoperative coordinate system, which can be the coordinate system of the intraoperative positioning system, the patient reference frame, the sensor system, or other physical positioning systems.

[0064] In this embodiment of the application, in the practical application of hepatobiliary endoscopic navigation, it is necessary to pre-construct a three-dimensional model of the hepatobiliary tissue. This three-dimensional model represents the structure of the actual hepatobiliary tissue, including anatomical structures such as bile ducts, blood vessels, and lesions. This three-dimensional model is used to provide a spatial framework of the hepatobiliary anatomical structure for intraoperative hepatobiliary navigation, so that the three-dimensional model can be aligned with the actual position and direction of the actual hepatobiliary tissue through registration calculation, thereby guiding instruments such as endoscopes to accurately reach the target position.

[0065] A first set of landmark points is pre-defined in the 3D model. These landmark points are model landmark points located in the preoperative 3D model coordinate system. They can be landmark points with anatomical features pre-selected by the user in the 3D model, such as bifurcation points of the biliary system, vascular bifurcation points, attachment points of the round ligament of the liver, and the lowest point of the gallbladder fossa. Furthermore, when registering the physical coordinate system of the actual hepatobiliary tissue with the 3D model coordinate system of the 3D model, each of the first landmark points serves as the original basis for registration.

[0066] In one embodiment, during the application of hepatobiliary endoscopic navigation, the terminal acquires the constructed 3D model and contacts the actual hepatobiliary tissue through a navigation pointer or an endoscope front-end sensor (hereinafter referred to as a probe) to acquire multiple second markers corresponding to each first marker point, forming a marker point set corresponding to each first marker point. One first marker point corresponds to multiple second marker points, and multiple second marker points constitute a marker point set; that is, one first marker point corresponds to one marker point set. Specifically, in other optional embodiments, sampling can also be assisted by an automated or semi-automatic positioning device, but the sampling results are still based on multiple 3D position data collected in the intraoperative coordinate system for the same anatomical landmark location. That is, the second marker point can be a 3D position point obtained by controlling the probe in the actual hepatobiliary tissue through automated equipment, or a 3D position point obtained by manually controlling the probe in the actual hepatobiliary tissue. Optionally, the sensor type can be an electromagnetic sensor, or an optical, inertial, or multimodal fusion sensor.

[0067] For automated equipment acquisition, the system can obtain real-time images of the actual hepatobiliary tissue captured by the probe, identify the location of the first landmark in the real-time image, and control the probe to acquire the second landmark based on the identification result. For manual acquisition by the user, the user actively controls the probe to acquire the point closest to the actual location corresponding to the first landmark, based on the intraoperative field of view and anatomical experience, as the second landmark. Due to the deformation of hepatobiliary tissue caused by respiration, the vibration of the probe during manual control, and the inherent errors in the identification and control of automated equipment, there is a certain error between the second landmark and the actual location of the first landmark in the actual hepatobiliary tissue. However, the acquisition goal of the second landmark is to approximate the actual location, so the distribution of a large number of second landmarks can reflect the spatial trend of the actual location, i.e., only a small number of second landmarks with larger errors exist. Based on the second landmark, subsequent analysis and calculation can determine the soft landmark corresponding to the first landmark in the actual hepatobiliary tissue. A soft landmark refers to a landmark representation obtained by statistically analyzing a set of landmarks, which includes at least information on the representative location and dispersion. The representative location is used to indicate the most likely location of the anatomical landmark in the intraoperative coordinate system; the dispersion information is used to characterize the sampling uncertainty of multiple second landmarks around the representative location.

[0068] In one specific embodiment, for the acquisition of the second marker point, the terminal can control the probe based on a pen-shaking sampling method. Specifically, the terminal controls the automated equipment to continuously and minutely move the probe within the local spatial neighborhood corresponding to the first marker point within a preset time window, thereby acquiring multiple discrete spatial positions as the second marker point. The local spatial neighborhood corresponding to the first marker point can be a spherical region with a radius of approximately 2 to 5 millimeters centered on the target anatomical location. This target anatomical location can be preset by the user and serves as a reference position for the first marker point in the 3D model within the actual hepatobiliary tissue. This reference position characterizes the "approximate location" or "fuzzy area" of the first marker point within the actual hepatobiliary tissue. The neighborhood scale can be preset based on the probe tip size, the accessible area of ​​the organ tissue, and the sensor accuracy.

[0069] For example, let's take the method of data collection using automated equipment as an example. Each first marker point corresponds to a local spatial neighborhood. Let's take an example to illustrate. Indicates the first The spatial range within which local sampling is permitted near the first marker point is not a random, large area, but rather a controlled local region surrounding the target anatomical location. Furthermore, the terminal controls the probe within a time window. Internal continuous acquisition One sample (e.g., 10 to 50 sampling points collected within 3 seconds); among which, This indicates the duration of continuous sampling at the same first marker point; Indicates the first The number of second marker samples actually collected from each first marker within the time window. The number of second markers corresponding to different first markers. The number of samples can be the same or different, and the set of marker points includes positional uncertainties caused by jitter, deformation, and noise. After sampling a first marker point, the terminal sequentially collects other preset first marker points in the same way, thereby obtaining an independent set of marker points corresponding to each first marker point.

[0070] Each sample (second marker point) represents a three-dimensional location. (If the pose can be obtained, it can also be recorded, but the position component should be used first for registration.) Among them, Indicates the first The first marker point The three-dimensional position coordinates of the second marker point obtained from the intraoperative sampling; This indicates the sampling number corresponding to the second marker point obtained by sampling the same first marker point; This represents three-dimensional Euclidean space, meaning the sample contains three spatial coordinate components: x, y, and z. These spatial coordinate components characterize the position information of the second marker point in the physical coordinate system. Therefore, after completing pen-shaking sampling for each first marker point, the terminal obtains a set of marker points for each first marker point. ,in, Indicates the first The set of markers corresponding to the first marker; These represent multiple samples continuously collected from the first marker point within a preset time window, serving as the second marker point. Each first marker point corresponds to a marker point set, meaning the second marker points in the set are obtained by sampling the same first marker point multiple times. Optionally, the second marker points can also be collected by sliding sampling along the anatomical structure.

[0071] In one embodiment, taking manual data acquisition by operator-controlled probes as an example, for each first landmark, the operator performs short-term continuous sampling near the anatomical landmark location corresponding to the first landmark using a navigation pointer, endoscope tip sensor, or equivalent positioning device. Specifically, within a local spatial neighborhood of the anatomical landmark location, multiple three-dimensional position data are continuously acquired within a preset time window, serving as multiple second landmarks. This local spatial neighborhood is a controlled local area surrounding the target anatomical landmark location, rather than a random large area. The preset time window can be set according to the actual operating scenario, instrument tip size, anatomical space size, and sensor sampling frequency. Through this short-term continuous sampling method, a single observation can be transformed into a sample set, thereby reflecting positional fluctuations caused by operator hand tremors, instrument slippage, soft tissue deformation, and sensor noise.

[0072] Therefore, pen-based sampling expands a single observation into a sample set by continuously collecting multiple second markers in a controlled local neighborhood for a short period of time. This can accurately reflect the positional fluctuations caused by instrument vibration, contact slippage, and minor tissue deformation. Compared with the "hard marker" method that uses single sampling results as soft markers, it avoids the random errors of single sampling being solidified into soft markers, thereby reducing error risk and providing a data foundation for subsequent uncertainty modeling. Moreover, the small sampling range and short time will not significantly increase the intraoperative burden.

[0073] Step 104: For each first marker point, perform marker point prediction based on the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point.

[0074] In this embodiment of the application, after obtaining the set of markers corresponding to each first marker, the terminal performs statistical or geometric analysis on the spatial distribution of the second markers through a marker prediction strategy to predict a soft marker that best represents the true spatial location of the first marker.

[0075] The marker prediction strategy is used to compensate for acquisition errors caused by probe jitter, liver and gallbladder tissue deformation, or probe sensor noise during acquisition, based on the spatial distribution information reflected by multiple sampled second marker points, and to generate soft marker points that closely approximate the actual locations of the first marker points. Specifically, the marker prediction strategy can be to calculate the spatial mean point based on the spatial coordinates of all the second marker points and use the spatial mean point as the soft marker point; or, in an optional embodiment, the marker prediction strategy can also be to perform spatial clustering on the second marker points, for example, using K-means or DBSCAN clustering algorithms to extract each cluster in the second marker points and using the centroid of the largest cluster as the soft marker point.

[0076] In an optional embodiment, the terminal can also perform quality assessment on the set of marker points according to preset evaluation indicators, calculate at least one evaluation indicator, determine the type of the set of marker points based on the evaluation indicators, and then adaptively select a soft marker point prediction strategy to calculate soft marker points. For example, the preset evaluation indicators include the dispersion index of the second marker points and the outlier index. The dispersion index can be the trace of the covariance matrix of the marker point set, the maximum eigenvalue, or the overall radius of the point set (the maximum distance between any two second marker points), used to reflect the spatial clustering degree of the second marker points; the outlier index can be the distance distribution of each second marker point relative to the candidate center point (the candidate center point can be the mean or the median), and the proportion of the number of second marker points whose distance exceeds a preset threshold is statistically analyzed. This proportion is used to assess whether there are obvious outlier sampling points in the set of marker points.

[0077] If the dispersion index is lower than the first preset threshold and the proportion of outliers is lower than the second preset threshold, the set of marker points is determined to be stable, indicating that the second marker points are highly clustered and have no obvious anomalies. If the dispersion index is lower than the first preset threshold and the proportion of outliers is higher than the second preset threshold, it indicates that some second marker points are far from the main cluster, and the set of marker points is determined to be outlier. If the dispersion index is higher than or equal to the first preset threshold and the proportion of outliers is still lower than the second preset threshold, the set of marker points is determined to be jittery, indicating that the second marker points are scattered, which is normal hand tremor or slight deformation. If the dispersion index is higher than or equal to the first preset threshold and the proportion of outliers is higher than the second preset threshold, it indicates that the dispersion is high and the proportion of outliers is high, indicating that outliers are the main factor causing dispersion, and the set is determined to be outlier.

[0078] If the type of the marker set is stable or jittery, it means that the overall distribution of the second marker can represent the true location, so the terminal determines the soft marker based on the spatial mean of the second marker. If the type of the marker set is outlier, in order to reduce the impact of outliers on the prediction of the true location, the terminal determines the soft marker based on the clustering algorithm.

[0079] Step 106: Based on the preset weighted registration optimization strategy, each first marker point and the soft marker point corresponding to each first marker point, perform registration calculation on the three-dimensional model coordinate system and the physical coordinate system to obtain the registration result.

[0080] The registration results are used for hepatobiliary navigation.

[0081] Optionally, the registration model used for registration calculation can be a rigid body registration model, or it can be a semi-rigid body registration model or a locally non-rigid body registration model.

[0082] In this embodiment, the terminal evaluates the first markers based on a preset weighted registration optimization strategy and the distribution of the second markers corresponding to each first marker. The evaluation result reflects the degree of influence of the first marker on the registration accuracy when used for registration calculation. This influence is then quantified into registration weights to control the contribution of each first marker to the registration process. For example, a more stable distribution of the second markers indicates that the soft markers determined based on the second markers are closer to their true positions, meaning the registration result obtained from the first markers corresponding to the second marker is more accurate, i.e., the confidence level of the first marker is higher. Therefore, the contribution of the first marker to the registration calculation should be increased, i.e., the registration weight of the first marker should be increased. Conversely, a larger sampling fluctuation of the second markers indicates a larger error between the soft markers and the true positions of the first markers, i.e., the lower the confidence level of the first markers. Therefore, the contribution of the first marker to the registration calculation should be weakened, and thus, the registration weight of the first marker should be reduced. Ultimately, the overall registration result can be mainly dominated by high-confidence observations, thereby effectively suppressing the interference of individual unstable points or anomalous sampling on the registration calculation.

[0083] After obtaining the registration weights of each first marker point, the terminal calculates the mapping relationship between the 3D model coordinate system and the physical coordinate system based on the relative positional relationships between the first marker points, the relative positional relationships between the soft marker points, and the correspondence between the soft marker points and the first marker points, combined with a preset weighted registration optimization strategy. Specifically, there are fixed relative positional relationships between the first marker points in the 3D model, such as relative distances and relative angles. Similarly, there are relative positional relationships between the soft marker points in the physical coordinate system. Therefore, an optimal rigid body transformation (rotation and translation) parameter is determined through the preset weighted registration optimization strategy, minimizing the overall metric of all weighted residuals (e.g., weighted sum of squares or weighted Mahalanobis distance), resulting in a rotation matrix and translation vector as the registration result. This aligns the overall spatial orientation of the preoperative 3D model with the physical coordinate system of the actual hepatobiliary tissue.

[0084] Ultimately, in practical applications of hepatobiliary navigation, this registration result will be used to map all anatomical structures in the preoperative 3D model (e.g., liver outline, bile duct tree, vascular distribution, lesion location, etc.) to the intraoperative physical coordinate system. Then, the terminal, combined with real-time tracked instrument pose, provides precise navigation guidance to the user on the screen in a fused display manner. For example, it displays the current position of the instrument in the hepatobiliary tissue, the distance to the target lesion, and directional prompts for the optimal travel path, thus achieving hepatobiliary navigation.

[0085] In the aforementioned soft-marker weighted hepatobiliary navigation registration method, multiple second markers are collected in the physical coordinate system to form a marker set for each first marker used for registration calculation in the 3D model coordinate system. A marker prediction strategy is then used to predict soft markers that represent the true position of the first markers. This reduces the impact of positional errors caused by random jitter and respiratory deformation during the acquisition of the second markers. Furthermore, the registration weight of each first marker is determined based on the confidence level. The registration weight controls the influence of using the first markers and their corresponding soft markers for registration calculation. Finally, a weighted registration optimization strategy is used to solve the registration problem, thereby improving the registration accuracy between the 3D model coordinate system and the physical coordinate system.

[0086] In an exemplary embodiment, for the prediction of soft markers, the central trend position can be predicted based on the distribution of the second markers, and the central trend position is used as the soft marker. Step 104 includes step 1041. Wherein:

[0087] Step 1041: For each first marker point, predict the central trend position of each second marker point according to the marker point prediction strategy and each second marker point, and determine the central trend position as a soft marker point.

[0088] In this embodiment of the application, after obtaining multiple second markers corresponding to each first marker, the terminal further performs spatial central tendency analysis on the second markers, calculates the central trend position that can represent the statistical significance of the marker through a preset marker prediction strategy, and determines the position as a soft marker.

[0089] Specifically, the marker prediction strategy used to predict the central trend position can be spatial mean method, spatial weighted mean method, geometric center method, etc. This application does not limit the marker prediction strategy used for central trend prediction, as long as it can extract a soft marker representing the central trend position from multiple discretely sampled second marker points.

[0090] In this embodiment, by using the central trend position of multiple sampling points as soft markers, the random errors caused by random jitter, contact slip and tissue deformation in a single sampling can be suppressed, making the soft markers closer to the real spatial position of the first marker, thereby improving the accuracy of subsequent registration calculations.

[0091] In an exemplary embodiment, step 1041 includes 1042. Wherein:

[0092] Step 1042: For each first marker point, calculate the spatial mean of the second marker point corresponding to the first marker point to obtain the central trend position of the marker point set.

[0093] In this embodiment, for each first marker point, the terminal calculates the spatial mean of each second marker point in the marker point set corresponding to the first marker point, and uses the spatial mean as the soft marker point corresponding to the first marker point. The calculation of the soft marker point is shown in the following formula (1):

[0094] (1)

[0095] in, Indicates the first Each first marker point corresponds to a representative position of a soft marker point, that is, the representative position of the first marker point in the physical coordinate system. This representative position represents the point in the physical coordinate system that is closest to the actual position of the first marker point. Indicates that for the first The number of samples taken from the first marker point is the number of second marker points; Indicates the first The second marker point The three-dimensional position obtained from the second sampling; Indicates the first The positions of the second marker points obtained by sampling the first marker point are summed. It is the spatial center of each second marker in the set of markers obtained by sampling the same first marker, and is used to represent the most likely true position of the first marker in the physical coordinate system.

[0096] In this embodiment, the spatial mean of multiple second marker points is used as the central trend position, which can suppress errors caused by random jitter, contact slippage and sensing noise, and obtain a representative point that is closest to the true position corresponding to the first marker point, thereby improving the positioning accuracy of the soft marker point and thus improving the accuracy of the registration calculation.

[0097] In an exemplary embodiment, the registration result is the rigid body transformation parameters from the 3D model coordinate system to the physical coordinate system; such as Figure 2 As shown, step 106 includes steps 202 to 206. Wherein:

[0098] Step 202: Evaluate each first marker point according to the preset evaluation strategy and each second marker point to obtain the confidence result corresponding to each first marker point.

[0099] In this embodiment, the preset evaluation strategy may be to calculate the dispersion of the marker point set based on the second marker points. For example, the dispersion may be the covariance of the marker point set. The terminal analyzes multiple second marker points corresponding to each first marker point according to the preset evaluation strategy, calculates the covariance matrix of the marker point set, and uses the covariance matrix as the confidence result of the first marker point. Here, the first marker points and soft marker points are in one-to-one correspondence. Soft marker points are generated based on second marker points, and registration calculation is implemented based on multiple first marker points and the soft marker points corresponding to each first marker point. Therefore, the confidence result represents the marker point pair formed by each first marker point and soft marker point. When used to perform registration calculation, it obtains the confidence of the registration result. Thus, the confidence result reflects the confidence of the first marker points and is also used to reflect the confidence of the soft marker points. Optionally, the preset evaluation strategy may also be to calculate the Euclidean distance between any two second marker points in the marker point set and use the maximum Euclidean distance as the overall radius to represent the overall span of the marker point set, indicating that the more concentrated the sampled second marker points are, the higher the confidence.

[0100] Specifically, the covariance matrix is ​​calculated by performing a second-order matrix operation on the deviations between all second marker points and their spatial mean, thus comprehensively quantifying the positional uncertainty and spatial dispersion of the first marker point during the sampling process. A smaller covariance indicates that the second marker points are tightly clustered, suggesting a higher confidence level for the first marker point; a larger covariance indicates that the sampling process is significantly affected by jitter, slippage, or tissue deformation, resulting in a lower confidence level for the first marker point.

[0101] Step 204: Based on the preset weighted registration optimization strategy and the confidence results of each first marker point, determine the registration weight corresponding to each soft marker point.

[0102] In this embodiment, the terminal uses the confidence result of the first marker point as the basis for weight division, thereby controlling the contribution of the first marker point to the registration calculation during the registration process. This allows the first marker point with a high confidence result and stable second marker point sampling to play a dominant role in the registration result, while the first marker point with a low confidence result and unstable second marker point sampling has its influence on the registration result automatically weakened. This results in obtaining the registration weight of the soft marker point's position value in the registration calculation.

[0103] Specifically, in the process of determining the registration weights corresponding to each soft marker point based on the preset weighted registration optimization strategy and the confidence results of each first marker point, the terminal first, in the three-dimensional model coordinate system... Selected from The first marker is The registration weights corresponding to the soft markers are: ,in, For the first The uncertainty parameters of each soft marker point, i.e., the covariance matrix of the second marker point regularized to the soft marker point. For the first The covariance matrix of each soft marker point; The coordinate system of the three-dimensional model before surgery can be the coordinate system of the three-dimensional liver and gallbladder model obtained by preoperative CT (Computed Tomography), MRI (Nuclear Magnetic Resonance Imaging), or MRCP (Magnetic Resonance Cholangiopancreatography). Indicates the first The first marker point is at Three-dimensional coordinates in; This indicates the total number of the first marker points participating in the registration; This indicates the intraoperative physical coordinate system, i.e., the coordinate system in which the intraoperative positioning system, patient reference frame, or sensor system is located. Indicates the relationship with the first The first marker The corresponding soft marker modeling representation, where, As a representative position, Due to the uncertainty of this location, it should be noted that and The correspondence is derived from preoperative numbering, anatomical semantics, and intraoperative sampling by number, rather than being simply determined by equal coordinate values ​​before registration. Since there is a one-to-one correspondence between soft markers and the first marker, the covariance matrix also reflects the confidence level of the first marker.

[0104] Step 206: Weight the spatial deviation between each first marker point and the soft marker point according to the registration weight, and determine the rigid body transformation parameters from the three-dimensional model coordinate system to the physical coordinate system based on the weighting result and the preset optimization target.

[0105] In this embodiment, the terminal performs registration calculations using a preset weighted registration optimization strategy based on the registration weights. The spatial deviations between each first marker point and the soft marker point are weighted using the registration weights, and the rigid body transformation parameters from the three-dimensional model coordinate system to the physical coordinate system are solved based on the weighting results and the preset optimization objective. The rigid body transformation is shown in the following formula (2):

[0106] (2)

[0107] in, This indicates the first marker point in the preoperative 3D model coordinate system. Rigid body transformation function to be converted to the physical coordinate system during surgery; Represents any first landmark point in the preoperative model; This represents the rotation matrix to be estimated, used to describe the attitude change of the preoperative 3D model relative to the intraoperative physical coordinate system; It represents a special orthogonal group in three dimensions, namely the set of legal three-dimensional rotation matrices that satisfy orthogonality and do not contain scaling or mirroring. This represents the three-dimensional translation vector to be estimated, used to describe the positional offset of the preoperative three-dimensional model relative to the intraoperative coordinate system; This represents three-dimensional Euclidean space. The practical function of this rigid body transformation is to convert the first landmark point in the preoperative three-dimensional model of the liver and gallbladder to the physical coordinate system during surgery.

[0108] For the preset optimization objective of rigid body transformation, this embodiment preferably adopts weighted least squares in the form of Mahalanobis distance, and its optimization objective is as shown in the following formula (3):

[0109] (3)

[0110] in, This indicates that the rotation matrix is ​​solved by optimization. Translation vector This minimizes the total weighted residuals of all first marker points. Indicates all Summation is performed on the first registration markers; Indicates the first The first landmark in the preoperative 3D model Predicted position in the intraoperative physical coordinate system after rigid body transformation; Indicates the first One soft marker point; Indicates the first Registration residual vector of each marker point; Indicates the first The inverse of the regularized covariance matrix of a soft marker is the registration weight of the first marker corresponding to that soft marker.

[0111] If the covariance of a certain soft marker is large, then The effective constraint weakens, and the contribution of the soft marker to the objective function is suppressed; the smaller the covariance, the greater the registration weight, thus achieving the effect of the first marker with stable sampling and the target dominating the registration calculation, while suppressing the contribution of the first marker with unstable sampling to the registration calculation.

[0112] When the registration weights use matrix weights When calculating the Mahalanobis distance, the terminal can be solved using any of the following methods:

[0113] Method 1: Whitening and SVD (Single Value Decomposition) are used to construct [a method] for each soft marker point. make (For example, Cholesky decomposition), left-multiply the residuals. Then it is converted to equivalent weighted least squares, and then solved iteratively or in a closed loop; among which, Indicates the first Whitening matrix or weight decomposition matrix of each soft marker; This represents the whitening matrix multiplied by itself after its transpose; the product is equal to the inverse covariance matrix. Its function is to transform the weighted residuals in Mahalanobis distance form into ordinary least squares form, making it easier to solve using SVD or other optimization methods.

[0114] Method 2: Iterative Least Squares (Gauss–Newton / LM): The Lie algebra parameterization update (or quaternion parameterization) is performed iteratively to optimize the above formula (3) until convergence.

[0115] In an optional embodiment, the registration weights can be simplified to scalar weights. As shown in the following formula (4):

[0116] (4)

[0117] Therefore, the preset optimization objective is shown in the following formula (5):

[0118] (5)

[0119] in, Indicates the first The scalar weights of the soft markers are matrix weights. The simplified form; Represents the covariance matrix The trace, i.e. the sum of the diagonal elements of the matrix, can be used to characterize the overall dispersion of the second marker used to calculate the soft marker in three-dimensional space. Represents trace operation; Represents the covariance matrix The determinant of the matrix is ​​related to the volume of the uncertain ellipsoid; Represents determinant operations; This means compressing the dimensions of three-dimensional volume into a scale quantity; Represents the Euclidean norm; Indicates the first The squared residuals of each soft marker. This scalar weighted form characterizes the more stable the sampling. or The smaller, The larger the value, the stronger its impact on registration calculations when using the soft marker and the first marker; conversely, the more unstable the sampling becomes. The smaller the value, the weaker the influence of the soft marker and the first marker on registration.

[0120] When using scalar weights At this time, the terminal can directly solve using weighted Procrustes / weighted SVD. .

[0121] In this embodiment, by introducing registration weights based on sampling confidence, differentiated constraints are constructed for stable and unstable soft markers, so that the first marker with high confidence and the soft markers dominate the registration solution, reducing the impact of unstable first markers and soft markers on the registration accuracy, thereby improving the registration accuracy.

[0122] In one exemplary embodiment, step 202 includes step 2021. Wherein:

[0123] Step 2021: Determine the distribution dispersion of the second marker points according to the preset evaluation strategy and each second marker point, and obtain the confidence result corresponding to the first marker point based on the distribution dispersion.

[0124] In this embodiment, after determining the location of the soft marker, the terminal determines the distribution dispersion of the second marker according to a preset evaluation strategy and each second marker, and obtains the confidence result corresponding to the first marker based on the distribution dispersion. The preset evaluation strategy is to calculate the covariance matrix of the second marker. This covariance matrix is ​​used to quantify the dispersion of the second marker collected for the first marker in different spatial directions, and is used to reflect the confidence of the registration calculation based on the first marker. The confidence result is calculated as shown in the following formula (6):

[0125] (6)

[0126] in, Indicates the first The covariance matrix of the samples collected for each first marker (second marker) is used to describe the sampling points around the mean (soft marker) for that first marker. The degree of dispersion and directionality; Indicates the first The offset vector of the next sampling position relative to the mean of the sample at that marker point; This represents the outer product of the offset vector, used to accumulate the variance in each direction of the three-dimensional space and the correlation between different directions; This indicates the transpose operation.

[0127] Furthermore, the terminal represents the soft marker based on its representative location (the location of the nearest real point to the soft marker) and the confidence score, as shown in the following formula (7):

[0128] (7)

[0129] in, Indicates the first One soft marker point; This indicates the representative location of the soft marker; This represents the uncertainty parameter of the soft marker. Compared with traditional hard markers (which only determine the true position of a single marker representing the first marker), the soft marker in this embodiment not only contains position information but also confidence information. This confidence information (confidence result) is used to control the contribution of the first marker and the soft marker to the registration result during the registration calculation.

[0130] In an optional embodiment, when When there may be singularity or a large condition number, regularization can be used, as shown in the following formula (8):

[0131] (8)

[0132] in, It is a small constant, used to ensure reversibility and numerical stability. Indicates the first One soft marker is used for the subsequent registration weighted regularization covariance matrix; This represents the regularization parameter, which is usually a small positive number; Representing the identity matrix, it is typically used in 3D position registration. The identity matrix. This is achieved by adding a small amount to the diagonal of the covariance matrix. This can avoid non-invertible, approximately singular, or numerically unstable covariance matrices.

[0133] In this embodiment, through the above modeling method, each soft marker is represented as a most likely representative location and a confidence level reflecting the sampling stability, which can provide an accurate processing basis for the subsequent registration stage and improve the registration accuracy.

[0134] In one exemplary embodiment, such as Figure 3 As shown, after step 106, the method further includes steps 302 to 304. Wherein:

[0135] Step 302: Determine the evaluation index of rigid body transformation parameters based on the registration weights and / or registration results.

[0136] In this embodiment of the application, after the weighted registration calculation is completed, the terminal calculates an evaluation index reflecting the registration consistency through the registration weight and / or registration result, evaluates the quality of the registration result, and triggers selective resampling and updating of individual first marker points accordingly to achieve iterative improvement.

[0137] The evaluation metrics may include at least one of uncertainty metrics, consistency metrics, and overall metrics. Uncertainty metrics may include the principal axis length of the covariance ellipsoid, the volume of the covariance ellipsoid, the largest eigenvalue of the covariance matrix, or the trace of the covariance matrix. Consistency metrics may include point-level residuals or weighted residuals. Overall metrics may include the weighted residual mean, the weighted residual maximum value, or other metrics used to characterize the overall registration quality. Specifically, the terminal determines the consistency metrics based on the registration weights and the uncertainty metrics based on the registration results. Optionally, the terminal may also determine an overall evaluation metric based on the consistency metrics and the uncertainty metrics. For example, the overall evaluation metric may be the mean of the consistency metrics for all first marker points and / or the maximum value of the uncertainty metrics.

[0138] Specifically, for the calculation of the registration weight index, the terminal calculates the weighted residual of the registration weight according to the following formulas (9) and (10) to obtain the consistency index:

[0139] (9)

[0140] (10)

[0141] in, Indicates the first The ordinary registration residuals of the first marker point reflect the transformed positions of the model points. Representation of soft markers The three-dimensional deviation between them; Indicates the first The weighted residual of the first marker point reflects the registration consistency after considering the sampling uncertainty of that point.

[0142] For the calculation of registration results, the terminal calculates the covariance ellipsoid principal axis length, ellipsoid volume, etc., based on the registration results. or ,in, Indicates the first The largest eigenvalue of the covariance matrix of the set of markers corresponding to the first marker is used to reflect the uncertainty of sampling the first marker in the direction of maximum discreteness. Indicates the first The trace of the covariance matrix of the set of markers corresponding to the first marker is used to reflect the overall sampling dispersion of the first marker.

[0143] in, , Uncertainty indices can be calculated after soft marker modeling, without depending on the registration results. ;and , And registration consistency indicators such as the weighted residual mean / maximum value need to be obtained after registration is completed. Calculated later.

[0144] Step 304: Generate visual feedback information based on the evaluation indicators and display the visual feedback information.

[0145] In this embodiment, the terminal generates intuitive visual feedback information based on evaluation indicators and displays it to the user in real time through a display interface (e.g., a navigation screen or an endoscope display), thereby supporting human-machine collaborative error correction and decision-making.

[0146] Specifically, when the evaluation index corresponding to a certain first marker point exceeds a preset threshold, the system generates a resampling prompt to alert the operator that the first marker point may have unstable sampling or inconsistent registration. After the operator confirms, multiple samplings are performed only within the local spatial neighborhood of that first marker point, and the corresponding soft marker points and registration results are updated based on the re-collected multiple second marker points. This method does not require repeating the sampling process for all marker points, enabling the location of problem points, local correction, and rapid updates.

[0147] For each first marker point, the terminal generates a three-dimensional uncertainty ellipsoid based on the covariance matrix. The center of the ellipsoid is located at the representative position of the soft marker point, and the axis of the ellipsoid is determined by the eigenvectors of the covariance matrix, with the semi-axis length proportional to the square root of the corresponding eigenvalue. A smaller ellipsoid volume and a shape closer to a sphere indicate more stable sampling at that point; conversely, an elongated or large ellipsoid suggests directional slippage or excessive jitter. And / or, in the registered 3D model fusion view, the terminal uses different colors or icons to label the weighted residual of each first marker point. If any of these exist... Exceeding the threshold ,or Exceeding the threshold If the uncertainty ellipsoid of a first marker point is larger than the preset size, or its corresponding weighted residual is marked in red, the terminal will proactively pop up a prompt window containing visual feedback information, asking the user whether to resample the currently abnormal first marker point. After the user confirms resampling, the terminal only resamples the abnormal first marker point, obtaining a new set of marker points consisting of second marker points, and performs the same registration process as described in the above embodiment, avoiding overall rework and improving clinical operability and stability.

[0148] In this embodiment, the sampling uncertainty and registration consistency of each marker point are displayed through visual feedback, enabling users to quickly locate outlier points and selectively resample them, avoiding overall registration rework and thus significantly improving registration efficiency.

[0149] In one exemplary embodiment, a soft-marker-weighted hepatobiliary navigation registration system is provided, wherein the system includes a positioning acquisition module and a registration calculation module, wherein:

[0150] The registration calculation module is used to obtain each first marker point and its corresponding marker point set in the 3D model coordinate system. The marker point set includes multiple second marker points, which are multiple marker points collected from deformed organs and tissues in the physical coordinate system for each single first marker point. For each first marker point, the module performs marker point prediction based on the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point. Based on a preset weighted registration optimization strategy, each first marker point, and the soft marker points corresponding to each first marker point, the module performs registration calculations between the 3D model coordinate system and the physical coordinate system to obtain the registration result. The registration result is used for hepatobiliary navigation.

[0151] The positioning and acquisition module is used to acquire the second marker points corresponding to each first marker point in organs and tissues that are deformed in the physical coordinate system.

[0152] In a specific embodiment, such as Figure 4 As shown, the positioning and acquisition module can specifically be a preoperative positioning and acquisition module, and the registration calculation module can specifically be a covariance-weighted registration module. The soft marker-weighted hepatobiliary navigation registration system also includes a preoperative image reconstruction module, a model marker selection module, a pen-shaking sampling module, a soft marker modeling module, a sampling quality evaluation module, a navigation display and quality feedback module, and a selective resampling feedback module.

[0153] The preoperative image reconstruction module is used to reconstruct a three-dimensional model of the liver and gallbladder based on preoperative CT, MRI, or MRCP images. The model landmark selection module is used to select the first landmark in the three-dimensional liver and gallbladder model. For example, the first landmark can be a user-defined landmark, bifurcation, or structural reference point. The pen-shaking sampling module is used to perform local short-term multi-point acquisition to obtain the second landmark acquired for the first landmark and form a landmark set. The soft landmark modeling module is used to use the spatial mean of the second landmark as a representative position based on the spatial distribution of the second landmark. The sampling quality evaluation module is used to evaluate the sampling stability of the second landmark based on the covariance matrix of the second landmark as confidence information and to model the soft landmark (soft landmark) in combination with the representative position. The navigation display and quality feedback module is used to display the three-dimensional liver and gallbladder model and the instrument in three dimensions based on the registration results and to provide feedback on the evaluation results of the sampling of each first landmark. The selective resampling feedback module is used to locate abnormal landmarks (abnormal first landmarks) according to the evaluation results and return the sampling abnormality to the pen-shaking sampling module, instructing the pen-shaking sampling module to resample for the abnormal landmark.

[0154] In one exemplary embodiment, such as Figure 5 As shown, an example of a soft-marker-weighted hepatobiliary navigation registration method is provided, wherein:

[0155] Step 501: Acquire 3D image data.

[0156] Step 502: Construct a three-dimensional model based on the three-dimensional image, and select the first landmark point in the three-dimensional model.

[0157] Step 503: Based on the pen-shaking sampling control sensor probe, perform short-term multiple samplings in the actual liver and gallbladder tissue to obtain the second marker point.

[0158] Step 504: Calculate the spatial mean of the second marker point to obtain the representative position, and calculate the covariance matrix of the second marker point to obtain the confidence result. Model the soft marker point corresponding to the first marker point based on the representative position and confidence.

[0159] Step 505: Perform covariance weighted registration calculation based on each first marker point and soft marker point.

[0160] Step 506: Display navigation based on registration results, and calculate residual index based on registration weight and / or registration results to obtain and display the visual feedback information corresponding to each first marker point.

[0161] Step 507: Determine whether the residual index meets the preset quality requirements. If the residual index does not meet the preset quality requirements, proceed to step 508; if the residual index meets the preset quality requirements, proceed to step 509.

[0162] Step 508: Resample the first marker point that does not meet the quality requirements to obtain a new second marker point, and return to step 504 to model the new soft marker point.

[0163] Step 509: Based on the current registration result, continue to display navigation information. The navigation display is used to assist the user in performing the surgery.

[0164] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0165] Based on the same inventive concept, this application also provides a soft-marker-weighted hepatobiliary navigation registration device for implementing the soft-marker-weighted hepatobiliary navigation registration method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more soft-marker-weighted hepatobiliary navigation registration device embodiments provided below can be found in the limitations of the soft-marker-weighted hepatobiliary navigation registration method described above, and will not be repeated here.

[0166] In one exemplary embodiment, such as Figure 6 As shown, a soft-marker-weighted hepatobiliary navigation registration device 600 is provided, comprising: an acquisition module 601, a prediction module 602, and a registration module, wherein:

[0167] The acquisition module 601 is used to acquire each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0168] Prediction module 602 is used to predict the first marker point for each first marker point according to the marker point prediction strategy and each second marker point, and determine the soft marker point corresponding to the first marker point.

[0169] The registration module 603 is used to perform registration calculations between the three-dimensional model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each first marker point and the soft marker point corresponding to each first marker point, and to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0170] In one embodiment, the prediction module 602 is specifically used to predict the central trend position of each second marker point according to the marker point prediction strategy and each second marker point for each first marker point, and determine the central trend position as a soft marker point.

[0171] In one embodiment, the prediction module 602 is specifically used to calculate the spatial mean of the second marker corresponding to each first marker to obtain the central trend position of the marker set.

[0172] In one embodiment, the registration result is the rigid body transformation parameters from the three-dimensional model coordinate system to the physical coordinate system; the registration module 603 is specifically used to evaluate each first marker point according to the preset evaluation strategy and each second marker point to obtain the confidence result corresponding to each first marker point.

[0173] Based on the preset weighted registration optimization strategy and the confidence results of each first marker point, the registration weight corresponding to each soft marker point is determined;

[0174] The spatial deviations between each first marker point and the soft marker point are weighted according to the registration weights, and the rigid body transformation parameters from the 3D model coordinate system to the physical coordinate system are determined based on the weighting results and the preset optimization target.

[0175] In one embodiment, the registration module 603 is specifically used to determine the distribution dispersion of the second marker points according to a preset evaluation strategy and each second marker point, and to obtain the confidence result corresponding to the first marker point based on the distribution dispersion.

[0176] In one embodiment, the device 600 further includes:

[0177] The evaluation module is used to determine the evaluation index of rigid body transformation parameters based on the registration weights and / or registration results.

[0178] The generation module is used to generate visual feedback information based on evaluation indicators and to display the visual feedback information.

[0179] Each module in the aforementioned soft-marker-weighted hepatobiliary navigation registration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, so that the processor can call and execute the corresponding operations of each module.

[0180] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a soft-marker-weighted hepatobiliary navigation registration method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0181] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0182] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0183] Obtain each first marker point in the 3D model coordinate system and the corresponding set of marker points for each first marker point; the set of marker points includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point;

[0184] For each first marker point, the marker point prediction strategy and each second marker point are used to predict the marker point of the first marker point and determine the soft marker point corresponding to the first marker point.

[0185] Based on a preset weighted registration optimization strategy, each first marker point and its corresponding soft marker point, the registration calculation between the 3D model coordinate system and the physical coordinate system is performed to obtain the registration result; the registration result is used for hepatobiliary navigation.

[0186] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0187] For each first marker point, the central trend position of each second marker point is predicted according to the marker point prediction strategy and each second marker point, and the central trend position is determined as a soft marker point.

[0188] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0189] For each first marker point, the spatial mean of the corresponding second marker point is calculated to obtain the central trend position of the marker point set.

[0190] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0191] The first marker is evaluated according to the preset evaluation strategy and each second marker, and the confidence result corresponding to each first marker is obtained.

[0192] Based on the preset weighted registration optimization strategy and the confidence results of each first marker point, the registration weight corresponding to each soft marker point is determined;

[0193] The spatial deviations between each first marker point and the soft marker point are weighted according to the registration weights, and the rigid body transformation parameters from the 3D model coordinate system to the physical coordinate system are determined based on the weighting results and the preset optimization target.

[0194] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0195] The distribution dispersion of the second marker points is determined based on the preset evaluation strategy and each second marker point, and the confidence level result corresponding to the first marker point is obtained based on the distribution dispersion.

[0196] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0197] Based on the registration weights and / or registration results, determine the evaluation indexes for rigid body transformation parameters;

[0198] Visual feedback information is generated based on evaluation metrics and then displayed.

[0199] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0200] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0201] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0202] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0203] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0204] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A soft-marker-weighted hepatobiliary navigation registration method, characterized in that, The method includes: Obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point; For each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point. Based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, the registration calculation is performed between the three-dimensional model coordinate system and the physical coordinate system to obtain the registration result; the registration result is used for hepatobiliary navigation.

2. The method according to claim 1, characterized in that, For each first marker point, the step of predicting the marker point based on the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point includes: For each first marker point, the central trend position of each second marker point is predicted according to the marker point prediction strategy and each second marker point, and the central trend position is determined as a soft marker point.

3. The method according to claim 2, characterized in that, For each of the first marker points, predicting the central trend position of each of the second marker points according to the marker point prediction strategy and each of the second marker points includes: For each of the first marker points, the spatial mean of the corresponding second marker points is calculated to obtain the central trend position of the set of marker points.

4. The method according to claim 1, characterized in that, The registration result is the rigid body transformation parameters from the 3D model coordinate system to the physical coordinate system; the registration calculation between the 3D model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each first marker point, and the soft marker points corresponding to each first marker point, to obtain the registration result, includes: Each first marker is evaluated according to a preset evaluation strategy and each second marker, and the confidence result corresponding to each first marker is obtained. Based on the preset weighted registration optimization strategy and the confidence results of each of the first marker points, the registration weights corresponding to each of the soft marker points are determined. The spatial deviations between each of the first marker points and the soft marker points are weighted according to the registration weights, and the rigid body transformation parameters from the three-dimensional model coordinate system to the physical coordinate system are determined based on the weighting results and the preset optimization target.

5. The method according to claim 4, characterized in that, The step of evaluating each first marker point according to a preset evaluation strategy and each second marker point to obtain the confidence result corresponding to each first marker point includes: The distribution dispersion of the second marker points is determined according to the preset evaluation strategy and each second marker point, and the confidence result corresponding to the first marker point is obtained according to the distribution dispersion.

6. The method according to claim 1, characterized in that, After performing registration calculations on the 3D model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each first marker point, and the soft marker points corresponding to each first marker point, and obtaining the registration result, the method further includes: The evaluation index of the rigid body transformation parameters is determined based on the registration weight and / or the registration result. Visual feedback information is generated based on the evaluation indicators and then displayed.

7. A soft-marker-weighted hepatobiliary navigation registration system, characterized in that, The system includes a positioning and acquisition module and a registration and calculation module, wherein: The registration calculation module is used to obtain each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the deformed organ tissue in the physical coordinate system for a single first marker point; for each first marker point, the first marker point is predicted according to the marker point prediction strategy and each second marker point to determine the soft marker point corresponding to the first marker point; based on the preset weighted registration optimization strategy, each first marker point and the soft marker point corresponding to each first marker point, the three-dimensional model coordinate system and the physical coordinate system are registered to obtain the registration result; the registration result is used for hepatobiliary navigation. The positioning and acquisition module is used to acquire the second marker points corresponding to each of the first marker points in the deformed organ tissue under the physical coordinate system.

8. A soft-marker-weighted hepatobiliary navigation registration device, characterized in that, The device includes: The acquisition module is used to acquire each first marker point in the three-dimensional model coordinate system and the marker point set corresponding to each first marker point; the marker point set includes multiple second marker points, which are multiple marker points collected in the organ tissue with deformation in the physical coordinate system for a single first marker point; The prediction module is used to predict the first marker point for each first marker point according to the marker point prediction strategy and each second marker point, and determine the soft marker point corresponding to the first marker point. The registration module is used to perform registration calculations between the three-dimensional model coordinate system and the physical coordinate system based on a preset weighted registration optimization strategy, each of the first marker points and the soft marker points corresponding to each of the first marker points, to obtain the registration result; the registration result is used for hepatobiliary navigation.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images