2d / 3d registration method for laparoscopic liver surgery navigation based on a PSO-softposit combined algorithm

Abstract

The application discloses a laparoscopic liver surgery navigation 2D / 3D registration method based on a PSO-SoftPOSIT combined algorithm, the method first obtains a global optimal pose through PSO algorithm iteration as an initial pose of the SoftPOSIT algorithm, then further registers the initial pose through the SoftPOSIT algorithm, and re-adds the obtained pose to the PSO algorithm, through iteration, finally the method not only solves the problem that current surgery navigation registration related algorithms can only use expensive binocular endoscopes, but also reduces the requirement of the SoftPOSIT method on the initial pose, and higher registration accuracy and better registration effect are achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of surgical navigation, and particularly relates to a laparoscopic liver surgery navigation 2D / 3D registration method based on a PSO-SoftPOSIT combined algorithm. BACKGROUND

[0002] Minimally invasive surgery has become a hot spot in the field of biomedical engineering in recent years due to its advantages of smaller incision, less bleeding, and lower risk of postoperative complications. Unlike traditional open surgery, doctors cannot directly observe the patient's internal situation during minimally invasive surgery, and can only rely on the limited field of view of the video transmitted by the endoscope on the monitor to perform surgery. Therefore, the endoscope plays a crucial role in surgical navigation as the "eyes".

[0003] However, during actual operation, the following problems still exist in surgical navigation through an endoscope:

[0004] 1. Due to the small incision of minimally invasive surgery and the narrow and long shape of surgical instruments, the position of the surgical instrument in contact with the body tissue is far from the handle operated by the doctor. In addition, the limited field of view of the endoscope video affects the doctor's operation. During the execution of the surgical operation, the doctor will have weak depth perception and narrow field of view, which increases the learning cost of the surgeon.

[0005] 2. Due to the complex arrangement of blood vessels and tissues in the human body, for surgeries involving delicate operations, in addition to the endoscope image information, multi-modal information from preoperative CT, magnetic resonance and other equipment is often needed as an aid during the surgery, which has higher requirements for the endoscope image.

[0006] As a key technology in the field of surgical navigation, 2D / 3D registration technology can register the real-time image data of the endoscope with the three-dimensional simulated point cloud data reproduced by the CT data, realize multi-modal data fusion display during the surgery, achieve augmented reality display effect, and provide reference for the doctor's operation, thereby solving the above problems.

[0007] Currently, the related methods for surgical navigation registration generally require binocular endoscopes to achieve the method, and the registration objects are mainly binocular endoscope image restored point clouds and CT data reproduced point clouds, that is, 3D-3D registration is achieved. However, binocular endoscopes have the disadvantages of low popularity, high price, and large data volume.

[0008] In the 2D / 3D registration method, the currently used SoftPOSIT method generally has high requirements for the initial value, which requires an accurate initial pose as the initial value for iteration. However, in reality, only the approximate range of the initial pose can be obtained.

[0009] Therefore, it is of great significance and value to deeply study the 2D / 3D registration method which only needs to know the initial pose range and can be realized by only using a monocular endoscope. SUMMARY

[0010] The purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a laparoscopic liver surgery navigation 2D / 3D registration method based on a PSO-SoftPOSIT combined algorithm, which not only solves the problem that the current surgery navigation registration related algorithm can only use an expensive binocular endoscope, but also reduces the initial pose requirement of the SoftPOSIT method, realizes higher registration accuracy, and better registration effect. That is, the present application can realize 2D / 3D registration under a monocular endoscope, and realize higher registration accuracy.

[0011] The present application adopts the following technical solutions:

[0012] A laparoscopic liver surgery navigation 2D / 3D registration method based on a PSO-SoftPOSIT combined algorithm, comprising the following steps:

[0013] (1) obtaining the preoperative CT image of the patient;

[0014] (2) constructing a preoperative 3D model of the patient's liver according to the CT image obtained in step (1);

[0015] (3) marking the contour landmarks in the preoperative 3D model of the patient's liver obtained in step (2); the contour landmarks are the upper curved boundary of the liver, the lower curved boundary of the liver and the falciform ligament;

[0016] (4) calibrating the monocular endoscope to obtain the intrinsic parameters of the monocular endoscope;

[0017] (5) obtaining the intraoperative endoscopic image of the patient using the monocular endoscope;

[0018] (6) marking the pixel region corresponding to the contour landmarks in the endoscopic image in step (3);

[0019] (7) obtaining the pose of the monocular endoscope relative to the preoperative 3D model of the patient's liver using the PSO-SoftPOSIT combined algorithm; and (8) performing 2D / 3D registration on the preoperative 3D model of the patient's liver based on the pose obtained in step (7).

[0020] In the above technical solution, further, the preoperative 3D model of the patient's organ after registration is virtually fused with the intraoperative endoscopic image of the patient to obtain a virtual-real fusion image.

[0021] Further, in step (3):

[0022] Please have experienced doctors manually mark the contour landmarks in the patient's preoperative 3D model of the liver using open source software, obtain the three-dimensional point coordinates corresponding to the contour landmarks, and confirm by multiple doctors.

[0023] Further, the step (6) includes the following steps:

[0024] Please have experienced doctors manually mark the pixel area corresponding to the contour landmarks of the patient's preoperative 3D model of the liver in the endoscopic image using open source software, and use the picture containing only the pixel area corresponding to the contour landmarks as the real picture, and confirm by multiple doctors.

[0025] Further, the step (7) includes the following steps:

[0026] (7-1) Set multiple virtual cameras in the space where the patient's preoperative 3D model of the liver is located to form a virtual camera group, and initialize the parameters of each virtual camera, including learning factors C1, C2, inertia weight γ, intrinsic parameters, pose and speed v;

[0027] (7-2) Map the three-dimensional point coordinates of the contour landmarks through the multiple virtual cameras respectively to obtain corresponding multiple two-dimensional pictures;

[0028] (7-3) Calculate the similarity between the multiple two-dimensional pictures obtained in step (7-2) and the real picture using the MI method (mutual information) respectively, and sort the multiple two-dimensional pictures according to the similarity, and take the virtual camera corresponding to the two-dimensional picture with the highest similarity as the global optimal virtual camera;

[0029] (7-4) Update the camera movement speed (including size and direction), and move the virtual camera model.

[0030] (7-5) According to the two-dimensional picture and pose corresponding to the global optimal virtual camera, use the SoftPOSIT algorithm to iteratively generate a new pose;

[0031] (7-6) Use the new pose to add a new virtual camera and replace the virtual camera model with the lowest similarity;

[0032] (7-7) Repeat steps (7-1)-(7-6) until the speed of the global optimal virtual camera is less than a threshold value, and use the pose corresponding to the global optimal virtual camera as the final pose.

[0033] Further, in step (7-1):

[0034] Initialize the learning factors C1, C2 and inertia weight γ of each virtual camera, specifically:

[0035] According to the normal distribution random value initialization;

[0036] The internal parameter, pose, and speed v of each virtual camera are initialized, specifically:

[0037] The monocular endoscope internal parameter obtained in step (4) is used to initialize the internal parameter of the virtual camera;

[0038] According to the prior knowledge of surgical operation, the pose value range of the virtual camera is determined;

[0039] The speed v of all virtual cameras is the same value.

[0040] Further, in the step (7-3):

[0041] The formula for calculating the similarity s by the MI method is:

[0042] s=MI(I d ,I i )=H(I d )+H(I i )-H(I d ,I i )

[0043] Where I d is the real picture, I i represents the two-dimensional picture obtained by the i-th virtual camera, and H represents the covariance.

[0044] Further, in the step (7-4):

[0045] The speed of the virtual camera movement is affected by the global optimal virtual camera model and the local optimal virtual camera model. The global optimal camera refers to the virtual camera model corresponding to the two-dimensional picture with the highest similarity, and the local optimal virtual camera model refers to the virtual camera with the highest similarity within a certain length radius (the specific radius size can be empirically valued) centered on the virtual camera model.

[0046] For the t-th iteration, the update formula of the virtual camera movement speed is:

[0047] v i (t+1)=γv i (t)+C1(r g (t)-r i (t))+C2(r l (t)-r i (t))

[0048] Where v i (t) is the speed of the i-th virtual camera at the t-th iteration, and v i(t+1) is the updated velocity, r g (t) is the global optimal virtual camera, r l (t) is the local optimal virtual camera, r i (t) is the six-element pose representation of the i-th virtual camera at the t-th iteration.

[0049] Moving the virtual camera means that the velocity v i (t+1) changes the camera pose matrix representation M i () of the virtual camera at the t-th iteration, and the change formula is:

[0050]

[0051] M i (t+1) is the matrix representation of the camera pose at the t+1-th iteration.

[0052] Further, the SoftPOSIT algorithm in step (7-5) is composed of SoftAssign algorithm and POSIT (Perpendicular Projection Scaling Iteration), and the specific implementation process includes the following steps:

[0053] (7-5-1) According to the pose corresponding to the global optimal virtual camera as the initial pose, an initial camera is constructed. According to the orthographic projection of the contour landmark, a maximum likelihood estimation model with 3D-2D correspondence relationship of the contour landmark is constructed by using the POSIT algorithm;

[0054] (7-5-2) The probability distribution of the 3D-2D correspondence relationship of the contour landmark is established by using the SoftAssign algorithm;

[0055] (7-5-3) According to the maximum likelihood estimation model, the probability distribution of the 3D-2D correspondence relationship is used to establish an expectation maximization registration model, and the singular value decomposition method is used to obtain the optimal solution of the expectation maximization registration model;

[0056] (7-5-4) According to the optimal solution obtained in step (7-5-3), the parameters of the expectation maximization registration model are updated to obtain a new camera pose, and the new camera pose is used as the initial pose in the iteration process of (7-5-1)-(7-5-3). When the composite determination condition is met, the iteration is ended, and a new pose is obtained.

[0057] Further, the step (7-5-4) includes:

[0058] The composite determination condition includes the following contents:

[0059] According to the new camera pose, a camera is constructed, and after each round of iteration, the three-dimensional points of the contour landmarks are mapped into corresponding two-dimensional pictures according to the camera; and a Halcon algorithm is used to calculate the matching degree between the two-dimensional pictures and the real pictures, and the matching degree needs to be less than a matching degree threshold value;

[0060] After each round of iteration, the change amount of the new camera pose compared with the pose of the last round of iteration needs to be less than a change amount threshold value.

[0061] The PSO and the SoftPOSIT algorithm are fused in the present application, the initial pose is provided through the PSO algorithm, so that the influence of the randomly guessed initial pose on the SoftPOSIT algorithm is reduced, and the registration effect is improved.

[0062] The present application provides a 2D / 3D registration process based on a monocular endoscope, provides a new solution for the spatial registration technology in surgical navigation, realizes better registration effect by using a cheaper and more popular monocular endoscope, and has a wider application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 It is a laparoscopic liver surgery navigation 2D / 3D registration method flowchart based on a PSO-SoftPOSIT combined algorithm.

[0064] Figure 2 It is a PSO-SoftPOSIT combined algorithm flowchart.

[0065] Figure 3 It is a SoftPOSIT algorithm flowchart. DETAILED DESCRIPTION

[0066] The technical solutions of the present application will be further described below with reference to the drawings, which are an explanation of the present application rather than a limitation.

[0067] Referring to Figures 1-3 The present embodiment provides a laparoscopic liver surgery navigation 2D / 3D registration method based on a PSO-SoftPOSIT combined algorithm, which mainly uses the global optimal pose obtained through the PSO algorithm iteration as the initial pose of the SoftPOSIT algorithm, further registers the initial pose through the SoftPOSIT algorithm, and re-joins the obtained pose into the PSO algorithm, and through iteration, finally registers the 2D image and the CT point cloud, as shown in Figure 1 The specific steps include the following steps:

[0068] (1) Obtain the preoperative CT image of the patient;

[0069] (2) Construct a preoperative 3D model of the patient's liver based on the CT image obtained in step (1);

[0070] (3) Mark the contour landmarks in the preoperative 3D model of the patient's liver obtained in step (2); the contour landmarks are the upper curved boundary of the liver, the lower curved boundary of the liver, and the falciform ligament;

[0071] (4) Calibrate the monocular endoscope to obtain the intrinsic parameters of the monocular endoscope;

[0072] (5) Obtain the intraoperative endoscopic image of the patient using the monocular endoscope;

[0073] (6) Mark the pixel region in the endoscopic image corresponding to the contour landmarks in step (3);

[0074] (7) Obtain the pose of the monocular endoscope relative to the preoperative 3D model of the patient's liver using the PSO-SoftPOSIT joint algorithm;

[0075] (8) Perform 2D / 3D registration on the preoperative 3D model of the patient's liver based on the pose obtained in step (7).

[0076] Fuse the preoperative 3D model of the patient's organ after registration with the intraoperative endoscopic image of the patient to obtain a virtual-real fusion image and display it.

[0077] It should be noted that the preoperative 3D model of the patient's liver in the present application is constructed using open source software based on the CT image, wherein the open source software refers to CT image 3D reconstruction software, such as MITK (Medical Image Interactive Toolkit). The preoperative 3D model of the patient's liver is saved in the form of a point cloud file (ply).

[0078] In this scheme, the contour landmarks in the preoperative 3D model of the patient's liver are marked using open source software, which can be manually marked by experienced doctors using Hepatug and other open source software, and the three-dimensional coordinates of the contour landmarks are saved. The contour landmarks are anatomically related features of the liver, including but not limited to the upper curved boundary of the liver, the lower curved boundary of the liver, the falciform ligament, etc., and need to be confirmed by multiple doctors after labeling.

[0079] In this scheme, the intraoperative endoscopic image of the patient is obtained using the monocular endoscope, and the endoscopic image is obtained by intercepting the endoscopic video frame. And the endoscopic image needs to contain obvious contour landmark features.

[0080] In the scheme, the image processing software is used to mark the pixel region corresponding to the contour landmark in the endoscope image, which can be manually marked by an experienced doctor using image processing software such as Photoshop, and after marking the contour landmark, the pixel coordinates of the contour landmark are extracted, and a picture containing only the contour landmark is generated and saved.

[0081] In the scheme, the pose of the camera is obtained by using the PSO-SoftPOSIT joint algorithm, and the PSO-SoftPOSIT algorithm is as shown in the following formula: Figure 2 The specific steps include the following steps:

[0082] A plurality of virtual cameras are arranged in the space where the preoperative 3D model of the patient's liver is located to form a virtual camera group, and the parameters of each virtual camera model are initialized, including learning factors C1, C2, inertia weight γ, intrinsic parameters, pose, and speed v.

[0083] The three-dimensional point coordinates of the contour landmark are mapped through the plurality of virtual cameras respectively to obtain a plurality of corresponding two-dimensional pictures.

[0084] The similarity between the plurality of two-dimensional pictures and the real picture is calculated by using the MI method respectively, and the plurality of two-dimensional pictures are sorted according to the similarity. The virtual camera corresponding to the two-dimensional picture with the highest similarity is taken as the global optimal virtual camera.

[0085] The moving speed of all virtual cameras is updated, and the virtual cameras are moved.

[0086] According to the two-dimensional picture and the pose corresponding to the global optimal virtual camera, a new pose is generated by using the SoftPOSIT algorithm.

[0087] The new virtual camera is added by using the new pose, and the virtual camera model with the lowest similarity is replaced.

[0088] The above steps are repeated until the speed of the global optimal virtual camera is less than a threshold value, and the pose corresponding to the global optimal virtual camera is taken as the final pose.

[0089] It should be noted that the virtual camera in the present application refers to that the intrinsic parameters of the monocular endoscope are used as the intrinsic parameters of the virtual camera, and a specific camera pose is used as the external parameter, and the displacement of the i-th virtual camera can be represented as The rotation can be represented as the rotation components in three directions The six-element pose expression of the pose is:

[0090]

[0091] And move in the CT model space with a speed v.

[0092] In the present scheme, a plurality of virtual cameras are arranged in the space where the preoperative 3D model of the patient's liver is located, to form a virtual camera group, and parameters of each virtual camera model are initialized, including learning factors C1 and C2, inertia weight γ, intrinsic parameters, pose, and speed v. Specifically, 500-1000 virtual cameras are arranged in the space where the preoperative 3D model of the patient's liver is located, to form a virtual camera group, and learning factors C1 and C2 and inertia weight γ of each virtual camera are initialized by randomly selecting values according to a normal distribution. The intrinsic parameters of the virtual camera are initialized by using the intrinsic parameters of the monocular endoscope; the pose range is obtained by preoperative prior knowledge, and the initialization pose of the virtual camera is uniformly selected within the pose range. The speed v of all virtual camera models is the same value, which is obtained by experience.

[0093] In the present scheme, the similarity s between a plurality of two-dimensional pictures and the real picture is calculated by using the MI method, and the two-dimensional pictures are sorted according to the similarity. The formula for judging the similarity by using the MI method is:

[0094] s=MI(I d ,I i )=H(I d )+H(I i )-H(I d ,I i )

[0095] where I d is the real picture, I i represents the two-dimensional picture obtained by the i-th virtual camera, and H represents the covariance.

[0096] In the present scheme, the camera moving speed is updated, and the virtual camera is moved, specifically:

[0097] The moving speed of the virtual camera is affected by the global optimal virtual camera and the local optimal virtual camera. The global optimal virtual camera refers to the virtual camera corresponding to the two-dimensional picture with the highest similarity in the virtual camera group, and the local optimal virtual camera refers to the virtual camera with the highest similarity within a certain length radius range centered on the virtual camera. Let r i be the six-element pose expression, and for the t-th iteration, the update formula of the moving speed of the virtual camera is:

[0098] v i (t+1)=γv i (t)+C1(r g (t)-r i(t)+C2(r l (t)-r i (t))

[0099] Among them, v i (t) represents the velocity of the i-th virtual camera in the t-th iteration, v i (t+1) represents the updated velocity, r g (t) represents the globally optimal virtual camera, r l (t) represents the locally optimal virtual camera.

[0100] It should be noted that, in this invention, moving the virtual camera model refers to using the speed v of the i-th virtual camera model in the (t+1)-th iteration. i (t+1) represents the camera pose matrix M of the virtual camera in the t-th iteration. i (t) is modified, and the formula is changed as follows:

[0101]

[0102] Among them, M i (t+1) is the matrix representation of the camera pose in the (t+1)th iteration.

[0103] In this scheme, the SoftPOSIT algorithm (e.g., based on the 2D image and pose corresponding to the globally optimal virtual camera) is used. Figure 3 (As shown) Iterative generation of a new pose specifically includes the following steps:

[0104] An initial camera is constructed using the pose corresponding to the globally optimal virtual camera as the initial pose; and a maximum likelihood estimation model POSIT with the 3D-2D correspondence of the contour landmarks is constructed based on the orthophoto projection of the contour landmarks.

[0105] The probability distribution of the 3D-2D correspondence between the outline landmarks is established using the SoftAssign algorithm;

[0106] Based on the maximum likelihood estimation model, using the probability distribution of the 3D-2D correspondence, an expectation-maximization registration model is established, and the optimal solution of the expectation-maximization registration model is obtained using the singular value decomposition method.

[0107] Based on the obtained optimal solution, the parameters of the expected maximum registration model are updated to obtain a new camera pose. The new camera pose is used as the initial pose in the above iterative process. When the composite judgment condition is met, the iteration ends and a new pose is obtained.

[0108] In this scheme, the conditions for determining the validity of a combination of criteria include the following:

[0109] According to the new camera pose, a camera is constructed, and after each round of iteration, the three-dimensional points of the contour landmarks are mapped into corresponding two-dimensional pictures according to the camera. A Halcon algorithm is used to calculate the matching degree between the two-dimensional pictures and the real pictures, and the matching degree is less than 10.

[0110] After each round of iteration, the change of the new camera pose compared to the pose of the last round of iteration is less than 0.01.

[0111] In the scheme, the new virtual camera is added using the new pose, and the virtual camera with the lowest similarity is eliminated. The process of adding the new virtual camera pose can include: adding a new virtual camera model, its intrinsic parameters, C1, C2 and inertial weight, and the speed v is the same as the global optimal virtual camera, the pose is initialized with the new pose, and finally it is added to the virtual camera group; the virtual camera with the lowest similarity is deleted from the virtual camera group.

Claims

1. A 2D / 3D registration method for laparoscopic liver surgery navigation based on a PSO-SoftPOSIT combined algorithm, characterized in that: It comprises the following steps: (1) obtaining preoperative CT images of a patient; (2) constructing a preoperative 3D model of the liver of the patient according to the CT images obtained in step (1); (3) marking contour landmarks in the preoperative 3D model of the liver of the patient obtained in step (2); the contour landmarks are the upper curved boundary, the lower curved boundary and the falciform ligament of the liver; (4) calibrating a monocular endoscope to obtain intrinsic parameters of the monocular endoscope; (5) obtaining an intraoperative endoscopic image of the patient using the monocular endoscope; (6) marking a pixel region in the endoscopic image corresponding to the contour landmarks in step (3); (7) obtaining the pose of the monocular endoscope relative to the preoperative 3D model of the liver of the patient using a PSO-SoftPOSIT joint algorithm; (8) performing 2D / 3D registration on the preoperative 3D model of the liver of the patient based on the pose obtained in step (7); The step (7) comprises the following steps: (7-1) setting a plurality of virtual cameras in the space where the preoperative 3D model of the liver of the patient is located to form a virtual camera group, initializing the parameters of each virtual camera, including learning factors C1, C2, inertia weight γ, intrinsic parameters, pose and velocity v; (7-2) mapping the three-dimensional point coordinates of the contour landmarks through the plurality of virtual cameras to obtain a plurality of corresponding two-dimensional pictures; (7-3) calculating the similarity between the plurality of two-dimensional pictures obtained in step (7-2) and the true picture respectively, and sorting the plurality of two-dimensional pictures according to the similarity, and taking the virtual camera corresponding to the two-dimensional picture with the highest similarity as the global optimal virtual camera; (7-4) updating the moving speed of all virtual cameras and moving the virtual cameras; (7-5) using the SoftPOSIT algorithm to iteratively generate a new pose according to the two-dimensional picture and the pose corresponding to the global optimal virtual camera; (7-6) using the new pose to add a new virtual camera and replace the virtual camera model with the lowest similarity; (7-7) repeating steps (7-1)-(7-6) until the speed of the global optimal virtual camera is less than a threshold value, and taking the pose corresponding to the global optimal virtual camera as the final pose; The step (7-5) comprises the following steps: (7-5-1) constructing an initial camera according to the pose corresponding to the global optimal virtual camera as an initial pose, and constructing a maximum likelihood estimation model with 3D-2D correspondence relationship of the contour landmarks using POSIT algorithm according to the orthographic projection of the contour landmarks; (7-5-2) using SoftAssign algorithm to establish the probability distribution of the 3D-2D correspondence relationship of the contour landmarks; (7-5-3) establishing an expectation maximization registration model using the probability distribution of the 3D-2D correspondence relationship according to the maximum likelihood estimation model, and obtaining the optimal solution of the expectation maximization registration model using singular value decomposition method; (7-5-4) According to the optimal solution obtained in step (7-5-3), the parameters of the expectation maximization registration model are updated, a new camera pose is obtained, the new camera pose is taken as the initial pose in the iteration process of (7-5-1)-(7-5-3), and the iteration is ended when the composite determination condition is met, so as to obtain a new pose.

2. The PSO-SoftPOSIT combined algorithm based 2D / 3D registration method for laparoscopic liver surgery navigation according to claim 1, wherein, In the step (7-1): The learning factors C1 and C2 and the inertia weight γ of each virtual camera are initialized, specifically, the values are randomly selected according to a normal distribution for initialization; The intrinsic parameters, pose and speed v of each virtual camera are initialized, specifically: The virtual camera is initialized by using the monocular endoscope intrinsic parameters obtained in step (4); According to the prior knowledge of surgical operation, the value range of the pose of the virtual camera is determined; The speed v of all virtual cameras is taken as the same value.

3. The PSO-SoftPOSIT combined algorithm based 2D / 3D registration method for laparoscopic liver surgery navigation according to claim 1, wherein, In the step (7-3): The formula for calculating the similarity s by using the MI method is: s = M I (I d , i ) s = M I (I d , i ) s = M I (I d , i ) where I d is the real picture, I i denotes the 2D picture obtained by the i-th virtual camera, and H denotes the covariance.

4. The PSO-SoftPOSIT combined algorithm based 2D / 3D registration method for laparoscopic liver surgery navigation according to claim 1, wherein, In the step (7-4): The update formula of the virtual camera moving speed is: v i (t+1) = γv i (t) + C1(r g (t) - r i (t)) + C2(r l (t) - r i (t)) wherein v i (t) is the velocity of the i-th virtual camera at the t-th iteration, v i (t+1) is the updated velocity, r g (t) is the global optimal virtual camera, r l (t) is the local optimal virtual camera, r i (t) is the i-th virtual camera at the t-th iteration in the six-element pose representation; the local optimal virtual camera refers to the virtual camera with the highest similarity within a certain length as the radius range centered on the virtual camera. Moving the virtual camera means changing the position of the virtual camera at iteration t+1 using the velocity v i (t+1) of the virtual camera at iteration t using the following formula: i (t) M i (t+1) is the matrix representation of the camera pose of the t+1th iteration.

5. The PSO-SoftPOSIT combined algorithm based 2D / 3D registration method for laparoscopic liver surgery navigation according to claim 1, wherein, In the step (7-5-4): The composite determination condition includes the following contents: According to the new camera pose, a camera is constructed, and after each round of iteration, the three-dimensional points of the contour landmarks are mapped into corresponding two-dimensional pictures according to the camera; the matching degree between the two-dimensional pictures and the real pictures is calculated by using the Halcon algorithm, and the matching degree needs to be less than the matching degree threshold; After each round of iteration, the change amount of the new camera pose compared with the pose of the last round of iteration needs to be less than the change amount threshold.

Images