Method for registering three-dimensional model with physical object, computer device and storage medium

Abstract

The application is suitable for the field of medical imaging technology, and provides a registration method of a three-dimensional model and a solid object, a computer device and a storage medium. The method calculates an initial registration matrix between a target solid object and a three-dimensional model thereof, and projects the three-dimensional model, a preset landmark on the three-dimensional model, an intraoperative landmark on the target solid object and an intraoperative point cloud onto a preset plane based on the initial registration matrix. Therefore, a user can drag a first projection point corresponding to the preset landmark left, right, up and down in a two-dimensional projection image to update the display position of the first projection point in the two-dimensional projection image, and then accurately correct the initial registration matrix. The three-dimensional model and the intraoperative point cloud are registered based on a target registration matrix obtained through correction, and the display position of the projection point in the two-dimensional projection image is updated, so that accurate registration of the target solid object and the three-dimensional model can be achieved. The method reduces the difficulty of landmark registration correction and improves the registration accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of medical imaging technology, and particularly relates to a registration method of a three-dimensional model and a physical object, a computer device and a storage medium. BACKGROUND

[0002] In computer assisted surgery (CAS) or robot assisted surgery (RAS), in order to accurately reproduce the preoperative surgical plan based on medical imaging data in the patient's body, it is necessary to first register the three-dimensional model of the actual anatomical structure with the actual anatomical structure. The registration process between the three-dimensional model and the actual anatomical structure usually includes landmark registration and surface registration. Landmark registration is usually based on each preset landmark on the three-dimensional model and each intraoperative landmark on the actual anatomical structure to solve the initial rigid transformation between the three-dimensional model and the actual anatomical structure. Surface registration is to realize accurate registration between intraoperative point clouds on the surface of the three-dimensional model and the actual anatomical structure based on the initial rigid transformation.

[0003] In related technologies, the intraoperative landmark on the actual anatomical structure is usually manually marked by a doctor. However, in minimally invasive surgery with a small incision, the doctor's field of view is limited, and it is difficult to accurately find and mark the intraoperative landmark corresponding to the preset landmark, which leads to the introduction of rotation deviation in landmark registration, thereby reducing the accuracy of subsequent surface registration, and even causing registration failure. Although related technologies can provide an adjustment interface for the initial rigid transformation, the operation is difficult, which causes the doctor to be unable to timely correct the registration error caused by landmark registration. SUMMARY

[0004] Therefore, the embodiments of the present application provide a registration method of a three-dimensional model and a physical object, a computer device and a storage medium to solve the technical problem that the traditional registration method of a physical object and its three-dimensional model cannot timely correct the registration error caused by the landmark registration process, which easily leads to registration failure.

[0005] In a first aspect, the embodiments of the present application provide a registration method of a three-dimensional model and a physical object, comprising:

[0006] obtaining a three-dimensional model of a target physical object, three-dimensional coordinates of each preset landmark on the three-dimensional model, three-dimensional coordinates of each intraoperative landmark on the target physical object corresponding to each preset landmark, and three-dimensional coordinates of an intraoperative point cloud on the target physical object;

[0007] calculating an initial registration matrix between the three-dimensional model and the target physical object according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks;

[0008] projecting the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters to generate and display a corresponding two-dimensional projection image; the two-dimensional projection image including a projection contour of the three-dimensional model, each first projection point corresponding to each preset landmark, each second projection point corresponding to each intraoperative landmark, and a plurality of third projection points corresponding to the intraoperative point cloud;

[0009] in response to a drag operation on the first projection point, updating a display position of the first projection point in the two-dimensional projection image in real time, updating a three-dimensional coordinate of the preset landmark corresponding to the first projection point according to the preset projection parameters and the updated display position of the first projection point, and calculating a target registration matrix between the three-dimensional model and the target entity object according to the updated three-dimensional coordinates of all the preset landmarks and three-dimensional coordinates of all the intraoperative landmarks;

[0010] registering the three-dimensional model and the intraoperative point cloud based on the target registration matrix, and updating display positions of all the projection points in the two-dimensional projection image.

[0011] In an optional implementation of the first aspect, the initial registration matrix between the three-dimensional model and the target entity object is calculated according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks, including:

[0012] determining a first centroid of all the preset landmarks according to the three-dimensional coordinates of all the preset landmarks, and respectively performing decentralization processing on each preset landmark based on the first centroid;

[0013] determining a second centroid of all the intraoperative landmarks according to the three-dimensional coordinates of all the intraoperative landmarks, and respectively performing decentralization processing on each intraoperative landmark based on the second centroid;

[0014] calculating a first optimal rotation matrix between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of all the preset landmarks after the decentralization processing and the three-dimensional coordinates of all the intraoperative landmarks;

[0015] calculating a first translation vector between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of the first centroid, the three-dimensional coordinates of the second centroid and the first optimal rotation matrix;

[0016] constructing the first optimal rotation matrix and the first translation vector into the initial registration matrix.

[0017] In an optional implementation of the first aspect, projecting the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters comprises:

[0018] Transforming the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud into a coordinate system in which the three-dimensional model is located through an inverse matrix of the initial registration matrix to obtain the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation;

[0019] Projecting the three-dimensional model, the preset landmarks, the intraoperative landmarks after coordinate transformation and the intraoperative point cloud after coordinate transformation onto a preset plane based on preset projection parameters.

[0020] In an optional implementation of the first aspect, the first projection point, the second projection point and the third projection point in the two-dimensional projection image are semitransparent dots of different colors respectively.

[0021] In an optional implementation of the first aspect, the target registration matrix between the three-dimensional model and the target entity object is calculated according to the three-dimensional coordinates of all the preset landmarks after update and the three-dimensional coordinates of all the intraoperative landmarks, comprising:

[0022] A third centroid of all the preset landmarks is determined according to the three-dimensional coordinates of all the preset landmarks after update, and the preset landmarks after update are respectively subjected to decentralization processing based on the third centroid;

[0023] A second centroid of all the intraoperative landmarks is determined according to the three-dimensional coordinates of all the intraoperative landmarks, and each of the intraoperative landmarks is subjected to decentralization processing based on the second centroid;

[0024] A second optimal rotation matrix between all the preset landmarks after update and all the intraoperative landmarks is calculated according to the three-dimensional coordinates of all the preset landmarks after update and the three-dimensional coordinates of all the intraoperative landmarks after decentralization processing;

[0025] A second translation vector between all the preset landmarks and all the intraoperative landmarks is calculated according to the three-dimensional coordinates of the third centroid, the three-dimensional coordinates of the second centroid and the optimal rotation matrix;

[0026] The second optimal rotation matrix and the second translation vector are constructed into a target registration matrix.

[0027] In an optional implementation of the first aspect, the target registration matrix is taken as a rigid transformation initial value of an iterative closest point algorithm, and the three-dimensional model and the intraoperative point cloud are registered by the iterative closest point algorithm to obtain a final registration matrix and a corresponding registration quality parameter;

[0028] If the registration quality parameter meets a preset requirement, display positions of the first projection point, the second projection point and the third projection point in the two-dimensional projection image are updated based on the final registration matrix.

[0029] In a second aspect, the embodiments of the present application provide a computer device, comprising:

[0030] A first obtaining unit is configured to obtain a three-dimensional model of a target entity object, three-dimensional coordinates of each preset landmark on the three-dimensional model, three-dimensional coordinates of each intraoperative landmark corresponding to each preset landmark on the target entity object, and three-dimensional coordinates of an intraoperative point cloud on the target entity object.

[0031] A first calculating unit is configured to calculate an initial registration matrix between the three-dimensional model and the target entity object according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks.

[0032] A virtual projecting unit is configured to project the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters, to generate and display a corresponding two-dimensional projection image; the two-dimensional projection image includes a projection contour of the three-dimensional model, each first projection point corresponding to each preset landmark, each second projection point corresponding to each intraoperative landmark, and a plurality of third projection points corresponding to the intraoperative point cloud.

[0033] A first processing unit is configured to update a display position of the first projection point in the two-dimensional projection image in real time in response to a drag operation on the first projection point, update the three-dimensional coordinates of the preset landmark corresponding to the first projection point according to the preset projection parameters and the updated display position of the first projection point, and calculate a target registration matrix between the three-dimensional model and the target entity object according to the updated three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks.

[0034] A second processing unit is configured to register the three-dimensional model and the intraoperative point cloud based on the target registration matrix, and update the display positions of all the projection points in the two-dimensional projection image.

[0035] In a third aspect, an embodiment of the present application provides another computer device, including a memory and a computer program stored in the memory and capable of running on a processor, and the processor implements the method according to any optional implementation manner of the first aspect when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the method according to any optional implementation manner of the first aspect.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a computer device, causes the computer device to implement the method according to any optional implementation manner of the first aspect.

[0038] The method for registering a three-dimensional model and a physical object, the computer device, the computer readable storage medium and the computer program product provided by the embodiments of the present application have the following beneficial effects:

[0039] The method for registering a three-dimensional model and a physical object provided by the embodiments of the present application calculates an initial registration matrix between a target physical object and its three-dimensional model, and projects the three-dimensional model, a preset landmark on the three-dimensional model, an intraoperative landmark on the target physical object and an intraoperative point cloud onto a preset plane based on the initial registration matrix, so that the user can update the display position of the first projection point corresponding to the preset landmark in the two-dimensional projection image by dragging the first projection point up, down, left and right in the two-dimensional projection image, and then correct the initial registration matrix accurately. The three-dimensional model and the intraoperative point cloud are registered based on the target registration matrix obtained by correction, and the display position of the projection point in the two-dimensional projection image is updated, so that the target physical object and the three-dimensional model can be accurately registered. That is, the method can reduce the difficulty of manual correction of the initial registration matrix, and improve the registration accuracy and efficiency of the target physical object and its three-dimensional model. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or related technical descriptions. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0041] Figure 1 A schematic flowchart of a method for registering a three-dimensional model and a physical object provided by an embodiment of the present application is shown in the figure.

[0042] Figure 2An implementation flowchart of S102 in a three-dimensional model and entity object registration method provided by an embodiment of the present application is shown in the figure;

[0043] Figure 3 A two-dimensional projection image provided by an embodiment of the present application is shown in the figure;

[0044] Figure 4 A dragging process of a first projection point in a two-dimensional projection image provided by an embodiment of the present application is shown in the figure;

[0045] Figure 5 An updating principle of an intraoperative landmark provided by an embodiment of the present application is shown in the figure;

[0046] Figure 6 An implementation flowchart of S104 in a three-dimensional model and entity object registration method provided by an embodiment of the present application is shown in the figure;

[0047] Figure 7 An updated two-dimensional projection image provided by an embodiment of the present application is shown in the figure;

[0048] Figure 8 A comparison of registration processes provided by known technologies and embodiments of the present application is shown in the figure;

[0049] Figure 9 A structure diagram of a computer device provided by an embodiment of the present application is shown in the figure;

[0050] Figure 10 A structure diagram of a computer device provided by another embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0051] The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and thus only serve as examples, but cannot be used to limit the protection scope of the present application.

[0052] In the description of the embodiments of the present application, the technical terms "comprise", "contain", "have" and any variants thereof mean "comprise but not limited to", unless otherwise specifically emphasized. In the description of the embodiments of the present application, unless otherwise specified, the technical term "multiple" refers to two or more than two, and the technical terms "at least one" and "one or more" refer to one, two or more than two. The technical terms "first", "second" and the like are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. The technical term "and / or" is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A existing alone, A and B existing together, and B existing alone. In addition, the character " / " in this paper generally represents that the front and rear associated objects have an "or" relationship.

[0053] The embodiments of the present application first provide a method for registering a three-dimensional model and a physical object. The method can be applied to a computer device. The computer device may, for example, include a mobile phone, a tablet computer, a notebook computer or a desktop computer, etc. The type of computer device is not limited by the embodiments of the present application.

[0054] Figure 1 A schematic flow chart of a method for registering a three-dimensional model and a physical object provided by the embodiments of the present application is shown in FIG. 1. As shown in the figure, the method can include S101-S105, which are described in detail as follows: Figure 1

[0055] S101, obtaining a three-dimensional model of a target physical object, three-dimensional coordinates of each preset landmark on the three-dimensional model, three-dimensional coordinates of each intraoperative landmark on the target physical object corresponding to each preset landmark respectively, and three-dimensional coordinates of an intraoperative point cloud on the target physical object.

[0056] The target physical object can refer to an actual anatomical site on a human body or other animal body that needs to be intervened by surgery, for example, a hip bone, a spinal vertebra or a femur of a human body, etc.

[0057] The three-dimensional model of the target physical object can be a virtual model obtained by three-dimensional reconstruction of medical image data of the target physical object. The medical image data can include magnetic resonance imaging (MRI) data or computed tomography (CT) data, etc.

[0058] ​The preset landmarks can be a series of feature points pre-labeled on the three-dimensional model before surgery. These feature points can be selected at positions with clear anatomical significance, such as the protrusions, recesses or edge intersections of bones, to ensure the uniqueness and recognizability of each preset landmark in space.

[0059] The intraoperative landmarks can be a series of feature points collected on the surface of the target entity object by a probe of the surgical navigation system during surgery, which are one-to-one corresponding to each preset landmark on the three-dimensional model.

[0060] The intraoperative point cloud can be a collection of a large number of surface points collected on the surface of the target entity object by a scanning device of the surgical navigation system during surgery. The intraoperative point cloud includes a number of surface points much larger than the intraoperative landmarks.

[0061] Optionally, the computer device can obtain the three-dimensional model of the target entity object and the three-dimensional coordinates of each preset landmark on the three-dimensional model from the local memory. Illustratively, the three-dimensional model can be represented by an isosurface. The isosurface is a three-dimensional mesh model representing the surface of the target entity object reconstructed from CT data based on a preset density threshold. The preset density threshold can be set according to actual conditions. The three-dimensional coordinates of the preset landmarks can be three-dimensional coordinates in the CT coordinate system corresponding to the CT device.

[0062] Optionally, the computer device can obtain the three-dimensional coordinates of each intraoperative landmark and the three-dimensional coordinates of the intraoperative point cloud from the surgical navigation system. Illustratively, the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud can be three-dimensional coordinates in the navigation coordinate system corresponding to the surgical navigation system.

[0063] S102, according to the three-dimensional coordinates of all preset landmarks and the three-dimensional coordinates of all intraoperative landmarks, calculate the initial registration matrix between the three-dimensional model and the target entity object.

[0064] The initial registration matrix can be an initial rigid transformation matrix for transforming the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud to the coordinate system of the three-dimensional model and the preset landmarks.

[0065] It should be understood that, since each intraoperative landmark collected during surgery can not accurately correspond to each preset landmark, the initial rigid transformation matrix can not be accurate, and a more accurate target rigid transformation matrix needs to be further calculated through manual correction subsequently.

[0066] Optionally, the computer device can calculate the initial registration matrix between the three-dimensional model and the target entity object through S1021-S1025 as shown in Figure 2

[0067] ​S1021, determine a first centroid of all preset landmarks according to the three-dimensional coordinates of all preset landmarks, and respectively perform decentralization processing on each preset landmark based on the first centroid.

[0068] The first centroid is a geometric center of all preset landmarks, and the three-dimensional coordinates of the first centroid in the CT coordinate system are x The axial component is a x average value of the axial components of all preset landmarks, y The axial component is a y average value of the axial components of all preset landmarks, z The axial component is a z average value of the axial components of all preset landmarks.

[0069] Optionally, the computer device can achieve the decentralization processing on each preset landmark by subtracting the three-dimensional coordinates of each preset landmark from the three-dimensional coordinates of the first centroid.

[0070] S1022, determine a second centroid of all intraoperative landmarks according to the three-dimensional coordinates of all intraoperative landmarks, and respectively perform decentralization processing on each intraoperative landmark based on the second centroid.

[0071] The second centroid is a geometric center of all intraoperative landmarks, and the three-dimensional coordinates of the second centroid in the navigation coordinate system are x The axial component is a x average value of the axial components of all intraoperative landmarks, y The axial component is a y average value of the axial components of all intraoperative landmarks, z The axial component is a z average value of the axial components of all intraoperative landmarks.

[0072] Optionally, the computer device can achieve the decentralization processing on each intraoperative landmark by subtracting the three-dimensional coordinates of each intraoperative landmark from the three-dimensional coordinates of the second centroid.

[0073] S1023, calculate a first optimal rotation matrix between all preset landmarks and all intraoperative landmarks according to the three-dimensional coordinates of all preset landmarks after the decentralization processing and the three-dimensional coordinates of all intraoperative landmarks.

[0074] The first optimal rotation matrix is a rotation matrix that can make the overall deviation between all preset landmarks after decentralization and all intraoperative landmarks after decentralization to be the smallest after rotation. The dimension of the first optimal rotation matrix can be 3x3. It should be noted that since the calculation process of the optimal rotation matrix of two three-dimensional coordinates in different coordinate systems is known technology, the calculation process of the first optimal rotation matrix can refer to the description in the related technology, and the embodiments of the present application will not be described in detail.

[0075] S1024, calculate a first translation vector between all preset landmarks and all intraoperative landmarks according to the three-dimensional coordinates of the first centroid, the three-dimensional coordinates of the second centroid, and the first optimal rotation matrix.

[0076] The first translation vector is used to define the displacement required to maximize the spatial coincidence of the rotated all preset landmarks and all intraoperative landmarks. The dimension of the first translation vector can be 3x1.

[0077] Specifically, the computer device can calculate the product of the first optimal rotation matrix and the three-dimensional coordinates of the first centroid, and subtract the three-dimensional coordinates of the second centroid from the product to obtain the first translation vector.

[0078] S1025, construct the first optimal rotation matrix and the first translation vector into an initial registration matrix.

[0079] The initial registration matrix is a 4x4 dimensional standard homogeneous coordinate transformation matrix.

[0080] Specifically, the computer device can fill the first optimal rotation matrix into the left upper corner 3x3 area of an empty 4x4 matrix, fill the first translation vector into the right upper corner 3x1 area of the 4x4 matrix, and fill the last row of the 4x4 matrix with [0, 0, 0, 1], and the filled 4x4 matrix is the initial registration matrix.

[0081] S103, project the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto the preset plane based on the initial registration matrix and the preset projection parameters to generate and display corresponding two-dimensional projection images; the two-dimensional projection images include the projection contour of the three-dimensional model, each first projection point corresponding to each preset landmark, each second projection point corresponding to each intraoperative landmark, and a plurality of third projection points corresponding to the intraoperative point cloud.

[0082] The preset projection parameters can be used to define the projection parameters required to project the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto the preset plane (e.g. the hip joint cup plane). For example, the preset projection parameters can include the three-dimensional coordinates of the virtual projection source in the CT coordinate system and the orientation of the virtual projection source. It should be noted that the specific representation of the orientation of the virtual projection source can refer to the description in the related art, and the embodiments of the present application will not be described in detail.

[0083] Optionally, the computer device can project the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto the preset plane by the following steps 1.1~1.2:

[0084] Step 1.1, transform the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud to the coordinate system in which the three-dimensional model is located by the inverse matrix of the initial registration matrix, to obtain the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation.

[0085] Specifically, the computer device can perform matrix multiplication operation on the three-dimensional coordinates of each intraoperative landmark and the inverse matrix of the initial registration matrix, respectively, to obtain the three-dimensional coordinates of each intraoperative landmark after coordinate transformation. The three-dimensional coordinates of the intraoperative landmarks after coordinate transformation refer to the three-dimensional coordinates of the intraoperative landmarks in the CT coordinate system.

[0086] Similarly, the computer device can perform matrix multiplication operation on the three-dimensional coordinates of the intraoperative point cloud and the inverse matrix of the initial registration matrix, to obtain the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation. The three-dimensional coordinates of the intraoperative point cloud after coordinate transformation refer to the three-dimensional coordinates of the intraoperative point cloud in the CT coordinate system.

[0087] Step 1.2, project the three-dimensional model, the preset landmarks, the intraoperative landmarks after coordinate transformation and the intraoperative point cloud after coordinate transformation onto the preset plane based on the preset projection parameters.

[0088] Alternatively, the computer device can calculate the two-dimensional coordinates of the first projection point, the second projection point and the third projection point corresponding to each preset landmark, each intraoperative landmark after coordinate transformation and the intraoperative point cloud after coordinate transformation on the preset plane, respectively, according to the preset projection parameters, the isosurface of the three-dimensional model, the three-dimensional coordinates of each preset landmark, the three-dimensional coordinates of each intraoperative landmark after coordinate transformation and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation, using a preset projection transformation manner (for example, orthogonal projection), and calculate the two-dimensional coordinates of each contour point included in the projection contour of the three-dimensional model on the preset plane, so as to project the three-dimensional model, the preset landmark, the intraoperative landmark after coordinate transformation and the intraoperative point cloud after coordinate transformation onto the preset plane.

[0089] Based on this, the computer device can draw the projection contour, the first projection point, the second projection point and the third projection point in the preset plane according to the two-dimensional coordinates of each contour point included in the projection contour, the two-dimensional coordinates of each first projection point, the two-dimensional coordinates of each second projection point and the two-dimensional coordinates of the intraoperative point cloud, so as to obtain a two-dimensional projection image, and display the two-dimensional projection image.

[0090] Alternatively, in order to facilitate the distinction between different types of projection points in the two-dimensional projection image, the first projection point, the second projection point and the third projection point in the two-dimensional projection image can be of different colors. Exemplarily, as shown in FIG. 31, Figure 3 Figure 3 The projection contour in FIG. 31 can be a red dot 32, the second projection point 33 can be a green dot, and the third projection point 34 can be a blue dot. ​

[0091] Furthermore, to facilitate users in determining the overlap rate between the first projection point and the second projection point after the first projection point is dragged to the second projection point, the first projection point, the second projection point, and the third projection point in the two-dimensional projection image can all be semi-transparent dots of different colors.

[0092] S104, in response to the drag operation on the first projection point, update the display position of the first projection point in the two-dimensional projection image in real time, and update the three-dimensional coordinates of the preset landmark corresponding to the first projection point according to the preset projection parameters and the updated display position of the first projection point. Based on the updated three-dimensional coordinates of all preset landmarks and the three-dimensional coordinates of all intraoperative landmarks, calculate the target registration matrix between the three-dimensional model and the target entity object.

[0093] It should be understood that, in order to correct the positional deviation between the first and second projection points caused by inaccurate initial registration matrix, the user can drag the first projection points in the two-dimensional projection image to maximize the overlap between each first projection point and its corresponding second projection point. For example, as... Figure 4 As shown in (a), the yellow arrow indicates a drag operation on the first projection point 32. Since this embodiment restricts the drag operation to a two-dimensional plane, allowing users to drag the first projection point up and down or left and right only within the two-dimensional plane, it reduces the difficulty of manually correcting the initial registration matrix and improves the registration efficiency between the target entity and its three-dimensional model compared to related technologies that correct the initial registration matrix in six dimensions (i.e., three rotational degrees of freedom and three translational degrees of freedom).

[0094] Based on this, when a computer device detects a drag operation targeting any first projection point in a two-dimensional projected image, it can respond to the drag operation by updating the display positions of all first projection points in the two-dimensional projected image in real time. It should be understood that when any first projection point is dragged, the display positions of all first projection points will change to maintain the inherent geometry of the three-dimensional model. For example, as... Figure 4 As shown in (b), the updated display position of the first projection point roughly coincides with the display position of the second projection point.

[0095] Furthermore, the computer device can update the three-dimensional coordinates of the preset landmark corresponding to the first projection point based on preset projection parameters and the updated display position of the first projection point. For example, for any first projection point, the computer device can map the updated display position of the first projection point to a virtual ray emitted from the virtual projection point to the surface of the three-dimensional model based on preset projection parameters, and determine the three-dimensional coordinates of the intersection of the virtual ray and the three-dimensional model in the CT coordinate system as the updated three-dimensional coordinates of the preset landmark corresponding to the first projection point, thereby realizing the update of the preset landmark.

[0096] For example, such as Figure 5 As shown, if the two-dimensional projection image is 51, the original display position of a first projection point in the two-dimensional projection image 51 is 52, the intraoperative landmark corresponding to the first projection point on the three-dimensional model 53 is 54, and the updated display position of the first projection point after the drag operation represented by the yellow arrow in the figure is 55, and the virtual projection source is 56, then the computer device can map the updated display position 55 of the first projection point to a virtual ray 57 pointing from the virtual projection source 56 to the three-dimensional model 53 based on preset projection parameters. The three-dimensional coordinates of the intersection point 58 of the virtual ray 57 and the three-dimensional model 53 are the updated three-dimensional coordinates of the preset landmark corresponding to the first projection point.

[0097] Optionally, computer equipment can be used through, for example Figure 6 S1041~S1045, shown below, calculates the target registration matrix between the 3D model and the target entity object, detailed as follows:

[0098] S1041, based on the updated three-dimensional coordinates of all preset landmarks, determine the third centroid of all preset landmarks, and based on the third centroid, perform decentralization processing on the updated preset landmarks respectively.

[0099] The third centroid is the geometric center of all updated preset landmarks, and its three-dimensional coordinates in the CT coordinate system are... x The axis components are updated after all preset landmarks. x The average value of the axial components, y The axis components are updated after all preset landmarks. y The average value of the axial components, z The axis components are updated after all preset landmarks. z The average value of the axial components.

[0100] Optionally, the computer device can achieve decentralized processing of each updated preset landmark by subtracting the updated three-dimensional coordinates of each preset landmark from the three-dimensional coordinates of the third centroid.

[0101] S1042, based on the three-dimensional coordinates of all intraoperative landmarks, determine the second centroid of all intraoperative landmarks, and based on the second centroid, perform decentralization processing on each intraoperative landmark.

[0102] It should be noted that S1042 is exactly the same as S1022. Therefore, the specific implementation process of S1042 can be referred to the relevant description in S1022, and will not be repeated here.

[0103] S1043, calculate a second optimal rotation matrix between all the updated preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of all the updated preset landmarks after decentralization and the three-dimensional coordinates of all the intraoperative landmarks.

[0104] The second optimal rotation matrix is a rotation matrix that can make the overall deviation between all the updated preset landmarks after decentralization and all the intraoperative landmarks after rotation be the smallest. The dimension of the second optimal rotation matrix can be 3x3. It should be noted that since the calculation process of the optimal rotation matrix of two three-dimensional coordinates in different coordinate systems is a known technology, the calculation process of the second optimal rotation matrix can refer to the description in the related technology, and the embodiments of the present application will not be described in detail.

[0105] S1044, calculate a second translation vector between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of the third centroid, the three-dimensional coordinates of the second centroid, and the optimal rotation matrix.

[0106] The second translation vector is used to define the displacement required to make all the updated preset landmarks after rotation and all the intraoperative landmarks maximally coincide in space. The dimension of the second translation vector can be 3x1.

[0107] Specifically, the computer device can calculate the product of the second optimal rotation matrix and the three-dimensional coordinates of the third centroid, and subtract the three-dimensional coordinates of the second centroid from the product to obtain the second translation vector.

[0108] S1045, construct the second optimal rotation matrix and the second translation vector into a target registration matrix.

[0109] The target registration matrix is a 4x4-dimensional standard homogeneous coordinate transformation matrix.

[0110] Specifically, the computer device can fill the second optimal rotation matrix into the left upper corner 3x3 area of an empty 4x4 matrix, fill the second translation vector into the right upper corner 3x1 area of the 4x4 matrix, and fill the last row of the 4x4 matrix by [0, 0, 0, 1]. The filled 4x4 matrix is the target registration matrix.

[0111] S105, register the three-dimensional model and the intraoperative point cloud based on the target registration matrix, and update the display positions of all the projection points in the two-dimensional projection image.

[0112] Optionally, the computer device can take the target registration matrix as the rigid transformation initial value of an iterative closest point (ICP) algorithm, and perform registration on the three-dimensional model and the intraoperative point cloud through the ICP algorithm to obtain a final registration matrix and a corresponding registration quality parameter, and when the registration quality parameter meets a preset requirement, update the display positions of the first projection point, the second projection point and the third projection point in the two-dimensional projection image based on the final registration matrix. It should be noted that the ICP is a known technology, and therefore the specific process of performing registration on the three-dimensional model and the intraoperative point cloud through the ICP algorithm can refer to the description in the related art, and the embodiments of the present application will not be described in detail.

[0113] The registration quality parameter can be used to measure the registration quality between the three-dimensional model and the intraoperative point cloud. The specific type of the registration quality parameter can be set according to actual needs, for example, the registration quality parameter can include any one or more of a root mean square error, an inlier ratio and an overlap rate. Based on this, the preset requirement can also be set according to actual needs, for example, the preset requirement can include any one or more of a root mean square error that cannot be less than a preset error threshold, an inlier ratio that is greater than a preset ratio threshold, and an overlap rate that is greater than a preset overlap rate.

[0114] Exemplarily, the two-dimensional projection image finally obtained after updating the display positions of the first projection point, the second projection point and the third projection point in the two-dimensional projection image can be as shown in Figure 7 Compared with the original two-dimensional projection image as shown in Figure 3 All the projection points in the projection image as shown in Figure 7 are within the projection contour.

[0115] In addition, the related art usually directly searches for a preset landmark that completely corresponds to an intraoperative landmark on a three-dimensional model when performing registration, and such a registration method has multiple solutions, especially when the number of landmarks is small, an infinite number of results that satisfy the constraints can be found on the three-dimensional model, as shown by the dashed line in (a) in Figure 8 , thereby causing the above search to have exponential complexity. As shown in (b) in Figure 8 , the present application can avoid global search, reduce registration efficiency and improve registration efficiency by combining two-dimensional dragging with local constraints.

[0116] It can be seen that the method for registering a three-dimensional model and a physical object provided in the embodiment comprises the following steps: calculating an initial registration matrix between a target physical object and a three-dimensional model of the target physical object; and projecting the three-dimensional model, preset landmarks on the three-dimensional model, intraoperative landmarks on the target physical object, and an intraoperative point cloud on the target physical object onto a preset plane based on the initial registration matrix. In this way, a user can drag a first projection point corresponding to a preset landmark in a two-dimensional projection image up, down, left, or right to update the display position of the first projection point in the two-dimensional projection image, thereby achieving accurate correction of the initial registration matrix. The three-dimensional model and the intraoperative point cloud are registered based on a target registration matrix obtained through correction, and the display position of the projection point in the two-dimensional projection image is updated, thereby achieving accurate registration of the target physical object and the three-dimensional model. That is, the method can reduce the difficulty of manual correction of the initial registration matrix while improving the registration accuracy and efficiency of the target physical object and the three-dimensional model.

[0117] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0118] Based on the method for registering a three-dimensional model and a physical object provided in the above embodiment, the present embodiment further provides an embodiment of a computer device for implementing the above method embodiment. Figure 9 A structural schematic diagram of a computer device provided in the present embodiment is shown. For ease of illustration, only parts related to the present embodiment are shown. As shown in Figure 9 The computer device 90 can include a first acquisition unit 901, a first calculation unit 902, a virtual projection unit 903, a first processing unit 904, and a second processing unit 905. Among them:

[0119] The first acquisition unit 901 is configured to acquire a three-dimensional model of a target physical object, three-dimensional coordinates of each preset landmark on the three-dimensional model, three-dimensional coordinates of each intraoperative landmark on the target physical object corresponding to each preset landmark, and three-dimensional coordinates of an intraoperative point cloud on the target physical object.

[0120] The first calculation unit 902 is configured to calculate an initial registration matrix between the three-dimensional model and the target physical object according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks.

[0121] The virtual projection unit 903 is configured to project the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters to generate and display a corresponding two-dimensional projection image; the two-dimensional projection image includes a projection contour of the three-dimensional model, each first projection point corresponding to each preset landmark, each second projection point corresponding to each intraoperative landmark, and a plurality of third projection points corresponding to the intraoperative point cloud.

[0122] The first processing unit 904 is configured to update a display position of the first projection point in the two-dimensional projection image in real time in response to a drag operation on the first projection point, update a three-dimensional coordinate of the preset landmark corresponding to the first projection point according to the preset projection parameters and the updated display position of the first projection point, and calculate a target registration matrix between the three-dimensional model and the target entity object according to the updated three-dimensional coordinates of all the preset landmarks and three-dimensional coordinates of all the intraoperative landmarks.

[0123] The second processing unit 905 is configured to register the three-dimensional model and the intraoperative point cloud based on the target registration matrix and update display positions of all the projection points in the two-dimensional projection image.

[0124] Optionally, the first calculation unit 902 is specifically configured to:

[0125] determine a first centroid of all the preset landmarks according to the three-dimensional coordinates of all the preset landmarks, and perform decentralization processing on each preset landmark based on the first centroid;

[0126] determine a second centroid of all the intraoperative landmarks according to the three-dimensional coordinates of all the intraoperative landmarks, and perform decentralization processing on each intraoperative landmark based on the second centroid;

[0127] calculate a first optimal rotation matrix between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of all the preset landmarks and all the intraoperative landmarks after the decentralization processing;

[0128] calculate a first translation vector between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of the first centroid, the three-dimensional coordinates of the second centroid and the first optimal rotation matrix;

[0129] construct the first optimal rotation matrix and the first translation vector into the initial registration matrix.

[0130] Optionally, the virtual projection unit 903 is specifically configured to:

[0131] transform the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud to a coordinate system in which the three-dimensional model is located through an inverse matrix of the initial registration matrix to obtain the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation;

[0132] project the three-dimensional model, the preset landmarks, the three-dimensional coordinates of the intraoperative landmarks after coordinate transformation and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation onto a preset plane based on preset projection parameters.

[0133] Optionally, the first projection point, the second projection point and the third projection point in the two-dimensional projection image are respectively semitransparent dots of different colors.

[0134] Optionally, the first processing unit 904 is specifically configured to:

[0135] determine a third centroid of all the preset landmarks according to the updated three-dimensional coordinates of all the preset landmarks, and perform decentralization processing on the updated preset landmarks respectively based on the third centroid;

[0136] determine a second centroid of all the intraoperative landmarks according to the three-dimensional coordinates of all the intraoperative landmarks, and perform decentralization processing on each of the intraoperative landmarks based on the second centroid;

[0137] calculate a second optimal rotation matrix between all the updated preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of all the updated preset landmarks after decentralization processing and the three-dimensional coordinates of all the intraoperative landmarks;

[0138] calculate a second translation vector between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of the third centroid, the three-dimensional coordinates of the second centroid and the optimal rotation matrix;

[0139] construct the second optimal rotation matrix and the second translation vector into a target registration matrix.

[0140] Optionally, the second processing unit 905 is specifically configured to:

[0141] use the target registration matrix as a rigid transformation initial value of an iterative closest point algorithm, and perform registration on the three-dimensional model and the intraoperative point cloud through the iterative closest point algorithm to obtain a final registration matrix and a corresponding registration quality parameter;

[0142] if the registration quality parameter meets a preset requirement, update the display positions of the first projection point, the second projection point and the third projection point in the two-dimensional projection image based on the final registration matrix.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units is taken as an example, and in actual application, the above functions can be completed by different functional units according to needs, that is, the internal structure of the computer device is divided into different functional units to complete all or part of the above described functions. The functional units in the embodiments can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of the functional units are only for mutual distinction, and do not limit the protection scope of the present application. The specific working process of each unit in the computer device can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0144] Please refer to Figure 10 , Figure 10 The structural schematic diagram of a computer device provided by another embodiment of the present application is shown in the figure. As shown in the figure, Figure 10 The computer device 11 provided by the embodiment can include a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110, for example, a program corresponding to the registration method of the three-dimensional model and the entity object. The processor 110 implements the steps in the above-mentioned registration method of the three-dimensional model and the entity object embodiment when executing the computer program 112, for example, S101-S105 shown in the figure. Or the processor 110 implements the functions of each unit in the above-mentioned computer device embodiment when executing the computer program 112. Figure 1

[0145] For example, the computer program 112 can be divided into one or more modules / units, one or more modules / units are stored in the memory 111 and executed by the processor 110 to complete the present application. One or more modules / units can be a series of computer program instruction segments that can complete a specific function, which is used to describe the execution process of the computer program 112 in the computer device 11. For example, the computer program 112 can be divided into a first acquisition unit, a first calculation unit, a virtual rendering unit, a first processing unit, and a second processing unit. The specific functions of each unit can refer to the related description in the corresponding embodiment, which will not be described here. Figure 9

[0146] Those skilled in the art can understand that Figure 10 The computer device 11 is only an example and does not constitute a limitation on the computer device 11, which can include more or fewer components than shown, or combine certain components, or different components.

[0147] ​​The processor 110 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0148] The memory 111 can be an internal storage unit of the computer device 11, for example, a hard disk or a memory of the computer device 11. The memory 111 can also be an external storage device of the computer device 11, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card or a flash card, or the like, which is equipped on the computer device 11. Further, the memory 111 can include both the internal storage unit and the external storage device of the computer device 11. The memory 111 is used to store computer programs and other programs and data required by the computer device. The memory 111 can also be used to temporarily store data that has been output or will be output.

[0149] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each step of the registration method of the three-dimensional model and the entity object in the above method embodiment.

[0150] The embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device implements the steps in each of the above method embodiments.

[0151] In the above embodiments, the description of each embodiment has its own focus. The parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0152] It should be noted that, unless otherwise specified, all technical terms used in the embodiments of the present application have the same meanings as those commonly understood by the person skilled in the art to which the present application belongs. The technical terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0153] The phrase "in one embodiment" in the description of the embodiments of the present application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase that in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that the embodiments described herein are merely example and can be combined with each other in their various aspects without limitation.

[0154] Those skilled in the art can understand that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0155] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method of registering a three-dimensional model with a physical object, the method comprising: The method comprises the following steps: acquiring a three-dimensional model of a target entity object, three-dimensional coordinates of each preset landmark on the three-dimensional model, three-dimensional coordinates of each intraoperative landmark corresponding to each preset landmark on the target entity object, and three-dimensional coordinates of an intraoperative point cloud on the target entity object; calculating an initial registration matrix between the three-dimensional model and the target entity object according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks; projecting the three-dimensional model, the preset landmarks, the intraoperative landmarks, and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters to generate and display a corresponding two-dimensional projection image; the two-dimensional projection image comprises a projection contour of the three-dimensional model, each first projection point corresponding to each preset landmark, each second projection point corresponding to each intraoperative landmark, and a plurality of third projection points corresponding to the intraoperative point cloud; in response to a drag operation on the first projection point, updating the display position of the first projection point in the two-dimensional projection image in real time, updating the three-dimensional coordinates of the preset landmark corresponding to the first projection point according to the preset projection parameters and the updated display position of the first projection point, and calculating a target registration matrix between the three-dimensional model and the target entity object according to the updated three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks; registering the three-dimensional model and the intraoperative point cloud based on the target registration matrix and updating the display positions of all the projection points in the two-dimensional projection image.

2. The method of claim 1, wherein, The method comprises the following steps: calculating an initial registration matrix between the three-dimensional model and the target entity object according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks, which comprises the following steps: determining a first centroid of all the preset landmarks according to the three-dimensional coordinates of all the preset landmarks, and performing decentralization processing on each preset landmark based on the first centroid; determining a second centroid of all the intraoperative landmarks according to the three-dimensional coordinates of all the intraoperative landmarks, and performing decentralization processing on each intraoperative landmark based on the second centroid; calculating a first optimal rotation matrix between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of all the preset landmarks after decentralization processing and the three-dimensional coordinates of all the intraoperative landmarks; calculating a first translation vector between all the preset landmarks and all the intraoperative landmarks according to the three-dimensional coordinates of the first centroid, the three-dimensional coordinates of the second centroid, and the first optimal rotation matrix; 3. The method of claim 1, wherein, constructing the first optimal rotation matrix and the first translation vector into the initial registration matrix. projecting the three-dimensional model, the preset landmarks, the intraoperative landmarks, and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters, which comprises the following steps: Transform the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud to a coordinate system in which the three-dimensional model is located through an inverse matrix of the initial registration matrix to obtain the three-dimensional coordinates of the intraoperative landmarks and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation; Project the three-dimensional model, the preset landmarks, the three-dimensional coordinates of the intraoperative landmarks after coordinate transformation and the three-dimensional coordinates of the intraoperative point cloud after coordinate transformation onto a preset plane based on preset projection parameters.

4. The method of claim 1, wherein, The first projection point, the second projection point and the third projection point in the two-dimensional projection image are respectively semitransparent circular dots of different colors.

5. The method of claim 1, wherein, According to the three-dimensional coordinates of all the updated preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks, a target registration matrix between the three-dimensional model and the target entity object is calculated, including: According to the three-dimensional coordinates of all the updated preset landmarks, a third centroid of all the preset landmarks is determined, and the updated preset landmarks are respectively subjected to decentralization processing based on the third centroid; According to the three-dimensional coordinates of all the intraoperative landmarks, a second centroid of all the intraoperative landmarks is determined, and each of the intraoperative landmarks is subjected to decentralization processing based on the second centroid; According to the three-dimensional coordinates of all the updated preset landmarks after decentralization processing and the three-dimensional coordinates of all the intraoperative landmarks, a second optimal rotation matrix between all the updated preset landmarks and all the intraoperative landmarks is calculated; According to the three-dimensional coordinates of the third centroid, the three-dimensional coordinates of the second centroid and the optimal rotation matrix, a second translation vector between all the preset landmarks and all the intraoperative landmarks is calculated; The second optimal rotation matrix and the second translation vector are constructed into a target registration matrix.

6. The method according to any one of claims 1 to 5, characterized in that, Based on the target registration matrix, the three-dimensional model and the intraoperative point cloud are registered, and the display positions of all the projection points in the two-dimensional projection image are updated, including: The target registration matrix is taken as a rigid transformation initial value of an iterative closest point algorithm, and the three-dimensional model and the intraoperative point cloud are registered through the iterative closest point algorithm to obtain a final registration matrix and a corresponding registration quality parameter; If the registration quality parameter meets preset requirements, the display positions of the first projection point, the second projection point and the third projection point in the two-dimensional projection image are updated based on the final registration matrix.

7. A computer device, comprising: including: The first acquisition unit is configured to acquire a three-dimensional model of a target entity object, three-dimensional coordinates of each preset landmark on the three-dimensional model, three-dimensional coordinates of each intraoperative landmark corresponding to each preset landmark on the target entity object, and three-dimensional coordinates of an intraoperative point cloud on the target entity object; The first calculation unit is configured to calculate an initial registration matrix between the three-dimensional model and the target entity object according to the three-dimensional coordinates of all the preset landmarks and the three-dimensional coordinates of all the intraoperative landmarks; a virtual projection unit configured to project the three-dimensional model, the preset landmarks, the intraoperative landmarks and the intraoperative point cloud onto a preset plane based on the initial registration matrix and preset projection parameters to generate and display a corresponding two-dimensional projection image; the two-dimensional projection image including a projection contour of the three-dimensional model, each first projection point corresponding to each of the preset landmarks, each second projection point corresponding to each of the intraoperative landmarks, and a plurality of third projection points corresponding to the intraoperative point cloud; a first processing unit configured to, in response to a drag operation on the first projection point, update a display position of the first projection point in the two-dimensional projection image in real time, update a three-dimensional coordinate of the preset landmark corresponding to the first projection point according to the preset projection parameters and the updated display position of the first projection point, and calculate a target registration matrix between the three-dimensional model and the target entity object according to the updated three-dimensional coordinates of all the preset landmarks and three-dimensional coordinates of all the intraoperative landmarks; a second processing unit configured to register the three-dimensional model and the intraoperative point cloud based on the target registration matrix, and update display positions of all the projection points in the two-dimensional projection image.

8. A computer device, comprising: A computer program product comprising a memory and a computer program stored in the memory and loadable into the working memory of a digital computer, the computer program comprising instructions which, when loaded into the working memory of the digital computer and executed by the digital computer, cause the digital computer to execute the method according to any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program product comprises a computer program which, when executed by a processor, causes the processor to carry out the method according to any one of claims 1-6.

10. A computer program product, characterised in that, The computer program product comprises a computer program which, when executed by a processor, causes the processor to carry out the method according to any one of claims 1-6. The computer program product comprises a computer program which, when executed by a processor, causes the processor to carry out the method according to any one of claims 1-6.

Images