Acetabular registration method, apparatus, medium and product

Abstract

The invention discloses an acetabulum registration method and device, a medium and a product, and relates to the technical field of medical image.The acetabulum registration method comprises the steps that first point cloud data corresponding to a pelvic bone surface model in a model local coordinate system is acquired; acquiring human body image data acquired by the image device, wherein the human body image data is in an image local coordinate system; performing data transformation on the first point cloud data to enable a position relation between the first point cloud data transformed into an image local coordinate system and a pelvic bone area in the human body image to meet a preset overlapping condition, and recording the transformed first point cloud data as second point cloud data; registration calculation is carried out based on the second point cloud data and the first point cloud data to obtain a first registration matrix, registration is carried out by combining an augmented reality technology and taking overlapping of a pelvic bone model projection position and a pelvic bone position as a target, and the problem of large registration error caused by unreasonable position marking can be effectively avoided.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to an acetabulum registration method, device, medium, and product. Background Art

[0002] In image-guided hip replacement surgery, establishing an accurate mapping relationship between the real-space coordinate system and the image-space coordinate system is the basis for a successful operation. The process of establishing this mapping relationship is called acetabular registration.

[0003] The currently commonly used acetabulum registration method is as follows: the staff will preset n marker points in the acetabulum of the pelvic surface model before the operation. During the operation, the doctor uses an optical probe to collect m (m>n) points at the corresponding positions of the patient's real acetabulum according to the position instructions of the marker points on the screen. The optical probe is attached with a number greater than or equal to 3 optical markers that can be recognized by the positioning camera. Finally, based on the collected point set and pelvic surface model data, combined with the point cloud matching method, the acetabulum registration matrix is calculated. However, the marker points on the pelvic surface model used for registration are concentrated in the acetabulum area. If the registration points collected by the probe do not match the marker points in the image, since the human acetabulum is spherical, the point set collected by the probe and the marker point set in the image can still be well matched through the spherical shell configuration, causing the system to mistakenly believe that the registration is successful, resulting in a large registration error. Summary of the Invention

[0004] The present application provides an acetabulum registration method, device, medium and product to solve the technical problem of large acetabulum registration error in related technologies.

[0005] To achieve the above objectives, the present application proposes an acetabulum registration method, which comprises:

[0006] Acquiring first point cloud data corresponding to the pelvic surface model in the local coordinate system of the model;

[0007] Acquire human body image data acquired by an imaging device, wherein the human body image data is in an image local coordinate system;

[0008] Performing data transformation on the first point cloud data so that a positional relationship between the first point cloud data transformed into the local image coordinate system and the pelvic region in the human body image satisfies a preset overlapping condition, and recording the transformed point cloud data that satisfies the preset overlapping condition as second point cloud data;

[0009] A registration calculation is performed based on the second point cloud data and the first point cloud data to obtain a first registration matrix.

[0010] In addition, to achieve the above-mentioned purpose, the present application also proposes a terminal device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the acetabulum alignment method as described above.

[0011] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the acetabulum alignment method described above are implemented.

[0012] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the acetabulum registration method described above are implemented.

[0013] One or more technical solutions proposed in this application have at least the following technical effects:

[0014] By acquiring first point cloud data corresponding to the pelvic surface model in the local coordinate system of the model; acquiring human body image data acquired by an imaging device, the human body image data being in the local coordinate system of the image; performing data transformation on the first point cloud data so that the positional relationship between the first point cloud data transformed into the local coordinate system of the image and the pelvic region in the human body image meets a preset overlapping condition, and recording the transformed first point cloud data as second point cloud data; performing registration calculation based on the second point cloud data and the first point cloud data to obtain a first registration matrix, and performing registration with the overlap of the projection position of the pelvic model and the pelvic position as the goal by combining augmented reality technology. Compared with the traditional method of manually marking and positioning on the image, this embodiment can effectively avoid the problem of large registration errors caused by unreasonable position marking, thereby improving the accuracy of acetabulum registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is an application scenario diagram of acetabulum registration provided in this application;

[0018] Figure 2A schematic diagram of a flow chart of an embodiment of the acetabulum registration method of the present application;

[0019] Figure 3 A structural diagram of a framing system according to an embodiment of the present application;

[0020] Figure 4 A schematic diagram of the process of projecting the pelvic model provided in an embodiment of the present application onto the local coordinate system of the image where the human body image is located;

[0021] Figure 5 A schematic diagram showing the overlap of a projection image of the pelvic model provided in an embodiment of the present application in the local image coordinate system and the patient's pelvic area;

[0022] Figure 6 Schematic diagram of the device structure of the hardware operating environment involved in the acetabulum registration method in the embodiment of the present application.

[0023] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0025] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0026] In image-guided hip replacement surgery, establishing an accurate mapping relationship between the real space coordinate system and the image space coordinate system is the basis for successful surgery. The process of establishing this mapping relationship is called acetabulum registration. Figure 1 As shown, Figure 1 The application scenario of acetabulum registration is shown. Before surgery, the patient needs to take a computed tomography (CT) image of the acetabulum. The surgical planning and surface model reconstruction of the patient's pelvis will be performed on this acetabulum scan image, which is in the image space. During the operation, an optical reference array will be fixed on the patient's pelvis. The optical reference array has more than 3 optical markers that can be recognized by the positioning camera ( Figure 1 The positioning camera collects the positions of the optical markers on the optical reference array and uses the triangulation method to determine the position of the optical reference array relative to the camera, that is, to obtain the local coordinate system F of the positioning camera. camera To the patient's pelvic optical reference array local coordinate system F boneRF The transformation matrix T cameraToBoneRF .

[0027] The acetabulum registration step is to obtain the optical reference array coordinate system F on the pelvisboneRF To the preoperative image space coordinate system F image The transformation matrix T boneRFtoImage When you get T boneRFtoImage After that, the system can image The pose of the surgical planning area in the camera →F boneRF →F image Convert to the local coordinate system F of the positioning camera camera The target position that the corresponding optical execution tool (such as a bone knife, bone drill, etc. with an optical reference array) that can be tracked by the positioning camera is further known.

[0028] After the acetabulum registration is completed, the system can calculate the transformation matrix T from the positioning camera coordinate system to the image coordinate system in real time. cameraToImage =T cameraToBoneRF ·T boneRFtoImage , suppose a surgical planning area is in F image The pose matrix under is A, then the actual execution tool is F camera The pose matrix B that needs to be achieved is calculated as follows:

[0029] B=T cameraToImage A=T cameraToBoneRF ·T boneRFtoImage A. Obviously, the T obtained by acetabulum registration boneRFtoImage , its error will be included in the positioning matrix B of the subsequent execution tool, that is, the accuracy of acetabulum registration directly affects the positioning accuracy of the system.

[0030] The acetabulum registration method commonly used in related technologies is: the staff will preset n marker points in the acetabulum of the pelvic surface model before the operation. During the operation, the doctor uses an optical probe to collect m (m>n) points at the corresponding positions of the patient's real acetabulum according to the position instructions of the marker points on the screen. The optical probe is attached with a number greater than or equal to 3 optical markers that can be recognized by the positioning camera. Finally, based on the collected point set and pelvic surface model data, combined with the point cloud matching method, the acetabulum registration matrix is calculated. However, the marker points on the pelvic surface model used for registration are concentrated in the acetabulum area. If the registration points collected by the probe do not match the marker points in the image, since the human acetabulum is spherical, the point set collected by the probe and the marker point set in the image can still be well matched through the spherical shell configuration, causing the system to mistakenly believe that the registration is successful, and the registration error is large.

[0031] Based on this, an embodiment of the present application provides an acetabulum registration method. The execution subject of this embodiment can be a terminal device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, or server. This embodiment and the following embodiments are described below using a terminal device as an example.

[0032] Please refer to Figure 2 , Figure 2 This is a flow chart of an embodiment of the acetabulum registration method of the present application. In this embodiment, the acetabulum registration method includes steps S10 to S40:

[0033] S10, obtaining first point cloud data corresponding to the pelvic surface model in the local coordinate system of the model.

[0034] In this embodiment, the terminal device can obtain a three-dimensional pelvic surface model based on the pre-acquired three-dimensional CT image of the patient's pelvis using a preset image segmentation algorithm or deep learning segmentation reconstruction, and record the spatial coordinate system of the pelvic surface model as the model local coordinate system F image , the pelvic surface model data is the first point cloud data.

[0035] S20, obtaining human body image data acquired by an imaging device, where the human body image data is in a local image coordinate system.

[0036] In some embodiments, the imaging device may be an RGB camera or an RGB-D camera. The RGB camera can obtain a color image of the three-dimensional scene information, and the RGB-D camera can simultaneously obtain the color image and depth information of the scene. In this embodiment, the spatial coordinate system where the data collected by the imaging device is located is the image local coordinate system F stereoCamera .

[0037] S30, performing data transformation on the first point cloud data so that the positional relationship between the first point cloud data transformed into the local coordinate system of the image and the pelvic area in the human body image meets a preset overlapping condition, and recording the transformed point cloud data that meets the preset overlapping condition as the second point cloud data.

[0038] Compared with the method of aligning based on marker points in related technologies, this embodiment transforms the first point cloud data with the goal of ensuring that the position between the pelvic model (i.e., the first point cloud data) and the patient's pelvic area meets the preset overlapping conditions. There is no need for manual marking in the image, which can reduce the alignment error caused by manual marking.

[0039] In some embodiments, the above-mentioned S30 may include: obtaining the posture data of the optical probe collected by the tracking device in the tracking local coordinate system, and converting the first point cloud data to the tracking local coordinate system based on the posture data to obtain third point cloud data; based on a predetermined first transformation matrix, converting the third point cloud data to the image local coordinate system to obtain transformed point cloud data, wherein the first transformation matrix is the transformation matrix between the image local coordinate system and the tracking local coordinate system; if the positional relationship between the transformed point cloud data and the pelvic area does not meet the preset overlapping condition, adjusting the posture data, and recalculating the point cloud data obtained after the first point cloud data is transformed into the image local coordinate system based on the adjusted posture data, until the positional relationship between the point cloud data obtained after the transformation and the pelvic area meets the preset overlapping condition, and recording the point cloud data that meets the preset overlapping condition as the second point cloud data.

[0040] S40: Perform registration calculation based on the second point cloud data and the first point cloud data to obtain a first registration matrix.

[0041] In some embodiments, the first point cloud data in the local coordinate system of the model is converted to the local coordinate system of the image, and the corresponding second point cloud data is obtained. Therefore, the terminal device can perform registration calculations on the second point cloud data and the first point cloud data through a point cloud registration algorithm, thereby obtaining the conversion relationship between the local coordinate system of the model where the first point cloud data is located and the local coordinate system of the image where the second point cloud data is located, that is, the first registration matrix.

[0042] This embodiment obtains first point cloud data corresponding to the pelvic surface model in the local coordinate system of the model; obtains human body image data collected by an imaging device, and the human body image data is in the local coordinate system of the image; performs data transformation on the first point cloud data so that the positional relationship between the first point cloud data transformed into the local coordinate system of the image and the pelvic area in the human body image meets the preset overlapping condition, and records the transformed first point cloud data as second point cloud data; performs alignment calculation based on the second point cloud data and the first point cloud data to obtain a first alignment matrix, and performs alignment with the overlap of the projection position of the pelvic model and the pelvic position as the goal by combining augmented reality technology. Compared with the traditional method of manually marking and positioning on the image, this embodiment can effectively avoid the problem of large alignment errors caused by unreasonable position marking, thereby improving the accuracy of acetabulum alignment.

[0043] In an optional implementation, the above S20 may include S201 to S203:

[0044] S201, obtaining the pose data of the optical probe collected by the tracking device in the tracking local coordinate system, and converting the first point cloud data into the tracking local coordinate system based on the pose data to obtain third point cloud data.

[0045] In some embodiments, the above S201 may include: based on the posture data and the pre-acquired probe calibration data in the probe local coordinates, calculating the transformation matrix between the tracking local coordinate system and the probe local coordinate system to obtain a second transformation matrix; then, based on the predetermined third transformation matrix and the second transformation matrix, calculating the transformation matrix between the model local coordinate system and the tracking local coordinates to obtain a fourth transformation matrix, wherein the third transformation matrix is the transformation matrix between the model local coordinate system and the probe local coordinate system; finally, based on the fourth transformation matrix, converting the first point cloud data to the tracking local coordinate system to obtain third point cloud data.

[0046] See Figure 3 , Figure 3 This is a structural diagram of the framing system involved in this embodiment. The framing system includes a tracking device and an imaging device. The tracking device is used to track the three-dimensional spatial position of the pelvic reference array and the probe reference array. The imaging device is used to capture the human body image and render the human body image and send it to the user end so that the user can intuitively observe the characteristics of the human body image. The tracking device can capture the three-dimensional spatial position of the pelvic reference array and the probe reference array in real time, and can be used in the tracking local coordinate system F of the tracking device. stereoCamera Reconstruct the probe local coordinate system F probe and the pelvic reference array local coordinate system F boneRF In the tracking local coordinate system F stereoCamera location.

[0047] In this embodiment, the tracking device and the imaging device have been pre-calibrated, that is, the imaging device is located in the local image coordinate system F rgbCamera To the tracking local coordinate system F where the tracking device is located stereoCamera The conversion matrix of is a known value, and the intrinsic parameter matrix K of the imaging device is a known value.

[0048] Before performing acetabulum registration, the terminal device calibrates the optical probe to obtain the probe calibration data in the local coordinate system of the optical probe, that is, the probe local coordinate system. Specifically, the probe local coordinate system F probe The coordinate system is determined by the reflective marker attached to it (for example, the center of a reflective marker is the origin of the coordinate system, the direction of the line connecting the marker and another marker is the y-axis of the coordinate system, the normal vector of the plane where all markers are located is the x-axis direction, and the x-axis cross product of the y-axis direction is the z-axis direction). Then, calibration points are selected in the probe's local coordinate system to obtain the probe calibration data.

[0049] In addition, after obtaining the pelvic surface model, the terminal device locates the pelvic surface model in the local coordinate system F image Import the pre-established probe model and adjust the probe model in the model local coordinate system F imageThe position in the middle makes the relative position relationship between the probe model and the pelvic surface model close to the state in which the surgical operator uses the probe to collect points in the acetabulum during surgery, and selects marked points on the probe model to obtain the probe point data in the local coordinate system of the model. Then, the transformation matrix between the local coordinate system of the model and the local coordinate system of the probe can be determined based on the probe calibration data and the probe point data, that is, the third transformation matrix.

[0050] The terminal device can obtain the pose data of the optical probe in the tracking local coordinate system through the tracking device, and combine it with the probe calibration data in the probe local coordinate system to perform point cloud registration calculation based on the pose data and the probe calibration data, thereby obtaining the transformation matrix between the tracking local coordinate system and the probe local coordinate system, and recording it as the second transformation matrix. Then, based on the third transformation matrix and the second transformation matrix, the transformation matrix between the model local coordinate system and the tracking local coordinate system, that is, the fourth transformation matrix, is obtained, so that the first point cloud data can be converted to the tracking local coordinate system through the fourth transformation matrix. Specifically, if the fourth transformation matrix is specifically expressed as a transformation matrix from the model local coordinate system to the tracking local coordinate system, then each point in the first point cloud data is multiplied by the fourth transformation matrix to obtain the third point cloud data; if the fourth transformation matrix is specifically expressed as a transformation matrix from the tracking local coordinate system to the model local coordinate system, then each point in the first point cloud data is multiplied by the inverse matrix of the fourth transformation matrix to obtain the third point cloud data.

[0051] S202 : Based on a predetermined first transformation matrix, transform the third point cloud data into the image local coordinate system to obtain transformed point cloud data, wherein the first transformation matrix is a transformation matrix between the image local coordinate system and the tracking local coordinate system.

[0052] After converting the first point cloud data to the tracking local coordinate system to obtain the third point cloud data, the first point cloud data can be converted to the image local coordinate system where the human body image is located through the transformation matrix between the image local coordinate system and the tracking local coordinate system, that is, the first transformation matrix, to obtain the changed point cloud data.

[0053] See Figure 4 , Figure 4 The figure shows the process of projecting the pelvic model to the local coordinate system of the image where the human body image is located. Figure 4 As shown, the local coordinate system of the imaging device is F rgbCamera The local coordinate system where the tracking device is located is the tracking local coordinate system F stereoCamera The tracking device can collect and track the three-dimensional spatial position of the pelvic reference array and the probe reference array, thereby obtaining the tracking local coordinate system F of the tracking device. stereoCamera With the probe local coordinate system F probeThe second transformation matrix between the local coordinate system F stereoCamera and the pelvic reference array local coordinate system F boneRF The transformation matrix between them, and by tracking the local coordinate system F stereoCamera With the probe local coordinate system F probe The second transformation matrix between the local coordinate system F stereoCamera and the pelvic reference array local coordinate system F boneRF The transformation matrix between the first point cloud data is obtained in the tracking local coordinate system F stereoCamera The position of the third point cloud data is obtained, and the projection of the pelvic model to the tracking local coordinate system F is obtained through the third point cloud data. stereoCamera The projected image is then based on the image local coordinate system F rgbCamera and tracking the local coordinate system F stereoCamera The transformation matrix between them maps the pelvic model to the image local coordinate system F rgbCamera middle.

[0054] S203, if the positional relationship between the transformed point cloud data and the pelvic area does not meet the preset overlapping conditions, the posture data is adjusted, and the point cloud data obtained after the first point cloud data is transformed into the local coordinate system of the image is recalculated based on the adjusted posture data, until the positional relationship between the point cloud data obtained after the transformation and the pelvic area meets the preset overlapping conditions, and the point cloud data that meets the preset overlapping conditions is recorded as the second point cloud data.

[0055] In this embodiment, when the positional relationship between the transformed point cloud data and the pelvic region meets the preset overlapping condition, it indicates that the projection image of the pelvic model in the local image coordinate system overlaps with the patient's pelvic region. Figure 5 , Figure 5 A schematic diagram showing the overlap of the projection image of the pelvic model in the local image coordinate system and the patient's pelvic region is shown.

[0056] In some embodiments, after each transformation round is completed, it is manually determined whether the point cloud data obtained after the transformation coincides with the pelvic area in the human body image. If so, a first instruction is input to the terminal device. After receiving the first instruction input manually, the terminal device determines that the positional relationship between the point cloud data obtained by projecting the first point cloud data in the current change round onto the human body image and the pelvic area in the human body image meets the preset overlapping condition; if it is manually determined that the point cloud data obtained after the transformation does not coincide with the pelvic area in the human body image, the position of the probe is manually moved so that the terminal device receives the latest posture information of the probe. After receiving the latest posture information of the probe, the terminal device determines that the positional relationship between the point cloud data obtained by projecting the first point cloud data in the current change round onto the human body image and the pelvic area in the human body image does not meet the preset overlapping condition, and adjusts the posture data of the probe to recalculate the point cloud data obtained after the first point cloud data is transformed into the local coordinate system of the image based on the adjusted posture data.

[0057] In other embodiments, the terminal device can obtain the coordinates (x, y) of all points in the transformed point cloud data, and then find the minimum and maximum values of the x-axis coordinates, as well as the minimum and maximum values of the y-axis coordinates, and determine the four vertices of the maximum range area formed by the transformed point cloud data, which are (x-axis coordinate minimum, y-axis coordinate minimum), (x-axis coordinate minimum, y-axis coordinate maximum), (x-axis coordinate maximum, y-axis coordinate minimum), and (x-axis coordinate maximum, y-axis coordinate maximum), and form a maximum range area based on the four vertices. In addition, the terminal device can process the human body image through deep learning (such as the Yolo algorithm) to obtain a detection frame of the pelvic area in the human body image, which is used to mark the contour area of the pelvis in the human body image. Then, the overlapping area ratio between the maximum range area and the detection frame of the pelvic area is calculated. If the overlapping area ratio is greater than a preset threshold, it is determined that the positional relationship between the transformed point cloud data and the pelvic area meets the preset overlapping condition. Otherwise, it is determined that the positional relationship between the transformed point cloud data and the pelvic area does not meet the preset overlapping condition. The overlapping area ratio = the overlapping area of the detection frames of the maximum range area and the pelvic area / the area of the merged area formed by merging the detection frames of the maximum range area and the pelvic area.

[0058] If the terminal device determines that the positional relationship between the transformed point cloud data and the pelvic region meets the preset overlapping condition, the transformed point cloud data obtained in step S202 is used as the second point cloud data. If the positional relationship between the transformed point cloud data and the pelvic region does not meet the preset overlapping condition, a probe movement prompt is generated to prompt the doctor to move the probe so that the position data of the optical probe changes. When a change is detected in the position data of the optical probe collected by the tracking device, the position data is adjusted, and the transformation matrix between the tracking local coordinate system and the probe local coordinate system is updated based on the adjusted position data to obtain an updated second transformation matrix. The position of the first point cloud data in the tracking local coordinate system is updated based on the updated second transformation matrix to obtain an updated third point cloud data. The point cloud data is then transformed to the image local coordinate system based on the updated point cloud data to obtain an updated transformed point cloud data. If the positional relationship between the updated transformed point cloud data and the pelvic region does not meet the preset overlapping condition, the position data is further adjusted until the positional relationship between the transformed point cloud data and the pelvic region meets the preset overlapping condition.

[0059] In a feasible implementation manner, the above S40 may include S401 to S404:

[0060] S401 , determining that in a transformation round from the first point cloud data to the second point cloud data, the first point cloud data is correspondingly transformed into point cloud data in the tracking local coordinate system, and recording the point cloud data as fourth point cloud data.

[0061] Specifically, the process of transforming the first point cloud data into the second point cloud data may involve one or more transformation rounds. A transformation round should be understood as the process of transforming the first point cloud data into the local coordinate system of the image based on the pose data collected by the tracking device. For example, S201 to S202 described above constitute one transformation round. When determining the second point cloud data, the terminal device may save the point cloud data (i.e., the fourth point cloud data) obtained by transforming the first point cloud data in the transformation round into the local tracking coordinate system, so that it can be directly retrieved when needed.

[0062] S402 : Based on the fourth point cloud data and the pre-acquired pelvic calibration data in the pelvic reference array local coordinate system, a transformation matrix between the tracking local coordinate system and the pelvic reference array local coordinate system is calculated to obtain a fifth transformation matrix.

[0063] Among them, the coordinate data of each marker point of the pelvic array in its pelvic array local coordinate system is pre-calibrated in the pelvic array, that is, the pelvic calibration data, and the pelvic marker data is pre-stored so that it can be directly retrieved when used. The fourth point cloud data represents the first point cloud data of the pelvic model mapped to the point cloud data in the tracking local coordinate system. After the terminal device determines the fourth point cloud data of the pelvic model in the tracking local coordinate system and the pelvic calibration data in the pelvic reference array local coordinate system, it can calculate the transformation matrix between the tracking local coordinate system and the pelvic reference array local coordinate system based on the fourth point cloud data and the pelvic calibration data.

[0064] S403 , updating the transformation matrix between the tracking local coordinate system and the light needle local coordinate system based on the fourth point cloud data, the first point cloud data and the third transformation matrix to obtain a final updated second transformation matrix.

[0065] After the terminal device determines that the first point cloud data is converted into point cloud data in the tracking local coordinate system in a transformation round that meets the preset overlapping conditions, that is, the fourth point cloud data, it can update the transformation matrix between the tracking local coordinate system and the model local coordinate system through the fourth point cloud data and the first point cloud data to obtain the final updated fourth transformation matrix, and based on the final updated fourth transformation matrix and the transformation matrix between the model local coordinate system and the probe local coordinate system, update the transformation matrix between the tracking local coordinate system and the probe local coordinate system to obtain the final updated second transformation matrix.

[0066] S404 : Obtain a first registration matrix based on the third transformation matrix, the fifth transformation matrix, and the final updated second transformation matrix.

[0067] Specifically, when the third transformation matrix is specifically expressed as a transformation matrix from the model local coordinate system to the probe local coordinate system, and the fifth transformation matrix is specifically expressed as a transformation matrix from the tracking local coordinate system to the pelvic reference array local coordinate system, and the finally updated second transformation matrix is specifically expressed as a transformation matrix from the tracking local coordinate system to the probe local coordinate system, the terminal device can calculate the first registration matrix T4 by the following formula:

[0068] T4=(T2) -1 ·T3·(T1) -1 ,

[0069] Wherein, T1 is the third conversion matrix, T2 is the final updated second conversion matrix, and T3 is the fifth conversion matrix.

[0070] Based on the embodiment 1 of the present application, in the embodiment 2 of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction, and no further details will be given later. On this basis, after step S40, the acetabulum registration method further includes step S50:

[0071] S50: Acquire a first acquisition point set acquired on the pelvic surface, and perform registration calculation based on the first acquisition point set, the first point cloud data, and the first registration matrix to obtain a target registration matrix.

[0072] It is understandable that after completing the preliminary registration based on the first acquisition point set and the first registration matrix, a rough correspondence between the first point cloud data and the second point cloud data can be obtained. However, this preliminary registration can usually only achieve a rough registration of the acetabulum, and its registration accuracy still has room for improvement. In order to further improve the accuracy of the registration, the conversion relationship between the model local coordinate system and the image local coordinate system can be further optimized based on the preliminary registration results (i.e., the first registration matrix) using the acquisition points collected by the optical probe on the pelvic surface.

[0073] In some embodiments, the number of acquisition points in the first acquisition point set can be greater than a first preset threshold and less than a second preset threshold. For example, the number of acquisition points in the first acquisition point set can be 3, 4, or 5. It is understood that in the coarse registration process of the above embodiment, since no manually marked markers are used for registration, the error in coarse registration is reduced, thereby reducing the subsequent reliance on fine registration. Therefore, a smaller number of acquisition points can be used to achieve fine registration during the fine registration process.

[0074] In some embodiments, the above S50 may include: converting the first acquisition point set into the local coordinate system of the model based on the first registration matrix to obtain fifth point cloud data; performing registration calculation based on the fifth point cloud data and the first point cloud data to obtain a sixth transformation matrix; and obtaining a target registration matrix based on the sixth transformation matrix and the first registration matrix.

[0075] Specifically, the terminal device first maps the first acquisition point set to the model local coordinate system through the first registration matrix, obtains the position of the first acquisition point set in the model local coordinate system, and obtains the fifth point cloud data. Then, the terminal device performs registration calculation on the first point cloud data and the fifth point cloud data that are both in the model local coordinate system to obtain the conversion relationship between the first point cloud data and the fifth point data, that is, the sixth conversion matrix. Then, the first registration matrix is corrected by the sixth conversion matrix to obtain the target registration matrix. Among them, the target registration matrix T6 is expressed as:

[0076] T6=(T5·(T4) -1 ) -1 =T4·(T5) -1 ,

[0077] Among them, T5 is the sixth transformation matrix and T4 is the first registration matrix.

[0078] More specifically, when performing registration calculation based on the fifth point cloud data and the first point cloud data to obtain the sixth transformation matrix, an iterative closest point (ICP) algorithm can be used to register the fifth point cloud data and the first point cloud data to obtain the sixth transformation matrix.

[0079] This embodiment can improve the accuracy of acetabulum registration by further performing fine registration based on the first registration matrix obtained by coarse registration.

[0080] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the acetabulum registration method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0081] The present application provides a terminal device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the acetabulum alignment method in the above-mentioned embodiment one.

[0082] Reference below Figure 6 , which shows a schematic diagram of the structure of a terminal device suitable for implementing the embodiments of the present application. The terminal device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The terminal device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0083] like Figure 6As shown, the terminal device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the terminal device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems may be connected to I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and communication device 1009. The communication device 1009 can allow the terminal device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a terminal device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0084] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0085] The terminal device provided in this application utilizes the acetabulum registration method described in the above-mentioned embodiment to resolve the technical issue of large acetabulum registration errors. Compared to the prior art, the terminal device provided in this application offers the same beneficial effects as the acetabulum registration method described in the above-mentioned embodiment. Other technical features of the terminal device are the same as those disclosed in the above-mentioned embodiment and are not further detailed here.

[0086] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0087] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0088] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the acetabulum registration method in the above-mentioned embodiment.

[0089] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0090] The computer-readable storage medium may be included in the terminal device, or may exist independently without being incorporated into the terminal device.

[0091] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0093] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0094] The computer-readable storage medium provided herein stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned acetabulum registration method, thereby resolving the technical issue of large acetabulum registration errors. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided herein are the same as those of the acetabulum registration method provided in the aforementioned embodiments, and are not further elaborated herein.

[0095] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned acetabulum registration method when executed by a processor.

[0096] The computer program product provided in this application can solve the technical problem of large acetabulum registration errors. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the acetabulum registration method provided in the above embodiment, and will not be repeated here.

Claims

1. A method for acetabulum registration, characterized in that: The method comprises: Acquiring first point cloud data corresponding to the pelvic surface model in the local coordinate system of the model; Acquire human body image data acquired by an imaging device, wherein the human body image data is in an image local coordinate system; Performing data transformation on the first point cloud data so that a positional relationship between the first point cloud data transformed into the local image coordinate system and the pelvic region in the human body image satisfies a preset overlapping condition, and recording the transformed point cloud data that satisfies the preset overlapping condition as second point cloud data; A registration calculation is performed based on the second point cloud data and the first point cloud data to obtain a first registration matrix.

2. The acetabulum registration method according to claim 1, wherein: The performing data transformation on the first point cloud data so that a positional relationship between the first point cloud data transformed into the local image coordinate system and the pelvic region in the human body image satisfies a preset overlapping condition, and recording the transformed point cloud data that satisfies the preset overlapping condition as second point cloud data includes: Acquire pose data of the optical probe in a tracking local coordinate system collected by the tracking device, and convert the first point cloud data into the tracking local coordinate system based on the pose data to obtain third point cloud data; Based on a predetermined first transformation matrix, the third point cloud data is transformed into the image local coordinate system to obtain transformed point cloud data, wherein the first transformation matrix is a transformation matrix between the image local coordinate system and the tracking local coordinate system; If the positional relationship between the transformed point cloud data and the pelvic region does not meet the preset overlapping condition, the posture data is adjusted, and the point cloud data obtained after the first point cloud data is transformed into the local coordinate system of the image is recalculated based on the adjusted posture data, until the positional relationship between the point cloud data obtained after the transformation and the pelvic region meets the preset overlapping condition, and the point cloud data that meets the preset overlapping condition is recorded as the second point cloud data.

3. The acetabulum registration method according to claim 2, wherein: The converting the first point cloud data into the tracking local coordinate system based on the pose data to obtain third point cloud data includes: Calculating a transformation matrix between the tracking local coordinate system and the probe local coordinate system based on the pose data and pre-acquired probe calibration data in the probe local coordinate system to obtain a second transformation matrix; Based on a predetermined third transformation matrix and the second transformation matrix, a transformation matrix between the model local coordinate system and the tracking local coordinate system is calculated to obtain a fourth transformation matrix, wherein the third transformation matrix is a transformation matrix between the model local coordinate system and the probe local coordinate system; Based on the fourth transformation matrix, the first point cloud data is transformed into the tracking local coordinate system to obtain the third point cloud data.

4. The acetabulum registration method according to claim 3, wherein: The performing registration calculation based on the second point cloud data and the first point cloud data to obtain a first registration matrix includes: Determine, in a transformation round in which the first point cloud data is transformed into the second point cloud data, point cloud data corresponding to the first point cloud data converted into the tracking local coordinate system, and record the data as fourth point cloud data; Calculating a transformation matrix between the tracking local coordinate system and the pelvic reference array local coordinate system based on the fourth point cloud data and pre-acquired pelvic calibration data in the pelvic reference array local coordinate system to obtain a fifth transformation matrix; updating the transformation matrix between the tracking local coordinate system and the probe local coordinate system based on the fourth point cloud data, the first point cloud data, and the third transformation matrix to obtain a final updated second transformation matrix; The first registration matrix is obtained based on the third transformation matrix, the fifth transformation matrix and the final updated second transformation matrix.

5. The acetabulum registration method according to claim 1, wherein: After performing registration calculation based on the second point cloud data and the first point cloud data to obtain a first registration matrix, the method further includes: A first acquisition point set acquired on the pelvic surface is acquired, and a registration calculation is performed based on the first acquisition point set, the first point cloud data, and the first registration matrix to obtain a target registration matrix.

6. The acetabulum registration method according to claim 5, wherein: The performing registration calculation based on the first acquisition point set, the first point cloud data, and the first registration matrix to obtain a target registration matrix includes: Converting the first acquisition point set into the model local coordinate system based on the first registration matrix to obtain fifth point cloud data; Performing registration calculation based on the fifth point cloud data and the first point cloud data to obtain a sixth transformation matrix; The target registration matrix is obtained based on the sixth transformation matrix and the first registration matrix.

7. The acetabulum registration method according to claim 6, wherein: The performing registration calculation based on the fifth point cloud data and the first point cloud data to obtain a sixth transformation matrix includes: An iterative closest point algorithm is used to align the fifth point cloud data with the first point cloud data to obtain the sixth transformation matrix.

8. A terminal device, characterized in that: The apparatus comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the acetabulum registration method according to any one of claims 1 to 7.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the acetabulum registration method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the acetabulum registration method according to any one of claims 1 to 7.

Images