Medical image registration method and device, computer equipment and storage medium

Abstract

The invention relates to a medical image registration method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring a three-dimensional image and a two-dimensional image of a target part, and determining an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image comprises an initial first segmentation line of the target part; determining an initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and determining a moved first segmentation line according to the obtained moved second segmentation line in response to a movement operation for the initial second segmentation line; obtaining a registration error between the two-dimensional image and the three-dimensional image according to the offset between the moved first segmentation line and the initial first segmentation line; and updating the initial registration result according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image. By adopting the method, the initial registration result can be corrected, and the accuracy of medical image registration is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a medical image registration method, device, computer equipment and storage medium. Background Art

[0002] The C-arm is a medical imaging device that can capture high-quality fluoroscopic images in real time. C-arm registration can align the fluoroscopic images captured by the C-arm with other medical images to achieve spatial alignment between the two, thereby providing support for interventional surgery and radiological diagnosis.

[0003] Taking the registration of preoperative CT (Computed Tomography) images with intraoperative fluoroscopic images as an example, in the prior art, a reference point is usually selected at the affected area, and the data of the reference point in the preoperative CT image and the intraoperative C-arm fluoroscopic image is registered to achieve image registration. This method has low registration accuracy and can easily lead to inaccurate C-arm registration.

[0004] Therefore, the current medical image registration technology has the problem of low accuracy. Summary of the invention

[0005] Based on this, it is necessary to provide a medical image registration method, apparatus, computer equipment, computer-readable storage medium and computer program product with high accuracy to address the above technical problems.

[0006] In a first aspect, the present application provides a medical image registration method. The method comprises:

[0007] Acquire a three-dimensional image and a two-dimensional image of the target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part;

[0008] According to the initial registration result, an initial second segmentation line corresponding to the initial first segmentation line is determined, and in response to a movement operation on the initial second segmentation line, a moved first segmentation line is determined according to the obtained moved second segmentation line;

[0009] Obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line;

[0010] The initial registration result is updated according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0011] In a second aspect, the present application also provides a medical image registration device. The device comprises:

[0012] An acquisition module, used to acquire a three-dimensional image and a two-dimensional image of a target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image contains an initial first segmentation line of the target part;

[0013] A moving module, configured to determine an initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and determine a moved first segmentation line according to the obtained moved second segmentation line in response to a moving operation on the initial second segmentation line;

[0014] An error module, used for obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line;

[0015] The registration module is used to update the initial registration result according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0016] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0017] Acquire a three-dimensional image and a two-dimensional image of the target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part;

[0018] According to the initial registration result, an initial second segmentation line corresponding to the initial first segmentation line is determined, and in response to a movement operation on the initial second segmentation line, a moved first segmentation line is determined according to the obtained moved second segmentation line;

[0019] Obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line;

[0020] The initial registration result is updated according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0022] Acquire a three-dimensional image and a two-dimensional image of the target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part;

[0023] According to the initial registration result, an initial second segmentation line corresponding to the initial first segmentation line is determined, and in response to a movement operation on the initial second segmentation line, a moved first segmentation line is determined according to the obtained moved second segmentation line;

[0024] Obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line;

[0025] The initial registration result is updated according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0026] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0027] Acquire a three-dimensional image and a two-dimensional image of the target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part;

[0028] According to the initial registration result, an initial second segmentation line corresponding to the initial first segmentation line is determined, and in response to a movement operation on the initial second segmentation line, a moved first segmentation line is determined according to the obtained moved second segmentation line;

[0029] Obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line;

[0030] The initial registration result is updated according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0031] The above-mentioned medical image registration method, device, computer equipment, storage medium and computer program product obtain a three-dimensional image and a two-dimensional image of the target part, determine the initial registration result between the two-dimensional image and the three-dimensional image, determine the initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and in response to the movement operation on the initial second segmentation line, determine the moved first segmentation line according to the obtained moved second segmentation line, obtain the registration error between the two-dimensional image and the three-dimensional image according to the offset between the moved first segmentation line and the initial first segmentation line, and update the initial registration result according to the registration error to obtain the target registration result between the two-dimensional image and the three-dimensional image; after determining the initial registration result, according to the initial registration result, the segmentation line in the three-dimensional image can be transformed into the segmentation line in the two-dimensional image, after moving the segmentation line in the two-dimensional image, it is transformed back to the segmentation line in the three-dimensional image, and the initial registration result is updated according to the offset of the segmentation line in the three-dimensional image before and after the movement, so as to obtain a more accurate target registration result, thereby improving the accuracy of medical image registration between the three-dimensional image and the two-dimensional image. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of a process of a medical image registration method in one embodiment;

[0033] Figure 2 is a schematic diagram of a CT image of a vertebral body and a segmentation line in one embodiment;

[0034] Figure 3 is a schematic diagram of a perspective image of a vertebral body and a segmentation line in one embodiment;

[0035] Figure 4 is a schematic diagram of characteristic points of a vertebral body in one embodiment;

[0036] Figure 5 is a schematic diagram of characteristic points of a vertebral body in a perspective image in one embodiment;

[0037] Figure 6 is a schematic diagram of an image registration system in one embodiment;

[0038] Figure 7 A schematic diagram of perspective image shooting in one embodiment;

[0039] Figure 8 A schematic diagram of a perspective image captured by an image acquisition device in one embodiment;

[0040] Fig. 9 is a schematic flow chart of a medical image registration method in another embodiment;

[0041] Fig.10 is a schematic diagram of coarse registration in one embodiment;

[0042] Fig.11 is a schematic diagram of image superposition in one embodiment;

[0043] Fig.12 is a structural block diagram of a medical image registration device in one embodiment;

[0044] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] In one embodiment, Figure 1 As shown, a medical image registration method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0047] Step S110, acquiring a three-dimensional image and a two-dimensional image of the target part, and determining an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part.

[0048] The target part may be a part that needs to be registered with medical images.

[0049] The three-dimensional image may be a three-dimensional medical image, for example, a three-dimensional CT image.

[0050] The two-dimensional image may be a two-dimensional medical image, for example, two two-dimensional X-ray images (fluoroscopic images) obtained by photographing the same target part from two different viewing angles.

[0051] The initial first segmentation line may be an initial segmentation line of the target part in the three-dimensional image.

[0052] In a specific implementation, a three-dimensional image and a two-dimensional image that need to be registered can be obtained for the target part, wherein the two-dimensional image can be two two-dimensional images obtained by photographing the target part from two different perspectives. Afterwards, the two-dimensional image and the three-dimensional image can be registered to obtain an initial registration result between the two-dimensional image and the three-dimensional image. The target part can also be segmented in the three-dimensional image to obtain an initial first segmentation line in the three-dimensional image.

[0053] Figure 2A schematic diagram of the CT image of the vertebra and the segmentation line is provided. Figure 2 , the human spinal vertebrae can be used as the target part, the preoperative CT image of the vertebrae can be used as the three-dimensional image to be registered, and the C-arm can be used to collect the perspective image of the vertebrae in real time during the operation as the two-dimensional image to be registered. The CT image and the perspective image can be roughly registered to obtain the mapping relationship between the two, and the mapping relationship can be used as the initial registration result. The vertebrae in the CT image can also be segmented to obtain the segmentation plane between the vertebrae, and two points can be selected from the segmentation plane to obtain the three-dimensional data of the two points, and the line between the two points can be used as the initial first segmentation line, such as Figure 2 As shown, two points A and B are selected from the dividing plane between the first vertebra and the second vertebra, and the connecting line AB is used as the initial first dividing line.

[0054] Step S120, determining an initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and determining a moved first segmentation line according to the obtained moved second segmentation line in response to a moving operation on the initial second segmentation line.

[0055] The initial second segmentation line may be an initial segmentation line of the target part in the two-dimensional image.

[0056] The moved second segmentation line may be a segmentation line of the target part in the two-dimensional image obtained after moving the initial second segmentation line in the two-dimensional image.

[0057] The moved first segmentation line may be a segmentation line of the target part in the three-dimensional image obtained by moving the initial second segmentation line in the two-dimensional image.

[0058] In a specific implementation, the initial first segmentation line in the three-dimensional image can be converted to the two-dimensional image according to the initial registration result to obtain the initial second segmentation line in the two-dimensional image. The initial second segmentation line in the two-dimensional image can also be moved to obtain the moved second segmentation line in the two-dimensional image. According to the initial registration result, the moved second segmentation line in the two-dimensional image is converted to the three-dimensional image to obtain the moved first segmentation line in the three-dimensional image.

[0059] Figure 3 A perspective image of the vertebra and a schematic diagram of the segmentation lines are provided. Figure 3 The C-arm can take real-time photos of the vertebral body from two different perspectives during surgery, obtaining two perspective images of the vertebral body, for example, the anteroposterior and lateral views of the vertebral body. Figure 2In the case of three-dimensional data of two points A and B, point A is projected into two perspective images according to the initial registration result to obtain a1 and a2 in the perspective images. Similarly, point B is projected into two perspective images according to the initial registration result to obtain b1 and b2 in the perspective images. Among them, a1, a2, b1, and b2 are all two-dimensional data. Then a1 and b1 can be connected to obtain the initial second segmentation line a1b1 in the frontal film, and a2 and b2 can be connected to obtain the initial second segmentation line a2b2 in the lateral film. When the segmentation of a1b1 and / or a2b2 is inaccurate, the inaccurate segmentation line can be moved so that it can just segment the first vertebra and the second vertebra. During the movement, the segmentation line is synchronously updated in the two perspective images, and the second segmentation lines a1'b1' and a2'b2' are obtained after the movement. According to the initial alignment result, a1', b1', a2', and b2' are projected back to the CT image, and the segmentation line A'B' between the first vertebra and the second vertebra in the CT image is recalculated and used as the first segmentation line after the movement.

[0060] Step S130, obtaining a registration error between the two-dimensional image and the three-dimensional image according to the offset between the first segmentation line after the movement and the initial first segmentation line.

[0061] In a specific implementation, after obtaining the moved first segmentation line in the three-dimensional image, the offset between the moved first segmentation line and the initial first segmentation line in the three-dimensional image can be determined, and the registration error between the two-dimensional image and the three-dimensional image can be determined based on the offset between the moved first segmentation line and the initial first segmentation line.

[0062] For example, the Euclidean distance between AB and A'B' may be calculated, and the Euclidean distance may be used as the registration error between the CT image and the fluoroscopic image.

[0063] Step S140 , updating the initial registration result according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0064] In a specific implementation, the initial registration result can be updated according to the registration error between the two-dimensional image and the three-dimensional image to obtain the target registration result between the two-dimensional image and the three-dimensional image.

[0065] For example, if the Euclidean distance between AB and A'B' is less than or equal to the preset distance threshold, it means that the registration error is small, and the initial registration result can be directly fine-aligned to obtain the target registration result; otherwise, if the Euclidean distance between AB and A'B' is greater than the preset distance threshold, return to step S110, and perform coarse alignment on the CT image and the fluoroscopic image again to obtain the initial registration result between the two, and detect the registration error until the registration error is less than or equal to the preset distance threshold, and then perform fine alignment on the final initial registration result to obtain the target registration result of the CT image and the fluoroscopic image.

[0066] The above-mentioned medical image registration method obtains a three-dimensional image and a two-dimensional image of the target part, determines the initial registration result between the two-dimensional image and the three-dimensional image, determines the initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and in response to the movement operation of the initial second segmentation line, determines the moved first segmentation line according to the obtained moved second segmentation line, obtains the registration error between the two-dimensional image and the three-dimensional image according to the offset between the moved first segmentation line and the initial first segmentation line, and updates the initial registration result according to the registration error to obtain the target registration result between the two-dimensional image and the three-dimensional image; after determining the initial registration result, according to the initial registration result, the segmentation line in the three-dimensional image can be transformed into the segmentation line in the two-dimensional image, and after moving the segmentation line in the two-dimensional image, it is transformed back to the segmentation line in the three-dimensional image, and the initial registration result is updated according to the offset of the segmentation line in the three-dimensional image before and after the movement, so as to obtain a more accurate target registration result, thereby improving the accuracy of medical image registration between the three-dimensional image and the two-dimensional image.

[0067] In one embodiment, the above step S110 may specifically include: determining a first feature point from a three-dimensional image, and determining a second feature point corresponding to the first feature point in the three-dimensional image from a two-dimensional image; performing three-dimensional coordinate conversion processing on the second feature point to obtain a third feature point; and aligning the first feature point and the third feature point to obtain an initial alignment result.

[0068] The first feature point may be three-dimensional data of the feature point of the target part in the three-dimensional image. The second feature point may be two-dimensional data of the feature point of the target part in the two-dimensional image. The third feature point may be three-dimensional data obtained by performing three-dimensional coordinate conversion on the second feature point.

[0069] In a specific implementation, the feature points of the target part can be determined, and the three-dimensional data of the feature point in the three-dimensional image can be used as the first feature point, and the two-dimensional data of the feature point in the two-dimensional image can be used as the second feature point. The second feature point can be converted into a three-dimensional coordinate to obtain a third feature point. The first feature point and the third feature point can then be aligned to obtain an initial alignment result between the two-dimensional image and the three-dimensional image.

[0070] Figure 4 A schematic diagram of the characteristic points of a vertebra is provided. Figure 4 , three feature points ①②③ as shown in the figure can be selected on the vertebral body. In the CT image, these three feature points are identified manually or automatically to obtain the first feature point, which are recorded as CT_Point_1(X1,Y1,Z1), CT_Point_2(X2,Y2,Z2), and CT_Point_3(X3,Y3,Z3) respectively.

[0071] Figure 5 A schematic diagram of the characteristic points of the vertebral body in a perspective image is provided. Figure 5 For the characteristic points ①②③, they can be identified manually or automatically in the anteroposterior and lateral views of the vertebral body, and the characteristic point Xray_Point_1_A (u 1A ,v 1A ), Xray_Point_2_A(u 2A ,v 2A ), Xray_Point_3_A(u 3A ,v 3A ), and the characteristic point Xray_Point_1_a(u 1a ,v 1a ), Xray_Point_2_a(u 2a ,v 2a ), Xray_Point_3_a(u 3a ,v 3a ), the feature points in the anteroposterior film and the feature points in the lateral film are taken as the second feature points, and based on the principle of epipolar geometry, the three-dimensional data of feature points ①②③ in the C-arm external parameter coordinate system are calculated according to the second feature points to obtain the third feature points. The numerical calculation method of singular value decomposition is used to calculate the coarse registration matrix between the first feature point and the third feature point, that is, the initial registration result.

[0072] In this embodiment, by determining a first feature point from a three-dimensional image, and determining a second feature point from a two-dimensional image corresponding to the first feature point in the three-dimensional image; performing three-dimensional coordinate conversion processing on the second feature point to obtain a third feature point; and aligning the first feature point and the third feature point to obtain an initial alignment result, the three-dimensional image and the two-dimensional image can be roughly aligned to improve the efficiency of medical image alignment.

[0073] In one embodiment, the two-dimensional image includes a first two-dimensional image and a second two-dimensional image; the above-mentioned step of determining the second feature point corresponding to the first feature point in the three-dimensional image from the two-dimensional image may specifically include: determining the second feature point in the first two-dimensional image in response to a point selection operation on the first two-dimensional image; generating a feature point auxiliary line in the second two-dimensional image based on the second feature point in the first two-dimensional image; and determining the second feature point in the second two-dimensional image in response to a point selection operation on the feature point auxiliary line.

[0074] The first two-dimensional image and the second two-dimensional image may be two two-dimensional images obtained by photographing the target part from two different viewing angles.

[0075] In a specific implementation, the first two-dimensional image and the second two-dimensional image of the target part can be displayed on the terminal. The feature points of the target part can be selected on the first two-dimensional image to obtain the second feature points in the first two-dimensional image. Based on the second feature points in the first two-dimensional image, feature point auxiliary lines can be generated in the second two-dimensional image. Then, the feature points of the target part can be selected on the feature point auxiliary lines of the second two-dimensional image to obtain the second feature points in the second two-dimensional image.

[0076] For example, Figure 5 As shown, the characteristic points ①②③ of the first vertebra are selected in the anteroposterior film to obtain the second characteristic point in the anteroposterior film as shown in the figure. Accordingly, based on the coordinate mapping relationship between the anteroposterior film and the lateral film, three auxiliary lines passing through the characteristic points of the first vertebra are generated in the lateral film according to the second characteristic point in the anteroposterior film, and the auxiliary lines can be selected to obtain the second characteristic point in the lateral film. Similarly, the characteristic points can be selected in the lateral film first, and the corresponding characteristic point auxiliary lines can be generated in the anteroposterior film, and the characteristic points in the anteroposterior film can be determined according to the characteristic point auxiliary lines.

[0077] In this embodiment, the second feature point in the first two-dimensional image is determined in response to a point selection operation on the first two-dimensional image; a feature point auxiliary line in the second two-dimensional image is generated based on the second feature point in the first two-dimensional image; the second feature point in the second two-dimensional image is determined in response to a point selection operation on the feature point auxiliary line, and the second feature point in the second two-dimensional image can be selected on the feature point auxiliary line according to the prompt of the feature point auxiliary line, thereby improving the efficiency of determining the second feature point.

[0078] In one embodiment, the above-mentioned step of performing three-dimensional coordinate conversion processing on the second feature point to obtain the third feature point may specifically include: determining the intrinsic parameters and extrinsic parameters of the image acquisition device of the two-dimensional image; the intrinsic parameters are used to characterize the coordinate mapping relationship between the two-dimensional image and the image acquisition device, and the extrinsic parameters are used to characterize the coordinate mapping relationship between the image acquisition device and the robotic arm; according to the intrinsic parameters and the extrinsic parameters, performing three-dimensional coordinate conversion processing on the second feature point to obtain the third feature point.

[0079] The image acquisition device may be a two-dimensional image acquisition device, for example, an X-ray machine installed on a C-arm.

[0080] The intrinsic parameter may be a coordinate transformation matrix between the two-dimensional image and the image acquisition device.

[0081] Among them, the external parameters can be the coordinate transformation matrix between the image acquisition device and the robot arm base.

[0082] In a specific implementation, for an image acquisition device for a two-dimensional image, its pre-set internal parameters can be obtained, and its external parameters can be determined. The projection matrix between the three-dimensional image and the two-dimensional image can be determined based on the internal and external parameters. The second feature points on the two two-dimensional images can be transformed into three-dimensional coordinates based on the projection matrix to obtain the third feature points represented by three-dimensional data.

[0083] In practical applications, projection matrices M1 and M2 can be calculated for two 2D images respectively, where M i =T 内参 *T 外参 , i = 1, 2, where T 内参 and T 外参 Represents internal and external parameters respectively, T 内参 It is a matrix obtained based on the inherent properties of the C-arm. It is the factory setting of the C-arm and represents the conversion relationship between the C-arm fluoroscopic image and the C-arm radiation source.

[0084] When the second feature point Xray_Point_1_A(u 1A ,v 1A ), Xray_Point_2_A(u 2A ,v 2A ), Xray_Point_3_A(u 3A ,v 3A ), Xray_Point_1_a(u 1a ,v 1a ), Xray_Point_2_a(u 2a ,v 2a ), Xray_Point_3_a(u 3a ,v3a ), two projection formulas are established

[0085]

[0086]

[0087] Among them, (u1, v1, 1) and (u2, v2, 1) are the homogeneous coordinates of the projection points of the feature points in the CT image on the two perspective images, that is, the second feature point, (X, Y, Z, 1) is the homogeneous coordinates of the feature points in the CT image in the external parameter coordinates, that is, the third feature point, Z c1 and Z c2 is a constant,

[0088]

[0089]

[0090] Eliminate Z c1 and Z c2 , we get four linear equations about X, Y, and Z

[0091]

[0092] By solving the linear equations simultaneously and the equations, the three-dimensional coordinates of the second characteristic point can be obtained, and the third characteristic point can be obtained. In practical applications, due to the presence of noise, the least square method can be used to solve the above equations.

[0093] In this embodiment, the intrinsic parameters and extrinsic parameters of the image acquisition device of the two-dimensional image are determined; the intrinsic parameters are used to characterize the coordinate mapping relationship between the two-dimensional image and the image acquisition device, and the extrinsic parameters are used to characterize the coordinate mapping relationship between the image acquisition device and the robotic arm; according to the intrinsic parameters and the extrinsic parameters, the second feature point is converted into a three-dimensional coordinate to obtain a third feature point. The two-dimensional feature points in the two two-dimensional images can be converted into three-dimensional feature points, which can be compared with the feature points in the three-dimensional image to determine the registration error and improve the accuracy of image registration.

[0094] In one embodiment, the robotic arm includes a calibration target, a flange device and a base; the above-mentioned step of determining the intrinsic parameters and extrinsic parameters of the image acquisition device of the two-dimensional image may specifically include: determining a first conversion matrix between the image acquisition device and the calibration target, a second conversion matrix between the calibration target and the flange device, and a third conversion matrix between the flange device and the base; and obtaining the extrinsic parameters of the image acquisition device according to the product of the first conversion matrix, the second conversion matrix and the third conversion matrix.

[0095] The first conversion matrix may be a conversion matrix between a camera coordinate system of an image acquisition device and a calibration target coordinate system. The second conversion matrix may be a conversion matrix between a calibration target coordinate system and a flange coordinate system. The third conversion matrix may be a conversion matrix between a flange coordinate system and a robot base coordinate system.

[0096] In a specific implementation, a robotic arm for performing surgical operations on a target part and an image acquisition device for acquiring a two-dimensional image of the target part in real time during the operation can be set, wherein a flange device is provided at the end of the robotic arm, the flange device is connected to a calibration target through an adapter, a marking ball is provided on the calibration target, the image acquisition device is controlled to shoot the calibration target, and the image acquisition device is calibrated according to the two-dimensional image obtained by shooting to obtain the external parameters of the image acquisition device. Specifically, the first conversion matrix between the camera coordinate system of the image acquisition device and the calibration target coordinate system, the second conversion matrix between the calibration target coordinate system and the flange coordinate system, and the third conversion matrix between the flange coordinate system and the robotic arm base coordinate system can be determined, and the first conversion matrix, the second conversion matrix and the third conversion matrix are multiplied to obtain the external parameters of the image acquisition device.

[0097] Figure 6 A schematic diagram of an image registration system is provided. Figure 6 The image registration system may include a robotic arm 201, an adapter 202, a calibration target 203, a C-arm 204, a fluoroscopic image 205 and a CT image 206, wherein the calibration target 203 and the adapter 202 form a quick-release structure, and a marking ball is provided on the calibration target 203. The user may control the robotic arm 201 to perform surgery on the affected area, and may also control the C-arm 204 to take fluoroscopic images of the affected area from multiple viewing angles, wherein each fluoroscopic image contains the marking ball in the calibration target 203.

[0098] Figure 7 A schematic diagram of perspective imaging is provided. Figure 7 The calibration target 203 is mounted on the adapter 202 of the robot arm 201, the perspective plane of the C arm 204 is parallel or approximately parallel to the target surface of the C arm, and the affected part is located at the center of the perspective plane and the target surface.

[0099] Figure 8 A schematic diagram of a perspective image captured by an image acquisition device is provided. Figure 8 The anteroposterior and lateral views of the spinal vertebrae taken by the image acquisition device are shown in the figure. Both the anteroposterior and lateral views are marked with the marking ball in the calibration target 203.

[0100] In practical applications, the calculation formula for the external parameters of the C-arm is:

[0101]

[0102] in, Represents the transformation matrix from the C-arm ray source to the robotic arm base, i.e., the external parameters. Indicates the position of the end flange of the robot arm relative to the base of the robot arm, which can be directly obtained by the robot arm control software. Represents the rigid body transformation from the calibration target to the robot flange, which is an inherent property determined by the mechanical dimensions. represents the transformation matrix from the C-arm ray source to the calibration target. A calculation example is obtained by the PNP (Perspective-n-Point) method in photogrammetry, that is, the three-dimensional coordinates of the center of the marking ball on the calibration target are measured by the measuring equipment, and the Figure 8 The two-dimensional coordinates of the center of the marking ball on the calibration target are extracted, and then the three-dimensional coordinates and two-dimensional coordinates of the center of the marking ball are calculated.

[0103] In this embodiment, by determining the first conversion matrix between the image acquisition device and the calibration target, the second conversion matrix between the calibration target and the flange device, and the third conversion matrix between the flange device and the base; according to the product of the first conversion matrix, the second conversion matrix and the third conversion matrix, the external parameters of the image acquisition device are obtained, and the coordinate transformation matrix between the image acquisition device and the robot arm base can be obtained, and the three-dimensional data corresponding to the two-dimensional perspective image can be obtained, the registration error is determined, and the accuracy of image registration is improved.

[0104] In one embodiment, the above-mentioned step S140 may specifically include: comparing the registration error with a preset threshold; when the registration error is greater than the preset threshold, determining a new second feature point from the two-dimensional image, performing three-dimensional coordinate transformation processing on the new second feature point, determining the registration error for the new registration result between the obtained new third feature point and the first feature point, and returning to the step of comparing the registration error with the preset threshold until the registration error is no greater than the preset threshold; and obtaining the target registration result based on the new registration result.

[0105] The preset threshold may be a preset threshold of the registration error.

[0106] In a specific implementation, the registration error can be compared with a preset threshold. If the registration error is less than or equal to the preset threshold, the initial registration result can be directly subjected to fine registration to obtain the target registration result. Otherwise, if the registration error is greater than the preset threshold, the process returns to the step of determining the initial registration result between the two-dimensional image and the three-dimensional image. For the first feature point in the three-dimensional image, a new second feature point corresponding to the first feature point is determined from the two-dimensional image, the new second feature point is transformed into a three-dimensional coordinate to obtain a new third feature point, the first feature point is registered with the new third feature point to obtain a new registration result, and the registration error is determined based on the new registration result until the registration error is less than or equal to the preset threshold, and the new registration result is subjected to fine registration to obtain the target registration result.

[0107] For example, according to the aforementioned embodiment, the Euclidean distance between AB and A'B' is determined as the registration error. If the Euclidean distance between AB and A'B' is less than or equal to the preset distance threshold, the initial registration result is directly determined as the target registration result; otherwise, if the Euclidean distance between AB and A'B' is greater than the preset distance threshold, return to step S110, perform coarse registration on the CT image and the fluoroscopic image again, obtain the initial registration result between the two, and detect the registration error until the registration error is less than or equal to the preset distance threshold, and use the final initial registration result as the target registration result of the CT image and the fluoroscopic image.

[0108] In this embodiment, the registration error is compared with a preset threshold; when the registration error is greater than the preset threshold, a new second feature point is determined from the two-dimensional image, and the new second feature point is transformed into a three-dimensional coordinate. For the new registration result between the new third feature point and the first feature point, the registration error is determined, and the process returns to the step of comparing the registration error with the preset threshold until the registration error is no greater than the preset threshold; the target registration result is obtained based on the new registration result, and the registration error can be reduced by repeating the coarse registration multiple times to obtain a target registration result with higher accuracy.

[0109] In one embodiment, after the above-mentioned step S140, it may further specifically include: determining an adjacent part of the target part; acquiring a three-dimensional image and a two-dimensional image of the adjacent part; including an initial first segmentation line of the adjacent part in the three-dimensional image of the adjacent part; using the target registration result as the initial registration result of the adjacent part, and returning to the step of determining an initial second segmentation line corresponding to the initial first segmentation line based on the initial registration result, so as to determine the target registration result between the three-dimensional image and the two-dimensional image of the adjacent part.

[0110] The adjacent part may be a part adjacent to the target part.

[0111] In a specific implementation, a three-dimensional image and a two-dimensional image of a part adjacent to the target part can be obtained, the target registration result can be used as an initial registration result of the adjacent part, and an initial first segmentation line of the adjacent part in the three-dimensional image can be obtained. According to the initial registration result, the initial first segmentation line can be transformed to an initial second segmentation line in the two-dimensional image, and the initial second segmentation line can be moved to obtain a moved second segmentation line. According to the initial registration result, the moved second segmentation line can be transformed back to the moved first segmentation line in the three-dimensional image, and according to the offset between the moved first segmentation line and the initial first segmentation line, a registration error between the three-dimensional image and the two-dimensional image of the adjacent part can be obtained. According to the registration error, the initial registration result can be updated to obtain a target registration result between the three-dimensional image and the two-dimensional image of the adjacent part.

[0112] For example, according to the above steps S110-S140, the CT image and the perspective image of the first vertebra are aligned to obtain a precise alignment result of the first vertebra, and the precise alignment result of the first vertebra is used as the initial alignment result of the second vertebra. According to the above steps S120-S140, the CT image and the perspective image of the second vertebra are aligned to obtain a precise alignment result of the second vertebra, and the precise alignment result of the second vertebra is used as the initial alignment result of the third vertebra. According to the above steps S120-S140, the CT image and the perspective image of the third vertebra are aligned to obtain a precise alignment result of the third vertebra, and this process is repeated to obtain precise alignment results of all vertebrae.

[0113] In this embodiment, by determining the adjacent part of the target part; obtaining a three-dimensional image and a two-dimensional image of the adjacent part; the three-dimensional image of the adjacent part includes an initial first segmentation line of the adjacent part; using the target registration result as the initial registration result of the adjacent part, and returning to the step of determining the initial second segmentation line corresponding to the initial first segmentation line based on the initial registration result, so as to determine the target registration result between the three-dimensional image and the two-dimensional image of the adjacent part, the target registration results of multiple target parts can be obtained by determining the initial registration result once, thereby improving the efficiency of medical image registration.

[0114] In order to facilitate those skilled in the art to have a deeper understanding of the embodiments of the present application, a specific example will be described below.

[0115] In the field of spinal surgery, intraoperative C-arms can be used to register patients. The registration process is divided into three stages: 1) Perform a preoperative CT scan on the patient and formulate a surgical plan; 2) Use the intraoperative C-arm to perform a perspective on the affected area and obtain two perspective images of the affected area at different angles; 3) Align the preoperative CT image with the intraoperative C-arm perspective image. During the alignment process, it is necessary to first determine the approximate position of the affected area, that is, rough image alignment, and then perform fine image alignment. Rough image alignment determines the success rate of fine image alignment. This application takes the spinal surgery navigation system as an example and proposes several rough image alignment implementation examples.

[0116] like Figure 6 As shown, the spinal surgery navigation system may include a surgical robot 201, an adapter 202 (calibration target mounting base), a calibration target 203 (forming a quick-release structure with the adapter), a C-arm 204, a fluoroscopic image 205, and CT image data 206. Figure 6 The user can operate the C-arm 204 to obtain perspective images of the affected part from multiple perspectives. At the same time, each perspective image will contain the marking points in the calibration target 203.

[0117] Fig. 9 A flowchart of a medical image registration method is provided. Fig. 9 , the medical image registration method may include the following steps:

[0118] Step S301, scanning the patient's preoperative CT image;

[0119] Step S302, segmenting the vertebral body that needs surgery;

[0120] Specifically, the preoperative CT images can be segmented manually or automatically, or automatically first, and then the automatic segmentation results can be fine-tuned to obtain the final segmentation results of the vertebrae. For example, the spinal surgery navigation system first automatically segments the vertebrae, and then the user confirms the automatic segmentation results. If the segmentation is accurate, the automatic segmentation results are directly used as the final segmentation results. If the segmentation is inaccurate, the automatic segmentation results can be manually modified.

[0121] Step S303, automatically identifying at least 3 feature points on the segmented vertebral body;

[0122] Specifically, the spinal surgery navigation system can automatically identify three feature points from the preoperative CT image and recommend the user to select them. The user can select these three feature points or add other feature points and select three of them for the next step. Figure 4 As shown, the feature points on the three surfaces of the pedicle of the vertebra that needs to be roughly registered are identified, and these three feature points are defined at the center of the surface.

[0123] Step S304, installing a calibration target at the end of the robotic arm, dragging the calibration target, and ensuring that when the C-arm acquires perspective images of the affected vertebra from two perspectives (e.g., anteroposterior and lateral views), the marking points in the calibration target also appear in the perspective images;

[0124] Among them, a C-arm fluoroscopic solution is as follows Figure 7 As shown, the calibration target 203 is installed on the adapter tool 202 of the robot arm, and the perspective plane of the C arm is approximately parallel and centered with the target surface of the C arm. Figure 8As shown, the left side is an AP radiograph of a spinal vertebra, and the right side is a lateral radiograph of a spinal vertebra. Both images are marked with marking points in the calibration target 203 .

[0125] Step S305, perform rough image registration and calculate the transformation matrix from the CT image coordinate system CTFrame to the robot base coordinate system BaseFrame, that is,

[0126] The rough registration process may include the following steps:

[0127] a) Select a vertebra to be registered, and manually or automatically identify at least three feature points of the pedicle on the CT data, CT_Point_1 (X1, Y1, Z1), CT_Point_2 (X2, Y2, Z2), and CT_Point_3 (X3, Y3, Z3), such as Figure 5 As shown;

[0128] b) Select 2 perspective images, such as Figure 8 , select the corresponding feature points on CT in these two images, such as Figure 5 After selecting a feature point on any image, an auxiliary line will appear on the other image. The user can select another feature point on the auxiliary line according to the prompt of the auxiliary line. Figure 5 An example schematic diagram of point selection is given in the figure. The corresponding feature points selected on the two images are Xray_Point_1_A(u 1A ,v 1A ), Xray_Point_1_a(u 1a ,v 1a ), Xray_Point_2_A(u 2A ,v 2A ,Xray_Point_2_a(u 2a, v 2a ), Xray_Point_3_A(u 3A ,v 3A ), Xray_Point_3_a(u 3a ,v 3a ).

[0129] c) According to the mathematical principle of epipolar geometry in photogrammetry, the 3D coordinates of the selected feature points in the C-arm external parameter coordinate system can be calculated, that is, the point pair Xray_Point_1_A(u 1A ,v 1A ), Xray_Point_1_a(u 1a ,v 1a ), Xray_Point_2_A(u 2A,v 2A ), Xray_Point_2_a(u 2a, v 2a ), Xray_Point_3_A(u 3A ,v 3A ), Xray_Point_3_a(u 3a ,v 3a ) calculates the 3D coordinates of the feature points in the C-arm external parameter coordinate system. The external parameter coordinate system refers to the reference coordinate system for the position and posture measurement of the calibration target ③. In this example, the external parameter coordinate system refers to the robot arm base coordinate system. The calculation formula for the C-arm external parameters can be

[0130]

[0131] in, Represents the transformation matrix from the C-arm ray source to the robotic arm base, that is, the external parameter matrix. Indicates the position of the flange at the end of the robot arm, which can be directly obtained by the robot arm control software. Represents the rigid body transformation from the calibration target to the robot flange, which is an inherent property determined by the mechanical dimensions. One example of calculation can be calculated by the PNP method in photogrammetry, that is, using the measuring equipment to measure the 3D coordinates of the center of the characteristic sphere on the calibration target, and Figure 8 The 2D coordinates of the corresponding feature ball extracted from

[0132] In order to calculate the 3D coordinates of the feature points in the external parameter coordinate system, it is necessary to calculate the projection matrix M of the C-arm during the two perspectives, where M = T 内参 *T 外参 ; where T 内参 It is a 3*4 matrix of the inherent properties of the C-arm. It is a mathematical expression of the conversion relationship between the C-arm fluoroscopic image and the radiation source. It is the factory property of the C-arm. 外参 The above method can be used to calculate, which is a 4*4 matrix. The projection matrices of the two C-arm perspectives are M1 and M2, which is a 3*4 matrix. Then the two cameras can establish two projection formulas

[0133]

[0134]

[0135] Among them, (u1, v1, 1) and (u2, v2, 1) are the homogeneous coordinates of the projection points of the CT feature points on the two view images, and (X, Y, Z, 1) is the homogeneous coordinates of the CT feature points in the external parameter coordinates. c1 and Z c2, we get four linear equations about X, Y, and Z

[0136]

[0137] By combining these four equations, the 3D coordinates of the feature points can be obtained. In practical applications, since data always has noise, the least squares method can be used to solve it.

[0138] d) Based on a set of corresponding 3D point coordinates obtained in step a) and step c), a numerical calculation method of singular value decomposition (svd) can be used to calculate the rough registration matrix of the current vertebra, which can be expressed as That is, the transformation matrix from the CT coordinate system to the robot base coordinate system.

[0139] It should be noted that if Fig.10 As shown, the patient's body position can also be corrected according to the patient's CT image so that the patient's body position matches the body position in the CT image, and the navigation probe is used to obtain the patient's corrected body position information, and a feature point is selected in the CT image to determine the feature point information corresponding to the feature point in the patient's corrected body position information, and the feature point in the CT image and the feature point information corresponding to the feature point in the patient's corrected body position information are aligned to obtain a coarse alignment matrix.

[0140] You can also stick marker points on the patient's body surface, perform a CT scan on the patient, obtain a preoperative CT image, extract the marker points in the preoperative CT image, and obtain the spatial position of the marker points during the operation. Rough alignment is performed based on the spatial positions of the marker points in the preoperative CT image and the marker points during the operation to obtain a rough alignment matrix.

[0141] Step S306, evaluating the accuracy of the rough registration, if it is not satisfied, returning to step S305;

[0142] Specifically, a segmentation line of the vertebra can be drawn according to the rough registration result, and the user judges the current rough registration effect according to the segmentation effect of the segmentation line. The specific steps may include:

[0143] e) Segment the vertebrae in the CT image and obtain any set of 3D points on the segmentation plane, such as Figure 2 As shown, A and B are a set of 3D points on the segmentation plane respectively;

[0144] f) Using the formula

[0145]

[0146]

[0147] Project the coordinates of point A (X, Y, Z) onto the corresponding two perspective images, denoted as a1 (u1, v1) and a2 (u2, v2). Similarly, point B can be projected into b1 and b2. Draw a1, a2, b1, and b2 corresponding to each vertebra on the perspective image, such as Figure 3 As shown, users can evaluate the accuracy of coarse registration based on the segmentation effect. In principle, the higher the coarse registration accuracy, the higher the success rate of subsequent fine registration.

[0148] Step S307, performing precise registration of the vertebral image;

[0149] Step S308, iteratively registering other vertebrae;

[0150] Specifically, the DRR (Digitally Reconstructed Radiograph) projection algorithm can be used to project the CT image into a "quasi-perspective image" according to the coarse registration matrix, and iteratively register according to the grayscale overlap to obtain the fine registration matrix and the final superposition effect image, such as Fig.10 As shown, the user can drag Fig.11 Use the CT–Xray slider below to see the degree of overlap between the DRR projection image and the true perspective image.

[0151] Step S309, evaluating the registration accuracy of all vertebrae, and repeating step S305 for vertebrae that do not meet the registration accuracy, until all vertebrae meet the registration accuracy;

[0152] Specifically, the fine registration result of the current vertebra may be used as the coarse registration result of the adjacent vertebra, and then the process returns to step S307 to perform fine registration until all vertebrae meet the registration requirements.

[0153] The above-mentioned medical image registration method can complete patient registration non-invasively. Compared with the traditional registration method, it can avoid secondary infection during surgery. It can also iteratively register other vertebrae by completing the coarse registration of a single vertebra. In addition, a coarse registration scheme based on image point selection and interactive point selection is proposed.

[0154] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0155] Based on the same inventive concept, the embodiment of the present application also provides a medical image registration device for implementing the medical image registration method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more medical image registration device embodiments provided below can refer to the limitations of the medical image registration method above, and will not be repeated here.

[0156] In one embodiment, Fig.12 As shown, a medical image registration device is provided, including: an acquisition module 410, a movement module 420, an error module 430 and a registration module 440, wherein:

[0157] An acquisition module 410 is used to acquire a three-dimensional image and a two-dimensional image of a target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part;

[0158] A moving module 420 is used to determine an initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and determine a moved first segmentation line according to the obtained moved second segmentation line in response to a moving operation on the initial second segmentation line;

[0159] An error module 430, configured to obtain a registration error between the two-dimensional image and the three-dimensional image according to an offset between the moved first segmentation line and the initial first segmentation line;

[0160] The registration module 440 is used to update the initial registration result according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

[0161] In one embodiment, the acquisition module 410 is also used to determine a first feature point from the three-dimensional image, and to determine a second feature point from the two-dimensional image corresponding to the first feature point in the three-dimensional image; perform three-dimensional coordinate conversion on the second feature point to obtain a third feature point; and align the first feature point and the third feature point to obtain the initial alignment result.

[0162] In one embodiment, the above-mentioned acquisition module 410 is also used to determine the second feature point in the first two-dimensional image in response to a point selection operation on the first two-dimensional image; generate a feature point auxiliary line in the second two-dimensional image based on the second feature point in the first two-dimensional image; and determine the second feature point in the second two-dimensional image in response to a point selection operation on the feature point auxiliary line.

[0163] In one embodiment, the acquisition module 410 is also used to determine the intrinsic parameters and extrinsic parameters of the image acquisition device of the two-dimensional image; the intrinsic parameters are used to characterize the coordinate mapping relationship between the two-dimensional image and the image acquisition device, and the extrinsic parameters are used to characterize the coordinate mapping relationship between the image acquisition device and the robotic arm; according to the intrinsic parameters and the extrinsic parameters, the second feature point is subjected to three-dimensional coordinate conversion processing to obtain the third feature point.

[0164] In one embodiment, the acquisition module 410 is also used to determine a first transformation matrix between the image acquisition device and the calibration target, a second transformation matrix between the calibration target and the flange device, and a third transformation matrix between the flange device and the base; and obtain the external parameters of the image acquisition device according to the product of the first transformation matrix, the second transformation matrix and the third transformation matrix.

[0165] In one embodiment, the above-mentioned registration module 440 is also used to compare the registration error with a preset threshold; when the registration error is greater than the preset threshold, a new second feature point is determined from the two-dimensional image, and the new second feature point is transformed into a three-dimensional coordinate. For the new registration result between the obtained new third feature point and the first feature point, the registration error is determined, and the process returns to the step of comparing the registration error with the preset threshold until the registration error is no greater than the preset threshold; and the target registration result is obtained according to the new registration result.

[0166] In one embodiment, the medical image registration device further includes:

[0167] An adjacent part determination module, used for determining an adjacent part of the target part;

[0168] A neighboring part acquisition module, used to acquire a three-dimensional image and a two-dimensional image of the neighboring part; the three-dimensional image of the neighboring part includes an initial first segmentation line of the neighboring part;

[0169] The adjacent part registration module is used to use the target registration result as the initial registration result of the adjacent part, and return to the step of determining the initial second segmentation line corresponding to the initial first segmentation line based on the initial registration result to determine the target registration result between the three-dimensional image and the two-dimensional image of the adjacent part.

[0170] Each module in the above-mentioned medical image registration device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0171] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.13 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a medical image registration method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0172] Those skilled in the art will understand that Fig.13The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0173] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0174] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0175] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0178] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A medical image registration method, characterized in that: The method comprises: Acquire a three-dimensional image and a two-dimensional image of the target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image includes an initial first segmentation line of the target part; According to the initial registration result, an initial second segmentation line corresponding to the initial first segmentation line is determined, and in response to a movement operation on the initial second segmentation line, a moved first segmentation line is determined according to the obtained moved second segmentation line; Obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line; The initial registration result is updated according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

2. The method according to claim 1, characterized in that The determining of an initial registration result between the two-dimensional image and the three-dimensional image comprises: Determine a first feature point from the three-dimensional image, and determine a second feature point from the two-dimensional image corresponding to the first feature point in the three-dimensional image; Performing three-dimensional coordinate conversion processing on the second feature point to obtain a third feature point; The first feature point and the third feature point are aligned to obtain the initial alignment result.

3. The method according to claim 2, characterized in that The two-dimensional image includes a first two-dimensional image and a second two-dimensional image; and determining the second feature point corresponding to the first feature point in the three-dimensional image from the two-dimensional image includes: In response to a point selection operation on the first two-dimensional image, determining a second feature point in the first two-dimensional image; generating a feature point auxiliary line in the second two-dimensional image according to the second feature point in the first two-dimensional image; In response to a point selection operation on the feature point auxiliary line, a second feature point in the second two-dimensional image is determined.

4. The method according to claim 2, characterized in that: The performing three-dimensional coordinate conversion processing on the second feature point to obtain a third feature point includes: Determine the intrinsic parameters and extrinsic parameters of the image acquisition device of the two-dimensional image; the intrinsic parameters are used to characterize the coordinate mapping relationship between the two-dimensional image and the image acquisition device, and the extrinsic parameters are used to characterize the coordinate mapping relationship between the image acquisition device and the robotic arm; According to the intrinsic parameters and the extrinsic parameters, three-dimensional coordinate transformation is performed on the second feature point to obtain the third feature point.

5. The method according to claim 4, characterized in that The mechanical arm includes a calibration target, a flange device and a base; the intrinsic parameters and extrinsic parameters of the image acquisition device for determining the two-dimensional image include: Determine a first conversion matrix between the image acquisition device and the calibration target, a second conversion matrix between the calibration target and the flange device, and a third conversion matrix between the flange device and the base; The extrinsic parameters of the image acquisition device are obtained according to the product of the first conversion matrix, the second conversion matrix and the third conversion matrix.

6. The method according to claim 2, characterized in that The updating process of the initial registration result according to the registration error to obtain the target registration result between the two-dimensional image and the three-dimensional image includes: comparing the registration error with a preset threshold; In the case where the registration error is greater than the preset threshold, a new second feature point is determined from the two-dimensional image, a three-dimensional coordinate conversion process is performed on the new second feature point, a registration error is determined for a new registration result between the obtained new third feature point and the first feature point, and the process returns to the step of comparing the registration error with the preset threshold until the registration error is no greater than the preset threshold; The target registration result is obtained according to the new registration result.

7. The method according to claim 1, characterized in that After the initial registration result is updated according to the registration error to obtain the target registration result between the two-dimensional image and the three-dimensional image, the method further includes: determining a site adjacent to the target site; Acquire a three-dimensional image and a two-dimensional image of the adjacent part; the three-dimensional image of the adjacent part includes an initial first segmentation line of the adjacent part; The target registration result is used as the initial registration result of the adjacent part, and the step of determining the initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result is returned to determine the target registration result between the three-dimensional image and the two-dimensional image of the adjacent part.

8. A medical image registration device, characterized in that: The device comprises: An acquisition module, used to acquire a three-dimensional image and a two-dimensional image of a target part, and determine an initial registration result between the two-dimensional image and the three-dimensional image; the three-dimensional image contains an initial first segmentation line of the target part; A moving module, configured to determine an initial second segmentation line corresponding to the initial first segmentation line according to the initial registration result, and determine a moved first segmentation line according to the obtained moved second segmentation line in response to a moving operation on the initial second segmentation line; An error module, used for obtaining a registration error between the two-dimensional image and the three-dimensional image according to an offset between the first segmentation line after the movement and the initial first segmentation line; The registration module is used to update the initial registration result according to the registration error to obtain a target registration result between the two-dimensional image and the three-dimensional image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images