Inter-image planning point synchronization device, electronic device, storage medium and related methods

Abstract

The application provides an inter-image planning point synchronization device, an electronic device, a storage medium and related methods. The method comprises: obtaining a first coordinate of a first planning point in a first image and a second coordinate of the first planning point in a second image; obtaining a third coordinate of a second planning point in the first image and a fourth coordinate of the second planning point in the second image; determining a fifth coordinate of the second planning point in the first image after position updating; determining corresponding alternative coordinates of the second planning point after position updating in the second image according to the first coordinate, the second coordinate, the fifth coordinate and a spatial distance; and determining a sixth coordinate of the second planning point in the second image after position updating from at least one alternative coordinate according to a distance between image points corresponding to the third coordinate and the fifth coordinate and a distance between image points corresponding to the fourth coordinate and the alternative coordinates. The inter-image planning point synchronization method provided by the application has high applicability.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of image processing, and in particular to an image inter-planning point synchronization device, an electronic device, a storage medium and related methods. BACKGROUND

[0002] Medical robots are used to assist physicians in medical institutions such as hospitals and clinics, and are intelligent service robots that can generate an operation plan according to the physician's preoperative planning of the implant path, determine the action program according to the operation plan, and then convert the action into the movement of the operation mechanism, thereby assisting physicians in interventional therapy.

[0003] Currently, when preoperative planning is performed, two images of the lesion in different directions need to be taken, and the planning points are planned synchronously according to the two images, that is, after the planning points are determined on one two-dimensional image, the corresponding planning points are determined in the other two-dimensional image through epipolar constraint, so as to realize the path planning of the implant.

[0004] However, the physician adjusts the planning points on one two-dimensional image to the planning points on the other two-dimensional image, and the length of the implant changes with the position of the planning points in the two two-dimensional images, so that the length of the implant during planning may exceed a reasonable range, resulting in errors after planning adjustment, and thus the existing image inter-planning point synchronization method has low applicability. SUMMARY

[0005] Therefore, the present application provides an image inter-planning point synchronization device, an electronic device, a storage medium and related methods to at least partially solve the above problems.

[0006] According to a first aspect of the present application, an image inter-planning point synchronization method is provided, which includes: obtaining a first coordinate of a first planning point in a first image and a second coordinate of the first planning point in a second image; obtaining a third coordinate of a second planning point in the first image and a fourth coordinate of the second planning point in the second image; determining a fifth coordinate of the second planning point in the first image after position updating, wherein the spatial distance between the second planning point and the first planning point before and after position updating is equal; determining at least one candidate coordinate of the second planning point after position updating in the second image according to the first coordinate, the second coordinate, the fifth coordinate and the spatial distance; and determining a sixth coordinate of the second planning point in the second image after position updating from the at least one candidate coordinate according to the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the image points corresponding to the fourth coordinate and each of the candidate coordinates.

[0007] In a possible implementation, the method further includes: determining the spatial coordinates of the first planning point according to the first coordinates and the second coordinates; determining the at least one spatial coordinate of the second planning point after the position update according to the spatial coordinates of the first planning point, the fifth coordinates, and the spatial distance, where a distance between the spatial coordinates of the first planning point and the spatial coordinates of the second planning point after the position update is equal to the spatial distance; and determining the at least one candidate coordinates corresponding to the second planning point after the position update in the second image according to the at least one spatial coordinate of the second planning point after the position update.

[0008] In a possible implementation, the method further includes: constructing a spherical equation (x-a) 2 +(y-b) 2 +(z-c) 2 =R 2 according to the spatial coordinates of the first planning point and the spatial distance, where (a, b, c) is used to represent the spatial coordinates of the first planning point, R is used to represent the spatial distance, and (x, y, z) is used to represent the spatial coordinates of a point on the spherical surface; determining a straight line equation of a straight line passing through the fifth coordinates and a spatial straight line of a shooting source of the first image; and calculating the at least one spatial coordinate of the second planning point after the position update according to the spherical equation and the straight line equation.

[0009] In a possible implementation, the method further includes: calculating the at least one candidate coordinates corresponding to the second planning point after the position update by using a projection equation according to the at least one spatial coordinate of the second planning point after the position update, where the projection equation includes:

[0010]

[0011] Z c is used to represent the depth of the at least one spatial coordinate of the second planning point after the position update in a camera coordinate system of a shooting source of the second image, f x is used to represent a focal length of the shooting source of the second image in the x-axis direction, f y is used to represent a focal length of the shooting source of the second image in the y-axis direction, and (u0, v0) is used to represent coordinates of an intersection between an optical axis of the shooting source of the second image and an imaging plane of the shooting source of the second image in a pixel coordinate system.3X3 Used to characterize the rotation matrix, T 3X1 Used to characterize the translation matrix, (X) w ,Y w Z w (x, y) is used to represent the updated spatial coordinates of the second planning point, and (x, y) is used to represent the alternative coordinates.

[0012] In one possible implementation, determining the sixth coordinate of the second planned point in the second image after the position update from the at least one candidate coordinate, based on the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates, includes: calculating the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates; if the distance between the third coordinate and the image points corresponding to the fifth coordinate is less than a preset first distance threshold, then the candidate coordinate with the smallest distance between the corresponding image point and the fourth coordinate is determined as the sixth coordinate.

[0013] According to a second aspect of this application, an implant position planning method is provided, the method comprising: acquiring a third image and a fourth image taken from different angles; determining a seventh coordinate of a first end of the implant in the third image and determining an eighth coordinate of the first end in the fourth image; determining a ninth coordinate of a second end of the implant in the third image and determining a tenth coordinate of the second end in the fourth image; determining an eleventh coordinate of the second end in the third image after the position update, wherein the spatial distance between the second end and the first end before and after the position update is equal to the length of the implant; determining at least one alternative coordinate in the fourth image corresponding to the position update of the second end based on the seventh coordinate, the eighth coordinate, the eleventh coordinate, and the length of the implant; and determining a twelfth coordinate of the second end in the fourth image from the at least one alternative coordinate based on the distance between the image points corresponding to the ninth coordinate and the eleventh coordinate, and the distance between the tenth coordinate and the image points corresponding to each of the alternative coordinates.

[0014] According to a third aspect of this application, an inter-image planning point synchronization device is provided. The device includes: a first acquisition module, configured to acquire a first coordinate of a first planning point in a first image and acquire a second coordinate of the first planning point in a second image; a second acquisition module, configured to acquire a third coordinate of the second planning point in the first image and acquire a fourth coordinate of the second planning point in the second image; a first determination module, configured to determine a fifth coordinate of the second planning point in the first image after the position update, wherein the spatial distance between the second planning point and the first planning point is equal before and after the position update; a second determination module, configured to determine at least one candidate coordinate corresponding to the second planning point after the position update in the second image based on the first coordinate, the second coordinate, the fifth coordinate, and the spatial distance; and a third determination module, configured to determine a sixth coordinate of the second planning point in the second image after the position update from the at least one candidate coordinate based on the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates.

[0015] According to a fourth aspect of this application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, the executable instruction causing the processor to perform the method as described in the first or second aspect of this application.

[0016] According to a fifth aspect of this application, a computer storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described in the first or second aspect of this application.

[0017] According to a sixth aspect of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform the method as described in the first or second aspect of this application.

[0018] According to the image planning point synchronization method provided in this application, the projection relationship between two images is determined by the coordinates of the first and second planning points in the two images. Then, based on the projection relationship and the updated coordinates of the second planning point in the first image, the updated coordinates of the second planning point in the second image can be determined, thus realizing the synchronization of planning points between images. That is, changing the position of a planning point in one image will also synchronize the adjustment in the other image, avoiding errors in planning results caused by manual adjustment of planning points in two images. Furthermore, since the planning points in the two images are synchronized according to the projection relationship, the length between planning points in the spatial coordinate system can be fixed during planning, making this method applicable to schemes that require fixed lengths for planning. Therefore, this image planning point synchronization method has high applicability. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0020] Figure 1 This is a flowchart of a method for synchronizing planning points between images provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a projection relationship provided in an embodiment of this application;

[0022] Figure 3 This is a flowchart of an implant placement planning method provided in an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an image planning point synchronization device provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of an implant placement planning device provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0027] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0028] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0029] Figure 1 This is a flowchart of an image planning point synchronization method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps 101 to 105:

[0030] Step 101: Obtain the first coordinates of the first planning point in the first image, and obtain the second coordinates of the first planning point in the second image.

[0031] The first image and the second image are two two-dimensional images of the same object taken from different directions. For example, the first image is a front view of the object, and the second image is a left view of the object. The position of the first planning point in the first image and the position in the second image correspond to the position of the first planning point in the object. Due to the different shooting angles of the first image and the second image, the first planning point is located in different positions in the first image and the second image, but both are located in the same part of the object image included in the first image and the second image.

[0032] It should be understood that the first planning point needs to be determined manually. The location of the first planning point is determined manually according to the needs. For example, in the preoperative planning of a screw placement surgery, the doctor determines the position of the screw head based on X-rays taken from two angles. After determining the position, the doctor marks the position as the first planning point in the two images. This first planning point corresponds to the planned position of the screw head in the human body.

[0033] Step 102: Obtain the third coordinate of the second planning point in the first image, and obtain the fourth coordinate of the second planning point in the second image.

[0034] Obtain the coordinates of the second planning point in the first image and the coordinates in the second image. The method for determining the second planning point is similar to that for determining the first planning point, and will not be repeated here.

[0035] Step 103: Determine the fifth coordinate of the second planning point in the first image after the position update, wherein the spatial distance between the second planning point and the first planning point is equal before and after the position update.

[0036] When the position of the second planning point is unreasonable, the coordinates of the second planning point in the first image after manual adjustment are obtained. For example, during the planning of a screw placement surgery, the second planning point corresponds to the tip of the bone screw. When the position of the screw tip is unreasonable, the doctor will adjust the position of the screw tip in the first image to make the overall position of the bone screw more reasonable. The distance between the coordinates of the second planning point before and after the update in the first image and the spatial point projected by the first planning point in space is equal to the distance between the first planning point and the spatial point projected by the second planning point. That is, updating the position of the second planning point does not change the distance between the first planning point and the second planning point in the spatial coordinate system.

[0037] It should be understood that after the position of the second planning point in the first image is updated, the distance between the first coordinate and the third coordinate in the first image is not necessarily equal to the distance between the first coordinate and the fifth coordinate. However, when projected into space, the spatial distance between the second planning point and the first planning point is equal before and after the position update.

[0038] It should also be understood that since the second planning point is modified only from the first image, and no physical objects are adjusted in the corresponding space during the planning process, the second image will not automatically change according to the planning point on the first image.

[0039] Step 104: Based on the first coordinate, the second coordinate, the fifth coordinate, and the spatial distance, determine at least one alternative coordinate corresponding to the updated position of the second planning point in the second image.

[0040] The spatial coordinates of the first planning point can be determined based on the first and second coordinates. Then, at least one spatial coordinate that is equidistant from the first planning point can be determined based on the fifth coordinate and the spatial distance. At least one alternative coordinate corresponding to the updated position of the second planning point in the second image can be determined based on the at least one spatial coordinate that is equidistant from the first planning point.

[0041] Step 105: Based on the distance between the image points corresponding to the third and fifth coordinates, and the distance between the fourth coordinate and the image points corresponding to each candidate coordinate, determine the sixth coordinate in the second image after the second planning point position is updated from at least one candidate coordinate.

[0042] Based on the distance between the coordinates of the second planning point before and after the update in the first image, and based on the fourth coordinate of the second planning point in the second image and the distance between each alternative coordinate of the second planning point in the second image after the update, the sixth coordinate of the second planning point in the second image after the update is determined.

[0043] In this embodiment, the projection relationship between two images is determined by the coordinates of the first and second planning points in the two images. Then, based on the projection relationship and the updated coordinates of the second planning point in the first image, the updated coordinates of the second planning point in the second image can be determined, thus achieving synchronization of planning points between images. That is, changing the position of a planning point in one image will also result in a synchronous adjustment in the other image, avoiding errors in planning results caused by manual adjustment of planning points in two images. Furthermore, since the planning points in the two images are synchronized according to the projection relationship, the length between planning points in the spatial coordinate system can be fixed during planning, making this method applicable to schemes that require fixed lengths for planning. Therefore, this method for synchronizing planning points between images has high applicability.

[0044] In one possible implementation, when determining at least one alternative coordinate corresponding to the updated position of the second planning point in the second image based on the first coordinate, the second coordinate, the fifth coordinate, and the spatial distance, the spatial coordinates of the first planning point can be determined based on the first coordinate and the second coordinate. Then, at least one alternative coordinate corresponding to the updated position of the second planning point can be determined based on the spatial coordinates of the first planning point, the fifth coordinate, and the spatial distance. The distance between the spatial coordinates of the first planning point and the updated spatial coordinates of the second planning point is equal to the spatial distance. Finally, at least one alternative coordinate corresponding to the updated position of the second planning point can be determined in the second image based on the updated spatial coordinates of the second planning point.

[0045] Based on the spatial coordinates of the shooting source of the first image and the first coordinates of the first planning point in the first image, a spatial straight line passing through the shooting source and the first coordinates of the first image can be determined. Based on the spatial coordinates of the shooting source of the second image and the second coordinates of the first planning point in the second image, a spatial straight line passing through the shooting source and the second planning point in the second image can be determined. The intersection of the two spatial straight lines is determined as the spatial coordinates of the first planning point. It should be noted that there are multiple ways to determine the spatial coordinates of the first planning point. This application only provides one example and does not limit the specific method for determining the spatial coordinates of the first planning point in this application.

[0046] Based on the spatial coordinates of the first planning point and the fifth coordinate, at least one spatial coordinate in the spatial coordinate system corresponding to the fifth coordinate and whose distance from the spatial coordinates of the first planning point is equal to the spatial distance can be determined. This spatial coordinate is a possible spatial coordinate after the position of the second planning point is updated. Based on the spatial coordinates after the position of the second planning point is updated, at least one alternative coordinate is determined in the second image. This alternative coordinate is used to indicate the possible coordinates in the second image after the position of the second planning point is fundamentally updated.

[0047] The source of the first image and the source of the second image can be imaging equipment such as a camera, a C-arm X-ray machine, and ultrasound, and this application does not limit them.

[0048] It should be understood that, due to the different shooting angles of the images, the positions of the first image source when shooting the first image and the second image source when shooting the second image are not fixed, and different spatial coordinates can be determined by applying different determination methods. Therefore, the spatial coordinates of the first planning point are not fixed.

[0049] In this embodiment, the distance between the spatial coordinates of the first planning point and the spatial coordinates of the second planning point before and after the position update is equal to the spatial distance. Therefore, the length between planning points in the spatial coordinate system can be fixed during planning. At least one spatial coordinate of the second planning point after the position update is determined by the fifth coordinate in the first image, and the coordinates of the second planning point in the second image after the update are solved based on the spatial coordinates. This can improve the accuracy of the coordinates of the second planning point after the update in the second image, realize the synchronization of planning points between images, and improve the accuracy and applicability of the planning point synchronization method between images.

[0050] Figure 2 This is a schematic diagram of a projection relationship provided in an embodiment of this application, such as... Figure 2 As shown, when determining at least one spatial coordinate K2 of the updated position of the second planning point based on the spatial coordinates K1 of the first planning point, the fifth coordinate L5, and the spatial distance, the following spherical equation can be constructed based on the spatial coordinates K1 of the first planning point and the spatial distance.

[0051] (xa) 2 +(yb) 2 +(zc) 2 =R 2

[0052] Where (a,b,c) is used to represent the spatial coordinates K1 of the first planning point, R is used to represent the spatial distance, and (x,y,z) is used to represent the spatial coordinates of the point on the sphere 201. Then, the equation of the straight line 202 passing through the fifth coordinate L5 and the shooting source of the first image is determined, and at least one spatial coordinate K2 after the position of the second planning point is updated is calculated based on the sphere equation and the straight line equation.

[0053] Using the spatial coordinates K1(a,b,c) of the first planning point as the center of the sphere and the spatial distance R as the radius, construct the spherical equation. Then, construct the spatial straight line equation based on the spatial coordinates of the first image's shooting source and the updated fifth coordinate L5 of the second planning point in the first image. Solve the spatial straight line equation and the spherical equation simultaneously to obtain at least one spatial coordinate K2. This spatial coordinate K2 is the spatial position that the second planning point can be in after the update.

[0054] It should be understood that solving the equations of the constructed spatial straight line and the constructed sphere simultaneously can result in three cases. The first case is two solutions, proving that the updated second planning point may be in two different positions. The second case is one solution, proving that the updated second planning point is in that position. The third case is no solution, proving that the spatial distance between the updated first planning point and the updated second planning point exceeds the range, in which case the planned position of the second planning point is unreasonable and needs to be replanned.

[0055] In this embodiment, by constructing the spherical equation and the spatial straight line equation and solving them simultaneously, at least one spatial coordinate K2 after the second planning point position is updated can be obtained. This realizes the projection of the two-dimensional coordinates in the first image T1 onto the spatial coordinate system, so that each candidate coordinate LB in the second image T2 can be solved through the spatial coordinates. Furthermore, the spatial coordinate system can fix the spatial distance between the first planning point and the second planning point, making the method applicable to planning that requires a fixed distance and improving the applicability of the planning point synchronization method between images.

[0056] In one possible implementation, when determining at least one alternative coordinate LB corresponding to the updated position of the second planning point in the second image based on at least one spatial coordinate K2 updated by the second planning point, at least one alternative coordinate LB corresponding to the updated position of the second planning point can be calculated based on each spatial coordinate K2 updated by the second planning point using the following projection equation.

[0057] The projection equations include:

[0058]

[0059] Z c f is used to characterize the depth of each spatial coordinate after the second planning point is updated in the camera coordinate system of the second image's capture source. x f is used to characterize the focal length of the shooting source in the x-axis direction for the second image. y The focal length of the source of the second image along the y-axis is used to characterize the focal length of the source of the second image. (u0, v0) is used to characterize the coordinates of the intersection point of the optical axis of the source of the second image and the imaging plane of the source of the second image in the pixel coordinate system. R 3X3 Used to characterize the rotation matrix, T 3X1 Used to characterize the translation matrix, (X) w ,Y w Z w (x,y) is used to represent the updated spatial coordinates of the second planning point, and (x,y) is used to represent the alternative coordinates.

[0060] In this embodiment, by substituting the updated spatial coordinates K2 of the second planning point into the projection equation, multiple alternative coordinates LB of the second planning point on the second image can be solved, thereby realizing the projection of the spatial coordinates on the spatial coordinate system onto the two-dimensional coordinates on the second image T2. Since the spatial coordinates on the spatial coordinate system are generated by the planning points on the first image T1, the synchronization of planning points between the first image T1 and the second image T2 is achieved.

[0061] In one possible implementation, such as Figure 2 As shown, when determining the sixth coordinate in the second image T2 after the second planning point position update from at least one candidate coordinate LB, based on the distance between the image points corresponding to the third coordinate L3 and the fifth coordinate L5, and the distance between the fourth coordinate L4 and the image points corresponding to each candidate coordinate LB, the distance between the fourth coordinate L4 and the image points corresponding to each candidate coordinate LB can be calculated respectively. If the distance between the image points corresponding to the third coordinate L3 and the fifth coordinate L5 is less than a preset first distance threshold, then the candidate coordinate LB with the smallest distance between its corresponding image point and the fourth coordinate L4 is determined as the sixth coordinate.

[0062] When the position of the second planning point moves only a small distance, such as Figure 2The third coordinate L3 and the fifth coordinate L5 in the second image will only move a small distance synchronously and will not change abruptly. Therefore, if the distance between the image points corresponding to the third coordinate L3 and the fifth coordinate L5 is less than the preset first distance threshold, the candidate coordinate with the smallest distance between the corresponding image point and the fourth coordinate L4 will be determined as the sixth coordinate. The sixth coordinate is the coordinate in the second image T2 after the second planning point is updated.

[0063] It should be understood that if the distance between the third coordinate L3 and the fifth coordinate L5 is large, that is, the second planning point moves a large distance, the system will divide the process into a series of small processes. In each small process, the distance between the third coordinate L3 and the fifth coordinate L5 is very small, which is much smaller than the first distance threshold. Furthermore, the system processing is a continuous process. Therefore, the above processing is also applicable to the case where the distance between the third coordinate L3 and the fifth coordinate L5 is large.

[0064] In this embodiment, by determining whether the distance between the third coordinate L3 and the fifth coordinate L5 in the first image T1 before and after each second planning point is updated is less than a first threshold, and calculating the distance between the fourth coordinate L4 of the second planning point in the second image T2 and each candidate coordinate LB, the coordinates of the second planning point in the second image T2 after the update can be determined. This process can accurately filter out the correct candidate coordinates, avoiding the situation where the prediction result is wrong due to the selection of incorrect candidate coordinates. Thus, the coordinates of the second planning point in the second image T2 after the update can be determined based on the coordinates of the second planning point in the first image T1 after the update, realizing the synchronization of planning points between images.

[0065] Figure 3 This is a flowchart of an implant placement planning method provided in an embodiment of this application, such as... Figure 3 As shown, the method includes the following steps 301 to 306:

[0066] Step 301: Obtain the third and fourth images taken from different angles.

[0067] During preoperative planning, third and fourth images of the lesion site where the implant needs to be placed are obtained from different angles using an X-ray machine.

[0068] It should be understood that implants generally need to be inserted into the body, so the imaging equipment is usually an X-ray machine, including but not limited to: C-arm X-ray machine, MRI equipment, etc.

[0069] Step 302: Determine the seventh coordinate of the first end of the implant in the third image, and determine the eighth coordinate of the first end in the fourth image.

[0070] Determine the corresponding coordinates of the implant head in the third and fourth images.

[0071] Step 303: Determine the ninth coordinate of the second end of the implant in the third image, and determine the tenth coordinate of the second end in the fourth image.

[0072] Determine the corresponding coordinates of the implant tail in the third and fourth images.

[0073] Step 304: Determine the eleventh coordinate in the third image after the position of the second end is updated, where the spatial distance between the second end and the first end before and after the position update is equal to the length of the implant.

[0074] Determine the coordinates of the implant tail in the third image after it has been moved.

[0075] It should be understood that since the length of the implant does not change during the positioning process, the distance between the spatial coordinates of the first end and the second end of the implant is a fixed distance, which is the length of the implant.

[0076] Step 305: Based on the seventh, eighth, and eleventh coordinates and the length of the implant, determine at least one alternative coordinate in the fourth image corresponding to the updated position of the second end.

[0077] Step 306: Based on the distance between the image points corresponding to the ninth and eleventh coordinates, and the distance between the tenth coordinate and the image points corresponding to each candidate coordinate, determine the twelfth coordinate in the fourth image after the second end position is updated from at least one candidate coordinate.

[0078] In this embodiment, the projection relationship between the two images is determined by the coordinates of the first and second ends of the implant in the two images. Then, based on the projection relationship and the coordinates of the second end of the implant in the third image after position update, the coordinates of the second end of the implant in the fourth image after position update can be determined. This achieves synchronization of the planning points between the two images. That is, if the position of the implant tip is changed in one image, the other image will also be adjusted synchronously. Thus, the position of the implant can be determined based on the two images to determine whether the position of the implant meets the treatment requirements. This avoids the situation where the planning result is incorrect due to the doctor manually adjusting the position of the implant in the two images. Therefore, the implant position planning method has high applicability.

[0079] It should be noted that, Figure 3 The implant position planning method shown is a specific application of the image planning point synchronization method in the embodiments of this application. For the specific implant position planning method, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0080] Figure 4 This is a schematic diagram of an image planning point synchronization device provided in an embodiment of this application, such as... Figure 4 As shown, the image-to-image planning point synchronization device 400 includes:

[0081] The first acquisition module 401 is used to acquire the first coordinates of the first planning point in the first image and the second coordinates of the first planning point in the second image.

[0082] The second acquisition module 402 is used to acquire the third coordinate of the second planning point in the first image and the fourth coordinate of the second planning point in the second image.

[0083] The first determining module 403 is used to determine the fifth coordinate of the second planning point in the first image after the position of the second planning point is updated, wherein the spatial distance between the second planning point and the first planning point is equal before and after the position update.

[0084] The second determining module 404 is used to determine at least one alternative coordinate corresponding to the updated position of the second planning point in the second image based on the first coordinate, the second coordinate, the fifth coordinate and the spatial distance.

[0085] The third determining module 405 is used to determine the sixth coordinate in the second image after the second planning point position is updated, based on the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the fourth coordinate and the image points corresponding to each candidate coordinate.

[0086] In this embodiment of the application, the first acquisition module 401 can be used to perform the above step 101, the second acquisition module 402 can be used to perform the above step 102, the first determination module 403 can be used to perform the above step 103, the second determination module 404 can be used to perform the above step 104, and the third determination module 405 can be used to perform the above step 105.

[0087] In one possible implementation, the second determining module 404 is used to determine the spatial coordinates of the first planning point based on the first coordinates and the second coordinates; determine at least one spatial coordinate of the updated position of the second planning point based on the spatial coordinates of the first planning point, the fifth coordinate, and the spatial distance, wherein the distance between the spatial coordinates of the first planning point and the updated spatial coordinates of the second planning point is equal to the spatial distance; and determine at least one alternative coordinate corresponding to the updated position of the second planning point in the second image based on the updated spatial coordinates of the second planning point.

[0088] In one possible implementation, the second determining module 404 is used to construct the following spherical equation (xa) based on the spatial coordinates and spatial distance of the first planning point. 2 +(yb) 2+(zc) 2 =R 2 (a,b,c) is used to represent the spatial coordinates of the first planning point, R is used to represent the spatial distance, and (x,y,z) is used to represent the spatial coordinates of a point on the sphere; determine the equation of the straight line passing through the fifth coordinate and the spatial source of the first image; calculate at least one spatial coordinate of the updated position of the second planning point based on the sphere equation and the straight line equation.

[0089] In one possible implementation, the second determining module 404 is used to calculate at least one alternative coordinate corresponding to the updated position of the second planning point based on the updated spatial coordinates of the second planning point using the following projection equation; wherein the projection equation includes:

[0090]

[0091] Z c f is used to characterize the depth of each spatial coordinate after the second planning point is updated in the camera coordinate system of the second image's capture source. x f is used to characterize the focal length of the shooting source in the x-axis direction for the second image. y The focal length of the source of the second image along the y-axis is used to characterize the focal length of the source of the second image. (u0, v0) is used to characterize the coordinates of the intersection point of the optical axis of the source of the second image and the imaging plane of the source of the second image in the pixel coordinate system. R 3X3 Used to characterize the rotation matrix, T 3X1 Used to characterize the translation matrix, (X) w ,Y w Z w (x,y) is used to represent the updated spatial coordinates of the second planning point, and (x,y) is used to represent the alternative coordinates.

[0092] In one possible implementation, the third determining module 405 is used to calculate the distance between the fourth coordinate and the image points corresponding to each candidate coordinate; if the distance between the third coordinate and the image points corresponding to the fifth coordinate is less than a preset first distance threshold, then the candidate coordinate with the smallest distance between the corresponding image point and the fourth coordinate is determined as the sixth coordinate.

[0093] Figure 5 This is a schematic diagram of an implant placement planning device provided in an embodiment of this application, as shown below. Figure 5 As shown, the implant placement planning device 500 includes:

[0094] The third acquisition module 501 is used to acquire the third and fourth images taken from different angles.

[0095] The fourth determining module 502 is used to determine the seventh coordinate of the first end of the implant in the third image and the eighth coordinate of the first end in the fourth image.

[0096] The fifth determining module 503 is used to determine the ninth coordinate of the second end of the implant in the third image and the tenth coordinate of the second end in the fourth image.

[0097] The sixth determining module 504 is used to determine the eleventh coordinate in the third image after the position of the second end is updated, wherein the spatial distance between the second end and the first end before and after the position update is equal to the length of the implant.

[0098] The seventh determining module 505 is used to determine at least one alternative coordinate in the fourth image corresponding to the updated position of the second end, based on the seventh coordinate, the eighth coordinate, the eleventh coordinate and the length of the implant.

[0099] The eighth determining module 506 is used to determine the twelfth coordinate in the fourth image after the second end position is updated from at least one candidate coordinate based on the distance between the image points corresponding to the ninth coordinate and the eleventh coordinate, and the distance between the tenth coordinate and the image points corresponding to each candidate coordinate.

[0100] In this embodiment of the application, the third acquisition module 501 can be used to perform the above step 301, the fourth determination module 502 can be used to perform the above step 302, the fifth determination module 503 can be used to perform the above step 303, the sixth determination module 504 can be used to perform the above step 304, the seventh determination module 505 can be used to perform the above step 305, and the eighth determination module 506 can be used to perform the above step 306.

[0101] It should be noted that the information interaction and execution process between the modules in the above-mentioned image planning point synchronization device and implant position planning device are based on the same concept as the aforementioned image planning point synchronization method embodiment. For details, please refer to the description in the aforementioned image planning point synchronization method embodiment, and it will not be repeated here.

[0102] Reference Figure 6 This document illustrates a schematic diagram of an electronic device according to an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0103] like Figure 6 As shown, the electronic device may include: a processor 602, a communications interface 604, a memory 606, and a communications bus 608.

[0104] in:

[0105] The processor 602, communication interface 604, and memory 606 communicate with each other via communication bus 608.

[0106] Communication interface 604 is used for communication with other electronic devices or servers.

[0107] The processor 602 is used to execute program 610, which can specifically execute the relevant steps in the above-described image planning point synchronization method or implant position planning method embodiments.

[0108] Specifically, program 610 may include program code that includes computer operation instructions.

[0109] Processor 602 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; one or more GPUs; or they may be processors of different types, such as one or more CPUs, one or more GPUs, and one or more ASICs.

[0110] Memory 606 is used to store program 610. Memory 606 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0111] Specifically, program 610 can be used to cause processor 602 to execute the image-to-image planning point synchronization method or implant location planning method in any of the foregoing embodiments.

[0112] The specific implementation of each step in program 610 can be found in the corresponding steps and units described in any of the foregoing embodiments of the inter-image planning point synchronization method or implant position planning method, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0113] In this embodiment, the projection relationship between two images is determined by the coordinates of the first and second planning points in the two images. Then, based on the projection relationship and the updated coordinates of the second planning point in the first image, the updated coordinates of the second planning point in the second image can be determined, thus achieving synchronization of planning points between images. That is, changing the position of a planning point in one image will also result in a synchronous adjustment in the other image, avoiding errors in planning results caused by manual adjustment of planning points in two images. Furthermore, since the planning points in the two images are synchronized according to the projection relationship, the length between planning points in the spatial coordinate system can be fixed during planning, making this method applicable to schemes that require fixed lengths for planning. Therefore, this method for synchronizing planning points between images has high applicability.

[0114] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the methods in the above-described multiple method embodiments.

[0115] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0116] The methods described above according to the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA) for such software processing. It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the image-to-image planning point synchronization method described herein. Furthermore, when a general-purpose computer accesses code used to implement the image-to-image planning point synchronization method shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the image-to-image planning point synchronization method shown herein.

[0117] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0118] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for synchronizing planning points between images, characterized in that, include: Obtain the first coordinates of the first planning point in the first image, and obtain the second coordinates of the first planning point in the second image, wherein the first image and the second image are two two-dimensional images of the same object taken from different directions, and the object is a bone nail located in the human body; Obtain the third coordinate of the second planning point in the first image, and obtain the fourth coordinate of the second planning point in the second image; The positions of the first planning point in the first image and the second image correspond to the positions of the first planning point in the object. The first planning point is in different positions in the first image and the second image, but both are in the same part of the object image included in the first image and the second image. The first planning point needs to be determined manually. The location of the first planning point is determined manually as needed. The first planning point includes at least the planned position of the bone nail cap in the human body. The positions of the second planning point in the first image and the second image correspond to the positions of the second planning point in the object. The second planning point is in different positions in the first image and the second image, but both are in the same part of the object image included in the first image and the second image. The second planning point needs to be determined manually. The location of the second planning point is determined manually as needed. The second planning point includes at least the planned position of the bone screw tip in the human body. Determine the fifth coordinate of the second planning point in the first image after the position update, wherein the spatial distance between the second planning point and the first planning point is equal before and after the position update, and wherein the position update of the second planning point is to obtain the coordinates of the second planning point in the first image after manual adjustment of the second planning point; Determine the spatial coordinates of the first planning point based on the first coordinates and the second coordinates; At least one spatial coordinate of the updated position of the second planning point is determined based on the spatial coordinates of the first planning point, the fifth coordinate, and the spatial distance, wherein the distance between the spatial coordinates of the first planning point and the updated spatial coordinates of the second planning point is equal to the spatial distance. Based on at least one updated spatial coordinate of the second planning point, determine at least one alternative coordinate in the second image corresponding to the updated position of the second planning point. Based on the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates, the sixth coordinate of the second planned point in the second image after the position update is determined from the at least one candidate coordinate.

2. The method according to claim 1, characterized in that, Determining at least one spatial coordinate of the updated position of the second planning point based on the spatial coordinates of the first planning point, the fifth coordinate, and the spatial distance includes: Based on the spatial coordinates of the first planning point and the spatial distance, the following spherical equation is constructed: (x-a) 2 +(y-b) 2 +(z-c) 2 =R 2 (a, b, c) are used to represent the spatial coordinates of the first planning point, R is used to represent the spatial distance, and (x, y, z) are used to represent the spatial coordinates of a point on the sphere. Determine the equation of the straight line passing through the fifth coordinate and the spatial straight line of the first image's shooting source; Calculate at least one spatial coordinate of the updated position of the second planning point based on the spherical equation and the straight line equation.

3. The method according to claim 1, characterized in that, The step of determining at least one alternative coordinate corresponding to the updated position of the second planning point in the second image based on at least one spatial coordinate updated by the second planning point includes: Based on the updated spatial coordinates of the second planning point, at least one alternative coordinate corresponding to the updated position of the second planning point is calculated using the following projection equation; The projection equations include: Z c f is used to characterize the depth of each spatial coordinate of the updated second planning point in the camera coordinate system of the source of the second image. x f is used to characterize the focal length of the shooting source in the x-axis direction of the second image. y The focal length of the source of the second image along the y-axis is used to characterize the focal length of the source of the second image. (u0, v0) represents the coordinates of the intersection point of the optical axis of the source of the second image and the imaging plane of the source of the second image in the pixel coordinate system. R 3X3 Used to characterize the rotation matrix, T 3X1 Used to characterize the translation matrix, (X) w ,Y w Z w (x, y) is used to represent the updated spatial coordinates of the second planning point, and (x, y) is used to represent the alternative coordinates.

4. The method according to any one of claims 1-3, characterized in that, The step of determining the sixth coordinate in the second image after the update of the second planned point position from the at least one candidate coordinate, based on the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates, includes: Calculate the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates; If the distance between the image points corresponding to the third coordinate and the fifth coordinate is less than a preset first distance threshold, then the candidate coordinate with the smallest distance between the corresponding image point and the fourth coordinate is determined as the sixth coordinate.

5. An implant placement planning method, characterized in that, include: Acquire third and fourth images taken from different angles, wherein the third and fourth images are images of the lesion location of the patient to which the implant needs to be inserted, taken from different angles; The seventh coordinate of the first end of the implant in the third image is determined, and the eighth coordinate of the first end in the fourth image is determined, wherein the first end of the implant is the head of the implant. The ninth coordinate of the second end of the implant in the third image is determined, and the tenth coordinate of the second end in the fourth image is determined, wherein the second end of the implant is the tail of the implant; Determining the eleventh coordinate in the third image after the position of the second end is updated includes determining the corresponding coordinate in the third image after the position of the tail of the implant is moved. The spatial distance between the second end and the first end before and after the position update is equal to the length of the implant. The spatial distance is the distance between the spatial coordinates of the first end and the second end of the implant in space. The spatial coordinates of the first end are determined based on the seventh and eighth coordinates. At least one spatial coordinate of the second end after position update is determined based on the spatial coordinate of the first end, the eleventh coordinate, and the length of the implant, wherein the distance between the spatial coordinate of the first end and the spatial coordinate of the second end after position update is equal to the length of the implant; Based on at least one updated spatial coordinate of the second end, determine at least one alternative coordinate in the fourth image corresponding to the updated position of the second end; Based on the distance between the image points corresponding to the ninth coordinate and the eleventh coordinate, and the distance between the tenth coordinate and the image points corresponding to each of the candidate coordinates, the twelfth coordinate in the fourth image after the second end position is updated is determined from the at least one candidate coordinate.

6. A device for synchronizing planning points between images, characterized in that, include: The first acquisition module is used to acquire the first coordinates of the first planning point in the first image and the second coordinates of the first planning point in the second image, wherein the first image and the second image are two two-dimensional images of the same object taken from different directions, and the object is a bone nail located in the human body. The second acquisition module is used to acquire the third coordinate of the second planning point in the first image and the fourth coordinate of the second planning point in the second image; The positions of the first planning point in the first image and the second image correspond to the positions of the first planning point in the object. The first planning point is in different positions in the first image and the second image, but both are in the same part of the object image included in the first image and the second image. The first planning point needs to be determined manually. The location of the first planning point is determined manually as needed. The first planning point includes at least the planned position of the bone nail cap in the human body. The positions of the second planning point in the first image and the second image correspond to the positions of the second planning point in the object. The second planning point is in different positions in the first image and the second image, but both are in the same part of the object image included in the first image and the second image. The second planning point needs to be determined manually. The location of the second planning point is determined manually as needed. The second planning point includes at least the planned position of the bone screw tip in the human body. The first determining module is used to determine the fifth coordinate of the second planning point in the first image after the position of the second planning point is updated, wherein the spatial distance between the second planning point and the first planning point is equal before and after the position update, and wherein the position update of the second planning point is to obtain the coordinates of the second planning point in the first image after manual adjustment of the second planning point. The second determining module is used to determine the spatial coordinates of the first planning point based on the first coordinates and the second coordinates; determine at least one spatial coordinate of the updated position of the second planning point based on the spatial coordinates of the first planning point, the fifth coordinate, and the spatial distance, wherein the distance between the spatial coordinates of the first planning point and the updated spatial coordinates of the second planning point is equal to the spatial distance; and determine at least one alternative coordinate corresponding to the updated position of the second planning point in the second image based on the updated spatial coordinates of the second planning point. The third determining module is used to determine, based on the distance between the image points corresponding to the third coordinate and the fifth coordinate, and the distance between the fourth coordinate and the image points corresponding to each of the candidate coordinates, from the at least one candidate coordinate. The sixth coordinate in the second image after the position of the second planning point is updated.

7. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the image planning point synchronization method as described in any one of claims 1-4 or the implant location planning method as described in claim 5.

8. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the image planning point synchronization method as described in any one of claims 1-4 or the implant position planning method as described in claim 5.

9. A computer program product, characterized in that, Includes computer instructions that instruct a computing device to perform the image-to-image planning point synchronization method as described in any one of claims 1-4 or the implant location planning method as described in claim 5.

Images