Systems and methods for determining a position of a bone fragment or of a surgical tool

Abstract

A method for determining a position of at least a first bone fragment in a body part or of at least one surgical tool during a surgery involving a displacement of the first bone fragment or the surgical tool relative to a reference bone, wherein a first marking device is fixed to the first bone fragment or to the surgical tool, the first marking device comprises an optical marker, said optical marker being optically detectable, and the method is performed by a processor.

Description

SYSTEMS AND METHODS FOR DETERMINING A POSITION OF A BONE FRAGMENT OR OF A SURGICAL TOOLCROSS REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 789,111, filed April 15, 2025, which is hereby incorporated by reference in its entirety.TECHNICAL FIELD

[0002] The relates to a method for determining a position of a bone fragment or of a surgical tool in a body part during a surgery involving the displacement of the bone fragment or the surgical tool, more particularly during a foot surgery, especially a surgery of the hallux valgus.BACKGROUND

[0003] The hallux valgus is a body part that may be deformed and as a consequence requires a surgery. It is well known that the deformation of the hallux valgus results from a cuneiformmetatarsal hypermobility upon a metatarsus varus. The metatarsus varus is accentuated at the bearing phase. There is then observed an inward deviation, called varus of the first metatarsal, whereas the first phalanx, retained by its joint capsule and the sesamoidal line, deviates outwards, which is called valgus.

[0004] To facilitate the surgery of the hallux valgus, surgical guides such as the one described in U.S. Patent No. 12,133,655 may be used. Such guides allow for a minimally invasive or percutaneous approach and simplify the surgery for the surgeon.

[0005] However, the surgeon, while using the guide, does not have visibility into the foot during the surgery. In other words, the surgeon does not directly see the movement of the bones with respect to each other and cannot be completely sure that the surgery is correctly performed. To overcome this issue, several X-ray pictures may taken during the surgery to monitor the displacement of the bone. However, this is not very practical, as it would require an X-Ray imaging device during the entire surgery and would expose medical staff and the patient to radiation at many stages of the surgery.

[0006] Hence, there is a need of a method for determining the position of at least one bone fragment or at least one surgical tool in a body part during a surgery involving the displacement of said bone fragment or surgical tool.SUMMARY OF THE DISCLOSURE

[0007] The disclosure aims at determining the position of at least a first bone fragment or of at least one surgical tool in a body part during a surgery involving the displacement of said first bone fragment or surgical tool in a simple and reliable way.

[0008] The disclosure thus aims at allowing the percutaneous treatment of a hallux valgus in a simple and reliable way.

[0009] According to a first aspect, there is provided a method for determining a position of at least a first bone fragment in a body part or of at least one surgical tool during a surgery involving a displacement of the first bone fragment or the surgical tool relative to a reference bone, wherein a first marking device is fixed to the first bone fragment or to the surgical tool, the first marking device comprises an optical marker, said optical marker being optically detectable, and the method is performed by a processor, the method comprising the following steps: a) obtaining an x-ray picture of the body part displaying an x-ray marker, said x-ray marker being x-ray detectable, the x-ray picture having been acquired with an x-ray imaging device, the x-ray marker having a known relative position with respect to the optical marker of the first marking device in a first frame when the x-ray picture is acquired, b) determining a position of the x-ray marker in the first frame and a position of the x-ray marker in a second frame associated with the x-ray picture, c) determining a transformation between the first frame and the second frame using said positions of the x-ray marker in the first frame and the second frame, d) determining a displacement of the optical marker of the first marking device in the first frame, and e) determining the position of the first bone fragment or of the surgical tool in the second frame as a function of said displacement of said optical marker and said transformation between the first frame and the second frame.

[0010] In some aspects, a second marking device is fixed to the reference bone, the second marking device comprises an optical marker being optically detectable and wherein the first frame is defined by the optical marker of the second marking device. In some aspects, the x-ray marker comprises at least six radiopaque elements. In some aspects, the radiopaque elements are spherical. In some aspects, the x-ray marker comprises at least three radiopaque elements in the first marking device and at least three radiopaque elements in the second marking device and wherein, in step b), determining the position of the x-ray marker in the first frame includes determining the position of each radiopaque elements using: (i) positions of the radiopaque elements of the first marking device in a frame defined by the optical marker of the first marking device, (ii) positions of the radiopaque elements of the second marking device in the frame defined by the optical marker of the second marking device which is the first frame, and (iii) atransformation between the frame defined by the optical marker of the first marking device and the frame defined by the optical marker of the second marking device which is the first frame. In some aspects, the positions of the radiopaque elements of the first marking device in a frame defined by the optical marker of the first marking device are determined using a first equation system expressing a relation between said positions of the radiopaque elements and the position of the optical marker of the first marking device in the frame defined by said optical marker and / or wherein the positions of the radiopaque elements of the second marking device in the first frame are determined using a second equation system expressing a relation between said positions of the radiopaque elements of the second marking device and the position of the optical marker of the second marking device in the first frame.

[0011] In some aspects, the method includes a step of segmenting at least the first bone fragment in the x-ray picture. In some aspects, the method includes a step of acquiring the X-ray picture with the x-ray imaging device.

[0012] In some aspects, the optical marker of the first marking device is configured to reflect or to emit one or more of visible light, ultraviolet light, and infrared light and the displacement of the optical marker of the first marking device in the first frame is determined using displacement data obtained from an optical sensor connected to the processor. In some aspects, the optical marker comprises at least three planar or spherical optically detectable elements configured to reflect or emit one or more of visible light, ultraviolet light, and infrared light.

[0013] In some aspects, step e) comprises generating an updated x-ray picture including an updated position of the first bone fragment or of the surgical tool. In some aspects, a third marking device is fixed to a second bone fragment, and wherein step d) comprises determining a displacement of the optical marker of the third marking device in the first frame, and a step e) of determining the position of the second bone fragment in the second frame as a function of said displacement of said optical marker and said transformation between the first frame and the second frame.

[0014] In some aspects, the first bone fragment and a second bone fragment are parts of a main bone and the method includes a step of displaying on the x-ray picture a cutting line for cutting the main bone to separate said main bone into the first bone fragment and the second bone fragment;

[0015] In some aspects, in step a), two x-ray pictures of the body part are obtained and each x-ray picture represents a different orientation of the body part.

[0016] In some aspects, the body part is a foot and the surgery corresponds to treatment of a hallux valgus.

[0017] According to a second aspect, there is provided a method for treating a hallux valgus percutaneously including a step x) of fixing a first marking device on a first bone fragment being a part of the first metatarsal and a second marking device on a reference bone being the first cuneiform, the second cuneiform or the base of the second metatarsal, a step y) of cutting the main bone to separate said main bone into the first bone fragment and a second bone fragment, a step z) of displacing the first bone fragment relative to the reference bone, said method comprising monitoring the position of the first bone fragment by performing the method for determining the position of the first bone fragment described previously.

[0018] According to a third aspect, there is provided a computing device comprising a processor configured to perform the method for determining the position of a first bone fragment or of a surgical tool described previously.

[0019] According to a fourth aspect, there is provided a non-transitory computer readable medium comprising a computer program product stored thereon comprising program code instructions for executing the method for determining the position of a first bone fragment or of a surgical tool described previously.BRIEF DESCRIPTION OF THE FIGURES

[0020] The detailed description is set forth with reference to the accompanying drawings. The drawings are provided for purposes of illustration only and merely depict example embodiments of the disclosure. The drawings are provided to facilitate understanding of the disclosure and shall not be deemed to limit the breadth, scope, or applicability of the disclosure. The use of the same reference numerals indicates similar, but not necessarily the same or identical components. However, different reference numerals may be used to identify similar components as well. Various embodiments may utilize elements or components other than those illustrated in the drawings, and some elements and / or components may not be present in various embodiments. The use of singular terminology to describe a component or element may depend on the context, encompass a plural number of such components or elements, and vice versa.

[0021] FIG. 1A is a perspective view of a marking device, according to one or more embodiments of the present disclosure.

[0022] FIG. IB is a bottom view of the marking device of FIG. 1A, without an anchoring pin, according to one or more embodiments of the present disclosure.

[0023] FIG. 2 is a bottom view of an x-ray marking having two of the marking devices of FIG. IB, according to one or more embodiments of the present disclosure.

[0024] FIG. 3A is a top view of another marking device, according to one or more embodiments of the present disclosure.

[0025] FIG. 3B is a left side view of the marking device of FIG. 2A, according to one or more embodiments of the present disclosure.

[0026] FIG. 3C is a right side view of the marking device of FIG. 2B, according to one or more embodiments of the present disclosure.

[0027] FIG. 4 depicts a system for determining the position of at least one bone fragment in a body part during a surgery involving displacement of said bone fragment, according to one or more embodiments of the present disclosure.

[0028] FIGS. 5 A is an x-ray picture of a transverse view of a foot equipped with two marking devices, according to one or more embodiments of the present disclosure.

[0029] FIG. 5B is an x-ray picture of a sagittal view of a foot equipped with two marking devices, according to one or more embodiments of the present disclosure.

[0030] FIG. 6 depicts an x-ray imaging device capturing x-ray pictures of a foot equipped with two marking devices, according to one or more embodiments of the present disclosure.

[0031] FIGS. 7A-7B are x-ray pictures of a foot having a displaced bone fragment, equipped with two marking devices, according to one or more embodiments of the present disclosure.

[0032] FIG. 8 depicts a method of affixing one or more marking devices to a foot, according to one or more embodiments of the present disclosure.

[0033] FIGS. 9A-9B illustrate a software interface for viewing the position of one or more marking devices and guiding the repositioning of a bone fragment, according to one or more embodiments of the present disclosure.DETAILED DESCRIPTION

[0034] This disclosure relates to, among other things, a method for determining the position of at least a first bone fragment or of at least one surgical tool in a body part during a surgery involving the displacement of said first bone fragment or surgical tool. In particular, this disclosure pertains to minimally invasive or percutaneous treatments of hallux valgus, using the positioning methods and devices described herein.

[0035] According to a first aspect, there is provided a method for determining a position of at least a first bone fragment in a body part or of at least one surgical tool during a surgeryinvolving a displacement of the first bone fragment or the surgical tool relative to a reference bone, wherein a first marking device is fixed to the first bone fragment or to the surgical tool, the first marking device comprises an optical marker, said optical marker being optically detectable, and the method is performed by a processor, the method comprising the following steps: a) obtaining an x-ray picture of the body part displaying an x-ray marker, said x-ray marker being x-ray detectable, the x-ray picture having been acquired with an x-ray imaging device, the x-ray marker having a known relative position with respect to the optical marker of the first marking device in a first frame when the x-ray picture is acquired, b) determining a position of the x-ray marker in the first frame and a position of the x-ray marker in a second frame associated with the x-ray picture, c) determining a transformation between the first frame and the second frame using said positions of the x-ray marker in the first frame and the second frame, d) determining a displacement of the optical marker of the first marking device in the first frame, and e) determining the position of the first bone fragment or of the surgical tool in the second frame as a function of said displacement of said optical marker and said transformation between the first frame and the second frame.

[0036] In some aspects, a second marking device is fixed to the reference bone, the second marking device comprises an optical marker being optically detectable and wherein the first frame is defined by the optical marker of the second marking device. In some aspects, the x-ray marker comprises at least six radiopaque elements. In some aspects, the radiopaque elements are spherical. In some aspects, the x-ray marker comprises at least three radiopaque elements in the first marking device and at least three radiopaque elements in the second marking device and wherein, in step b), determining the position of the x-ray marker in the first frame includes determining the position of each radiopaque elements using: (i) positions of the radiopaque elements of the first marking device in a frame defined by the optical marker of the first marking device, (ii) positions of the radiopaque elements of the second marking device in the frame defined by the optical marker of the second marking device which is the first frame, and (iii) a transformation between the frame defined by the optical marker of the first marking device and the frame defined by the optical marker of the second marking device which is the first frame. In some aspects, the positions of the radiopaque elements of the first marking device in a frame defined by the optical marker of the first marking device are determined using a first equation system expressing a relation between said positions of the radiopaque elements and the position of the optical marker of the first marking device in the frame defined by said optical marker and / or wherein the positions of the radiopaque elements of the second marking device in the first frame are determined using a second equation system expressing a relation between saidpositions of the radiopaque elements of the second marking device and the position of the optical marker of the second marking device in the first frame.

[0037] In some aspects, the method includes a step of segmenting at least the first bone fragment in the x-ray picture. In some aspects, the method includes a step of acquiring the X-ray picture with the x-ray imaging device.

[0038] In some aspects, the optical marker of the first marking device is configured to reflect or to emit one or more of visible light, ultraviolet light, and infrared light and the displacement of the optical marker of the first marking device in the first frame is determined using displacement data obtained from an optical sensor connected to the processor. In some aspects, the optical marker comprises at least three planar or spherical optically detectable elements configured to reflect or emit one or more of visible light, ultraviolet light, and infrared light.

[0039] In some aspects, step e) comprises generating an updated x-ray picture including an updated position of the first bone fragment or of the surgical tool. In some aspects, a third marking device is fixed to a second bone fragment, and wherein step d) comprises determining a displacement of the optical marker of the third marking device in the first frame, and a step e) of determining the position of the second bone fragment in the second frame as a function of said displacement of said optical marker and said transformation between the first frame and the second frame.

[0040] In some aspects, the first bone fragment and a second bone fragment are parts of a main bone and the method includes a step of displaying on the x-ray picture a cutting line for cutting the main bone to separate said main bone into the first bone fragment and the second bone fragment;

[0041] In some aspects, in step a), two x-ray pictures of the body part are obtained and each x-ray picture represents a different orientation of the body part.

[0042] In some aspects, the body part is a foot and the surgery corresponds to treatment of a hallux valgus.

[0043] According to a second aspect, there is provided a method for treating a hallux valgus percutaneously including a step x) of fixing a first marking device on a first bone fragment being a part of the first metatarsal and a second marking device on a reference bone being the first cuneiform, the second cuneiform or the base of the second metatarsal, a step y) of cutting the main bone to separate said main bone into the first bone fragment and a second bone fragment, a step z) of displacing the first bone fragment relative to the reference bone, saidmethod comprising monitoring the position of the first bone fragment by performing the method for determining the position of the first bone fragment described previously.

[0044] According to a third aspect, there is provided a computing device comprising a processor configured to perform the method for determining the position of a first bone fragment or of a surgical tool described previously.

[0045] According to a fourth aspect, there is provided a non-transitory computer readable medium comprising a computer program product stored thereon comprising program code instructions for executing the method for determining the position of a first bone fragment or of a surgical tool described previously.Marking Device and X-Ray Marker

[0046] Referring now to the drawings, FIGS. 1 A-1B depict a marking device 100 according to one embodiment. The marking device 100 may be configured to assist a surgeon in determining a position of at least one bone fragment or surgical tool in a body part during a surgery involving a displacement of the bone fragment or surgical tool.

[0047] In embodiments, the marking device 100 includes an optical marker 102. The optical marker 102 is optically detectable. In other words, the optical marker 102 can be detected (i.e., its position may be detected) using visible, ultraviolet, and / or infrared light. The optical marker 102, therefore, may be adapted to reflect and / or emit visible, ultraviolet, and / or infrared light so that the position of the optical marker 102 may be determined. Hence, using an optical sensor (not shown) which can detect reflected or emitted visible, ultraviolet, and / or infrared light, the position of the optical marker 102 can be determined. For example, if the optical marker 102 is adapted to reflect visible, ultraviolet, and / or infrared light, the optical sensor may be configured to emit visible, ultraviolet and / or infrared light which will be reflected by the optical marker 102 and sensed by the optical sensor such that a position of the optical marker 102 may be deduced. According to preferred embodiments, the optical marker 102 includes one or more visible, ultraviolet, and / or infrared light reflecting and / or emitting elements 104.

[0048] In embodiments, as shown in FIGS. 1A-1B, the light reflecting and / or emitting elements 104 may be disposed within the optical marker 102 such that the light reflecting and / or emitting elements 104 are present on the top surface 106 of the optical marker 102. However, it would be understood that the light reflecting and / or emitting elements 104 may be present on the bottom surface 108 of the optical marker 102, rather than on the top surface 106, while achieving the same functional purpose. In some embodiments, such as those shown in FIGS.1A-1B, the reflecting and / or emitting elements 104 are planar. The reflecting and / or emitting elements 104 may each be a planar surface configured to reflect visible, ultraviolet, and / orinfrared light. For example, the planar surfaces may be made of infrared reflective plastic tape. In other embodiments (not shown), the reflecting and / or emitting elements 104 are not planar (i.e., the reflecting and / or emitting elements 104 sit within different planes and protrude from and / or are inset within the top surface 106 and / or the bottom surface 108 of the optical marker).

[0049] In some embodiments, as shown in FIGS. 1A-1B, the optical marker 102 has a triangular shape with at least three visible, ultraviolet, and / or infrared light reflecting and / or emitting elements 104 similarly arranged in a triangular pattern. In these embodiments, the light reflecting and / or emitting elements 104 are circular and occupy a majority of the surface area of the top surface 106 and the bottom surface 108 of the optical marker. However, in other embodiments, the optical marker 102 may include any other number of visible, ultraviolet, and / or infrared light reflecting and / or emitting elements 104. Additionally, the visible, ultraviolet, and / or infrared light reflecting and / or emitting elements 104 may be any other shape or combinations of shapes and / or any other size or combinations of sizes, and may be arranged at any other locations on the marking device 100 other than those shown in FIGS. 1 A-1B.

[0050] According to preferred embodiments, the optical marker 102 includes at least three visible, ultraviolet, and / or infrared light reflecting elements 104. In these embodiments, the marking device 100 may be cheaper and easier to manufacture since it does not require energy to emit light. In these embodiments, the optical marker 102, as described herein, may be detectable using infrared light. This may enable detection of the position of the optical marker 102 at ambient light levels, since there is no interference between visible light and infrared light. In this way, the position of the optical marker 102 may be detected in many different environmental conditions.

[0051] In embodiments, the marking device 100 also includes at least one radiopaque element 110 that is detectable or viewable using x-ray imaging. In some embodiments, as shown in FIGS. 1A-1B, each marking device 100 includes four radiopaque elements 110 distributed around the perimeter of the optical marker 102. As shown in FIGS. 1A-1B, the radiopaque elements 110 are present on the bottom surface 108 of the optical marker 102. However, it would be understood that the radiopaque elements may be present on the bottom surface 108 and / or top surface 106 of the optical marker 102, or within the optical marker 102, without sacrificing the function thereof. Similarly, any number of radiopaque elements 110 may be present in any configuration within the optical marker 102. In some embodiments, the radiopaque elements 110 are spherical, however the radiopaque elements 110 may be any shape suitable for use with the optical marker 102 as would be understood by one skilled in the art.

[0052] In embodiments, as shown in FIG. 2, two or more marking devices 100 may be provided to form an x-ray marker 150. The x-ray marker 150 may include at least six radiopaque elements 110, all of which, or only some of which, may be contained on a single marking device 100. According to preferred embodiments, the x-ray marker 150 preferably includes eight radiopaque elements 110, with four radiopaque elements 110a being provide on a first marking device 100a and four radiopaque elements 110b being provided on a second marking device 100b. While the radiopaque elements 110 are depicted as having a particular shape and size and arrangement on the optical marker 102, the radiopaque elements 110 may be any other size or combinations of sizes and / or shape or combinations of shapes, and may be arranged in any configuration on the optical marker 102.

[0053] The radiopaque elements 110 are opaque to x-rays, such that x-rays cannot pass through the radiopaque elements 110. Thus, on an x-ray image, the radiopaque elements 106 will appear black (or dark relative to the rest of the image).

[0054] In embodiments, the reflecting and / or emitting elements 104 of the optical marker 102 of each marking device 100 are located with respect to the radiopaque elements 110 of the marking device 100 such that, knowing the positions of the radiopaque elements 110 in said marking device 100, the positions of the reflecting and / or emitting elements 104 of the optical marker 102 can be deduced using a software system configured for use with the marking device 100. In other words, if the position of the part of the x-ray marker 150 in the marking device 100 is known, the position of the optical marker 102 of said marking device 100 may be ascertained. Knowing the positions of the radiopaque elements 110 in a marking device 100, then it is possible, using the corresponding positioning software, to determine the positions of the reflecting and / or emitting elements 104, since the relative position between the radiopaque elements 110 and the reflecting and / or emitting elements 104 is known. Thus, the radiopaque elements 110 in the marking device 100 are located such that there is only one possible position of the reflecting and / or emitting elements 104 relative to the radiopaque elements 110.

[0055] According to preferred embodiments, the marking device 100 includes three reflecting and / or emitting elements 104 and four radiopaque elements 110. The coordinates of the reflecting and / or emitting elements 104 are respectively expressed as the vectors Vl(xl, yl, zl), V2(x2, y2, z2), and V3(x3, y3, z3). The coordinates of the radiopaque elements 110 are respectively expressed as the vectors Wl(xl, yl, zl), W2(x2, y2, z2), W3(x3, y3, z3), and W4(x4, y4, z4). An equation linking those coordinates maybe expressed as f(Vl, V2, V3)=W1, g(Vl, V2, V3)=W2, h(Vl, V2, V3)=W3, and i(Vl, V2, V3)=W4, with f, g, h, and i being functions.

[0056] In other words, the marking device 100 is provided as a single component, where the marking device 100 includes an optical marker 102 which is an array of light reflecting and / or emitting elements 104, and one or more radiopaque elements 110. In preferred embodiments, the marking device 100 has three reflecting and / or emitting elements 104 that together form the optical marker 102, and the marking device 100 has an array of four radiopaque elements 110, with the array of radiopaque elements 110 forming part of or the entirety of an x-ray marker 150. For example, in embodiments where only one marking device 100 is used, the four radiopaque elements 110 of the marking device 100 together form the x-ray marker 150. In embodiments where multiple marking devices 100 are used, the radiopaque elements 110 of all the marking devices 100 form the x-ray marker 150. In embodiments, the marking device 100 also includes an anchoring pin 112 for fixing the marking device 100 to a bone. In some embodiments, as shown in FIG. 1 A, the anchoring pin 112 extends from a joint 114 that is fixed to the bottom surface 108 of the optical marker 102. In these embodiments, the anchoring pin 112 may be removed (and reattached) to the optical marker 102 via the joint 114. The joint 114 may also include knob 116 for adjusting the position of the anchoring pin 112 (i.e., the angle and / or direction of the anchoring pin 112) relative to the plane of the optical marker 102. In other embodiments, the anchoring pin 112 may be fixed relative to the optical marker 102. In some embodiments, also as shown in FIG. 1A, the anchoring pin 112 may include a thread 118 for screwing the anchoring pin 112 into a bone. However, the marking device 100 may also be secured to a bone using any other suitable mechanism such as a nail or an adhesive tape.

[0057] Referring now to FIGS. 3 A-3C, a marking device according to another embodiment is shown. The marking device 200 may be similar to the marking device 100 shown and described with respect to FIGS. 1A-1B, in that the marking device 200 includes an optical marker 202 having at least one reflecting and / or emitting element 204 and one or more radiopaque element 210 arranged on said optical marker 202. As shown in FIG. 3C, the light reflecting and / or emitting elements 204 may be arranged on the top surface 206 of the optical marker 102, while the radiopaque elements 210 span between the top surface 206 and the bottom surface 208 of the optical marker 202. As with marking device 100, the marking device 200 preferably includes three reflecting and / or emitting elements 204 and four radiopaque elements 206 arranged around the perimeter of the optical marker 202. Like with the marking device 100 of FIGS. 1A-1B, the position of the optical marker 202 and light reflecting and / or emitting elements 204 may be determined based on the position of the radiopaque elements 210, when the position of the radiopaque elements 210 is detected using x-ray imaging.

[0058] According to preferred embodiments, the marking device 200 has suitable dimensions for use in a minimally invasive or percutaneous surgical method. For use in a minimally invasive surgical procedure, it is preferable that the marking device 200 has a small profile (i.e., an overall surface area of about 12 cm2or less) so as to not obstruct the surgeon while the procedure is being performed. In some embodiments, the optical marker 202 has a maximum distance, in the largest direction, of between about 3 cm to about 4 cm, preferably about 3.5 cm, and has a maximum distance, in the smallest direction, of about 2.5 cm to about 3.5 cm, preferably about 3 cm.

[0059] In some embodiments, the light reflecting and / or emitting elements 204 may be arranged on the optical marker 202 in a triangular or substantially triangular configuration, as shown in FIG. 3A. A maximum distance between mid-points of the reflecting and / or emitting elements 204 may be between about 1.5 cm and about 2 cm, preferably about 1.75 cm, and a minimum distance between mid-points of the reflecting and / or emitting elements 204 may be between about 1 cm and about 2 cm, preferably about 1.5 cm. According to preferred embodiments, the radiopaque elements 210 are arranged in a trapezoidal arrangement surrounding the array of reflecting and / or emitting elements 204, as shown in FIG. 3C. In these embodiments, the optical marker 202 may also be trapezoidal, or substantially trapezoidal, with each radiopaque element 210 being arranged near a comer of the optical marker 202.

[0060] In embodiments, as shown in FIGS. 3B-3C, the marking device 200 also includes an anchoring pin 212 for fixing the marking device 200 to a bone. As shown in FIGS. 3B-3C, the anchoring pin 212 may have a bevelled tip 214 for inserting the marking device 200 into a bone. The bevelled tip 214 may be effective to secure the marking device 200 within the bone while also preventing movement, particularly rotational movement, of the device 200 relative to the bone once it has been inserted. In some embodiments, as shown in FIGS. 3B-3C, the anchoring pin 212 is fixed to the optical marker 202. In other embodiments, the anchoring pin 212 may be removably or adjustably attached to the optical marker 202, similar to the anchoring pin 112 described with respect to FIG. 1 A.

[0061] While the x-ray marker 150 is shown and described herein with respect to the marking device 100 of FIGS. 1A-1B, it would be understood that the x-ray marker 150 may also be formed of two or more marking devices 200 of FIGS. 3A-3C, or a combination of marking devices 100 and marking devices 200. Furthermore, any number of marking devices may be used for performing the methods as described herein, and the marking devices may include any combination of features as described with respect to FIGS. 1 A-3C.Positioning System

[0062] Referring now to FIG. 4, a system 10 is provided to perform a method for determining the position of at least one first bone fragment in a body part during a surgery involving the displacement of said first bone fragment with respect to a reference bone. While the system 10 is described herein with reference to the marking device 100 and the components and features thereof, it would be understood that the system 10 and other methods described herein may also be implemented with either marking device 100 or marking device 200 as previously described. The system 10 may include one or more marking devices 100, a computing device 20, an x-ray imaging device 30, a display 40, an optical sensor 50, and / or a surgical tool 60 (which may optionally be equipped with a marking device 100).

[0063] In embodiments, the computing device includes a processor 22 (or a number of processors) configured to perform the following steps: (i) obtaining an x-ray image of the body part (BP) displaying an x-ray marker 150, said x-ray marker 150 being x-ray detectable, the x-ray picture having been acquired with an x-ray imaging device 30, the x-ray marker 150 having a known relative position with respect to the optical marker 102a of the first marking device 100a in a first frame when the x-ray picture is acquired; (ii) determining a position of the x-ray marker 150 in the first frame and a position of the x-ray marker 150 in a second frame associated with the x-ray picture; (iii) determining a transformation between the first frame and the second frame using said positions of the x-ray marker 150 in the first frame and the second frame; (iv) determining a displacement of the optical marker 102a of the marking device 100a in the first frame; and (v) determining the position of the first bone fragment or of the surgical tool 60 in the second frame as a function of said displacement of said optical marker 102 and said transformation between the first frame and the second frame.

[0064] The processor 22 may be the processor 22 of a computing device 20 which is used by the medical staff in the operating room. Said computing device 20, and thus the system 10, preferably includes memory 24 configured to store x-ray images and / or any data used to perform the method for determining the position of at least one bone fragment. The computing device 20 may be any type of device with processing capabilities, such as a desktop or laptop computer, tablet, smartphone, etc.

[0065] While FIG. 4 shows a single computing device 20, this is not intended to be limiting and any other number of computing devices may also be included in the system 10. Additionally, the system 10 may include one or more remote computing devices (such as remote server(s)). That is, some or all of the processing associated with the method described hereinmay be performed locally at the computing device 20 and / or transmitted to a remote computing device for processing.

[0066] The system preferably includes an x-ray marker 150. As explained previously, the x-ray marker 150 is x-ray detectable. In other words, the x-ray marker 150 is viewable on an x-ray picture. The x-ray marker 150 may be used to a picture of a body part equipped with at least a first marking device 100a as described in greater detail below.

[0067] In some embodiments, the x-ray marker 150 includes only one marking device 100. In other preferred embodiments, two or more marking devices 100 form the x-ray marker 150, such that different parts of the x-ray marker 150 are provided on different marking devices 100. For example, the x-ray marker 150 may be divided in two parts: a first part of the x-ray marker 150 being a first marking device 100a and a second part of the x-ray marker 150 being a second marking device 100b.

[0068] The x-ray marker 150 may include one or more radiopaque elements 110, preferably at least six radiopaque elements 110, and even more preferably eight radiopaque elements 110. As shown in FIG. 2, four radiopaque elements 110a are included in a first marking device 100a and four radiopaque elements 110b are included in a second marking device 100b.

[0069] In embodiments, the system 10 includes a display 40 configured to be connected to the processor 22, and thus to the computing device 20, and configured to display x-ray images acquired by the x-ray imaging device 30. The connection between the processor 22 and the display 40 may be wired or wireless. The display 40 may also be configured to display any other types of relevant information described herein or otherwise.

[0070] In embodiments, the system 10 includes the x-ray imaging device 30. The x-ray imaging device 30 is configured to take x-ray images. In some embodiments, the x-ray imaging device 30 is a C-arm x-ray imaging device, which is capable of taking high quality x-ray images. However, any other type of x-ray imaging devices 30 may also be used. Preferably, the x-ray imaging device 30 is configured to be connected to the processor 22 such that the processor 22 can receive x-ray images taken by the x-ray imaging device 30. The connection between the processor 22 and the X-ray imaging device 30 may be wired or wireless.

[0071] In embodiments, the system 10 includes an optical sensor 50 configured to detect visible, ultraviolet, and / or infrared light. In an example, especially when at least one marking device 100 includes an optical marker 102 which is configured to reflect visible, ultraviolet, and / or infrared light, the optical sensor 50 is configured to emit visible, ultraviolet, and / or infrared light and detect reflected visible, ultraviolet, and / or infrared light. Preferably, the optical sensor 50 is an optical infrared camera. However, any other number of optical sensors50 may also be used and the optical sensors 50 may also be any other type of optical sensor 50 configured to detect visible, ultraviolet, and / or infrared light.

[0072] Preferably, the optical sensor 50 is configured to be connected to the processor 22 such that the optical sensor 50 can send to the processor 22 data regarding reflected visible, ultraviolet and / or infrared light. More precisely, the optical sensor 50 is configured to send data regarding the positions of the optical marker 102, more specifically the positions of the reflecting and / or emitting elements 104 of the optical marker 102 to the processor 22. The connection between the processor 22 and the optical sensor 50 may be wired or wireless.

[0073] For example, the optical sensor 50 may be configured to track the position, and the displacement, of the optical marker 102 and to send displacement data related to the optical marker 102 to the processor 22. In that case, the optical sensor 50 includes a sensing module (not shown) configured to determine positions of optical marker(s) 102 and includes a processing module (not shown) to process the position data in order to compute a displacement and obtain displacement data. The processing module may be in a same casing than the sensing module or may be included in a different device and be connected to the sensing module with or without a wire. Alternatively, the processing module is included in the processor 22.

[0074] In some embodiments, the optical sensor 50 is a stereoscopic camera emitting infrared light and the optical marker 102 is adapted to reflect infrared light. To determine the position of the optical marker 102, the optical sensor 50 emits infrared light. The light is reflected by the optical marker 102 and the optical sensor 50 detects the reflected light. As a function of the reflected light, the optical sensor 50 can determine the position of the optical marker 102. Alternately, or in addition, the optical marker 102 is adapted to emit light, for example infrared, and the optical sensor 50 is adapted to detect light. When the optical marker 102 emits light, the optical sensor 50 detects the light and can determine the position of the optical marker 102.

[0075] In some embodiments, the optical sensor 50 may also be configured to determine the 3D positions of optical markers 102. In other words, the optical sensor 50 is configured to determine 3D coordinates of optical markers 102. In these embodiments, the optical sensor 50 may be, for example, a stereoscopic camera.

[0076] In embodiments, the processor 22 (and / or any other elements of the system 10) may be configured to communicate via a communications network (not shown). The communications network may include, but is not limited to, any one of a combination of different types of suitable communications networks such as, for example, broadcasting networks, cable networks, public networks (e.g., the Internet), private networks, wirelessnetworks, cellular networks, or any other suitable private and / or public networks. Further, the communications network may have any suitable communication range associated therewith and may include, for example, global networks (e.g., the Internet), metropolitan area networks (MANs), wide area networks (WANs), local area networks (LANs), or personal area networks (PANs). In addition, communications network may include any type of medium over which network traffic may be carried including, but not limited to, coaxial cable, twisted-pair wire, optical fiber, a hybrid fiber coaxial (HFC) medium, microwave terrestrial transceivers, radio frequency communication mediums, white space communication mediums, ultra-high frequency communication mediums, satellite communication mediums, or any combination thereof.

[0077] The processor(s) 22 may include any suitable processing unit capable of accepting digital data as input, processing the input data based on stored computer-executable instructions, and generating output data. The computer-executable instructions may be stored, for example, in the data storage and may include, among other things, operating system software and application software. The computer-executable instructions may be retrieved from the data storage and loaded into the memory 24 as needed for execution. The processor may be configured to execute the computer-executable instructions to cause various operations to be performed. Each processor may include any type of processing unit including, but not limited to, a central processing unit, a microprocessor, a microcontroller, a Reduced Instruction Set Computer (RISC) microprocessor, a Complex Instruction Set Computer (CISC) microprocessor, an Application Specific Integrated Circuit (ASIC), a System-on-a-Chip (SoC), a field-programmable gate array (FPGA), and so forth.

[0078] The computing device 20 (or any other computing device) may also include data storage (not shown). The data storage may store program instructions that are loadable and executable by the processor 22, as well as data manipulated and generated by the processor(s) 22 during execution of the program instructions. The program instructions may be loaded into the memory 24 as needed for execution. The memory 24 may be volatile memory (memory that is not configured to retain stored information when not supplied with power) such as random access memory (RAM) and / or non-volatile memory (memory that is configured to retain stored information even when not supplied with power) such as read-only memory (ROM), flash memory, and so forth. In various implementations, the memory 24 may include multiple different types of memory, such as various forms of static random access memory (SRAM), various forms of dynamic random access memory (DRAM), unalterable ROM, and / or writeablevariants of ROM such as electrically erasable programmable read-only memory (EEPROM), flash memory, and so forth.

[0079] Various program modules and / or applications may be stored in the data storage, the program modules and / or applications having computer-executable instructions that, when executed by the processor 22, cause various operations to be performed. The memory 24 may have loaded from the data storage one or more operating systems (O / S) that may provide an interface between other application software (for example dedicated applications, a browser application, a web-based application, a distributed client-server application, etc.) executing on the server and the hardware resources of the server. More specifically, the O / S may include a set of computer-executable instructions for managing the hardware resources of the server and for providing common services to other application programs (for example managing memory allocation among various application programs). The O / S may include any operating system now known or which may be developed in the future including, but not limited to, any mobile operating system, desktop or laptop operating system, mainframe operating system, or any other proprietary or open-source operating system.

[0080] Using this system 10, as described in greater detail below, the surgeon may precisely monitor the position of one or more bone fragments and / or instruments during a minimally invasive or a percutaneous surgery without needing to take a series of x-ray pictures at several times throughout the surgery. Indeed, thanks to the optical marker(s) 102, the position of the first bone fragment BF1 (as shown in FIGS. 5A-5B and FIGS. 7A-7B) or of a surgical tool 60 (as shown in FIG. 4) may be determined without having to acquire X-ray pictures during the surgery. Only one x-ray image, for each desired orientation of the body part (BP), which is acquired at the beginning of the surgery, is needed.Bone Fragment Positioning Method

[0081] The system 10 as described herein may be used to perform a method for determining the position of at least a first bone fragment BF1 in a body part BP or of a surgical tool 60 during a surgery involving the displacement of said first bone fragment BF1 with respect to a reference bone BR. The bones are for example viewable in the X-ray pictures illustrated in FIGS. 5A-5B and FIGS. 7A-7B.

[0082] The reference bone BR is a bone which is not displaced during the surgery (i.e., the referenced bone BR remains unaffected during the surgery. While the reference bone BR may be displaced because the body part as a whole is displaced, the position of reference bone BR relative to the body part BP remains unchanged. The reference bone BR is a bone which servesas a reference to determine the displacement of bone fragment(s) and / or surgical tool(s) 60 during the procedure.

[0083] In embodiments, the first bone fragment BF1 is a part of the main bone BM. The main bone BM preferably also has a second bone fragment BF2. In some embodiments, the surgery involves cutting the main bone BM to separate said main bone BM into the first bone fragment BF1 and the second bone fragment BF2.

[0084] According to preferred embodiments, the body part BP is a foot and the surgery is a surgery for treating a hallux valgus, preferably using a surgical guide such as that described in U.S. Patent No. 12,133,655.

[0085] In these preferred embodiments, the first bone fragment BF1 is a part of the first metatarsal, preferably a fragment at the distal end of the first metatarsal. The second bone fragment BF2 may be the base of the first metatarsal. In these embodiments, the reference bone is the first cuneiform, the second cuneiform, the base of the second metatarsal, or any other immobile bone that can thus serve as a reference point. In the illustrated examples, the reference bone is the second cuneiform.

[0086] In some embodiments, a first marking device 100a may be attached to the first bone fragment BF1. For example, the first marking device 100a may be fixed to the first bone fragment BF1 via the anchoring pin 112 (or via any other suitable mechanism depending on the configuration of the first marking device 100a). The first marking device 100a includes an optical marker 102a, preferably including reflecting and / or emitting elements 104a as described above. According to preferred embodiments, the first marking device 100a also includes radiopaque elements 110a as described above, and the first marking device 100a forms a part of the x-ray marker 150. Preferably, the first marking device 100a includes three, more preferably four, radiopaque elements 110a. The letter “a” after the numerical references (102, 104, 110) only indicates that it refers to the first marking device 100a. While reference is made herein to certain numbers of markers on a marking device, this is not intended to be limiting, and a marking device may include any other number of such markers.

[0087] In other embodiments, the first marking device 100a may be attached to a surgical tool 60. This allows to determine the position of the surgical tool 60 during the surgery.

[0088] According to preferred embodiments, as shown in FIG. 4, a first marking device 100a is fixed to the first bone fragment BF1 and another marking device lOOd is fixed to the surgical tool 60. In some instances, multiple marking devices lOOd may be affixed to the surgical tool 60 or a plurality of surgical tools 60. The marking device lOOd includes an optical marker 102d having one or more light reflecting and / or emitting elements 104d as previouslydescribed. The marking device lOOd, in preferred embodiments, does not form part of the x-ray marker 150 and therefore does not include any radiopaque elements. The letter “d” after the numerical references (100, 102) only indicates that it refers to the particular marking device lOOd.

[0089] In embodiments, a second marking device 100b may be attached to the reference bone BR. The second marking device 100b may be attached to the reference bone BR via the anchoring pin 112 (or via any other suitable mechanism depending on the configuration of the first marking device 100a).

[0090] The second marking device 100b allows to monitor precisely the position of the first bone fragment BF 1 , or any other bone fragment equipped with a marking device, or of a surgical tool 60 even if the patient, and thus the body part BP, moves. Indeed, the second marking device 100b makes it possible to determine the displacement of at least the first bone fragment BF1 or surgical tool 60 with regards to the reference bone BR. Since the reference bone BR is not supposed to be displaced inside the body part during the surgery, the displacement of the first bone fragment BF1 or surgical tool 60 can be correctly determined even if the body part moves or is moved. The second marking device 100b includes an optical marker 102b, preferably including reflecting and / or emitting elements 104b as described above. According to preferred embodiments, the second marking device 100b also includes radiopaque elements 110b as previously described, forming part of the x-ray marker 150. Preferably, the second marking device 100b includes three, more preferably four, radiopaque elements 110b. The letter “b” after the numerical references (102, 104, 106) only indicates that it refers to the second marking device 100b.

[0091] In preferred embodiments, the first marking device 100a and the second marking device 100b may each include three, more preferably four, radiopaque elements 110, such that the x-ray marker 150 has six, more preferably eight, radiopaque elements 110, with the x-ray marker 150 including only the first marking device 100a and the second marking device 100b.

[0092] In other embodiments, there is the first marking device 100a attached to the first bone fragment BF1 including three, more preferably four, radiopaque elements 110b, the second marking device 100b attached to the second bone fragment BF2 including three, more preferably four, radiopaque elements 100b, and the fourth marking device lOOd fixed to a surgical tool 60 having no radiopaque elements 110 such that the fourth marking device lOOd does not form a part of the x-ray marker 150. In some instances, the fourth marking device lOOd includes radiopaque elements 1 lOd, but still does not form a part of the x-ray marker 150.

[0093] In some embodiments, a third marking device 100c may be attached to the reference bone BR. The third marking device 100c includes an optical marker 102c with reflecting and / or emitting elements 104c as described above. The third marking device 100c may or may not include radiopaque elements 106c as described above. The letter “c” after the numerical references (102, 104, 106) only indicates that it refers to the third marking device 100c. In embodiments, if the body part is a foot and the surgery is a hallux valgus surgery, the third marking device 100c is fixed on the of second cuneiform. As shown in FIG. 4, each marking device 100a, 100b, 100c is preferably fixed to the body part BP such that the optical marker 102 and the radiopaque elements 110 are out of the body part BP so that the position of the optical marker 102 can be determined using the optical sensor 50.

[0094] After the marking devices 100 have been attached to the patient as needed for the desired procedure, the first step of the positioning method includes acquiring, by the x-ray imaging device 30, at least one x-ray picture of the body part BP. For example, as shown in FIG. 6, during the surgical procedure the body part BP may be placed proximate to the x-ray imaging device 30 such that the X-ray picture of the body part BP may be obtained. The body part BP is already equipped with the first marking device 100a and / or a surgical tool 60 is equipped with a fourth marking device lOOd. The body part BP may already be equipped with the second marking device 100b and, in some embodiments, may also already be equipped with the third marking device 100c as illustrated in FIG. 4. Preferably, the x-ray imaging device 30 is a C-arm as illustrated in FIG. 6. However, any other type of x-ray imaging device 30 may also be used.

[0095] In this step, several x-ray pictures (for example, using the x-ray imaging device 30) may be taken, with each x-ray picture representing the body part BP in a different orientation, the body part BP being equipped at least with the first marking device 100a. Preferably, two x-ray pictures are acquired, one x-ray picture representing a sagittal view of the body part and the other x-ray picture representing a transverse view of the body part (however, any other number of x-ray pictures may also be obtained), as shown in FIGS. 5A-5B and FIGS. 7A-7B.

[0096] When the x-ray picture is acquired, the x-ray marker 150 must be located such that it will be viewable on the x-ray picture and such that the relative position between the x-ray marker 150 with respect to the optical marker 102a of the first marking device 100a and / or fourth marking device lOOd in a first frame is known by the processor 22.

[0097] For example, the first frame may be the frame defined by the optical sensor 50 and the first marking device 100a may be the entire x-ray marker 150 (as opposed to only being a portion thereof). The position of the x-ray marker 150 with regards to the optical marker 102aof the first marking device 100a may therefore be determined based on the known structure of the first marking device 100a.

[0098] In some embodiments, the first frame is a frame defined by the optical marker 102b of the second marking device 100b, specifically by the reflecting and / or emitting elements 104b of the optical marker 102b. When the second marking device 100b is used, the position of the first bone fragment BF1 and / or the second bone fragment BF2 and / or surgical tool 60 may be determined precisely and correctly even if the body part moves. In these embodiments, the first marking device 100a and the second marking device 100b together form the x-ray marker 150.

[0099] In these embodiments, the relative position (Pl) of the radiopaque elements 110b of the second marking device 100b are known relative to location of the optical marker 102b since the structure of the second marking device 100b is known and the position of the optical marker 102b can be optically determined. Moreover, the relative position (P2) of the optical marker 102a of the first marking device 100a with respect to the optical marker 102b of the second marking device 100b can also be determined using the optical sensor 50. The known relative positions of the first marking device 100a and the second marking device 100b may allow the surgeon (or another individual performing the method described herein) to switch between a frame defined by the optical marker 102a of the first marking device 100a and a frame defined by the optical marker 102b of the second marking device 100b (the first frame in this case). Moreover, in the frame of the optical marker 102a of the first marking device 100a, the relative position (P3) of the radiopaque elements 110a of the first marking device 100a with respect to the optical marker 102a of the first marking device 100a is known since the structure of the first marking device 100a is known and the position of the optical marker 102a can be optically determined.

[0100] Using P2 and P3, the relative positions of the part radiopaque elements 110 of the first marking device 100a and the second marking device 100b (together forming the x-ray marker 150) can be determined. Using Pl and P2, the relative position of the radiopaque elements 110b of the second marking device 100b (forming part of the x-ray marker 150) and the optical marker 102a of the first marking device 100a in the first frame is known. The x-ray marker 150 therefore has a known position relative to the optical marker 102a of the first marking device 100a in a first frame when an x-ray image has been obtained.

[0101] The method next includes a step of obtaining the x-ray picture of the body part BP with the x-ray marker 150 displayed, the x-ray picture having been acquired by the x-ray imaging device 30. In embodiments, this step includes obtaining two x-ray pictures, eachpicture representing the body part BP in a different orientation. The steps of the method as described herein may be repeated, if necessary, to obtain the desired positioning information.

[0102] As shown in FIGS. 5A-5B and FIGS. 7A-7B, the radiopaque elements 110 of the x-ray marker 150 are viewable in the x-ray picture in the form of points, shown as black dots in FIGS. 5A-5B and FIGS. 7A-7B. In embodiments, the x-ray picture is displayed by the screen 40. Any of the captured images may also be stored in the memory 24 of the computing device 20 or transmitted to a remove computing device for storage.

[0103] The method also includes a step of segmenting at least the first bone fragment BF1 in the x-ray picture. In preferred embodiments, this includes segmenting the main bone BM, which includes the first bone fragment BF1, in the x-ray picture. The precise means for doing so would be understood by one skilled in the art.

[0104] In embodiments, the main bone BM, which includes the first bone fragment BF1 and, in some instances, the second bone fragment BF2, is segmented in the x-ray picture to determine a contour of the main bone BM. This contour will, after the cutting of the main bone BM, be cut in two contours: one of the first bone fragment BF1 and one of the second bone fragment BF2.

[0105] In some embodiments, the segmenting is fully or partially automatically performed. For example, the processor 20 may be configured to perform the segmenting of the main bone BM in the x-ray picture (e.g., using and artificial intelligence) and an operator may correct the automated segmenting, if necessary, by selecting manually points of the desired contour).

[0106] In embodiments, the screen 40 displays a contour of the first bone fragment BF1 or of the main bone BM to illustrate the segmenting as illustrated in FIGS. 5A-5B and 7A-7B, and also in FIGS. 9A-9B.

[0107] The positioning method also includes a step of determining a position of the x-ray marker 150 in the first frame and in a second frame associated with the x-ray picture. As used herein, “frame” means frame of reference, which may be a system of geometric axes in relation to which measurements of position can be made.

[0108] For example, the first frame is a frame defined by the optical sensor 50, i.e., the frame according to which the optical sensor 50 is configured to (its intrinsic parameters) determine positions. In other words, the first frame is the frame relative to which the position of an optical marker 102 is determined by the optical sensor 50.

[0109] According to preferred embodiments, the first frame is a frame defined by the optical marker 102b of the second marking device 100b, specifically by the reflecting and / or emitting elements 104b of the optical marker 102b. The first frame can, for example, be defined bydetecting the positions of the reflecting and / or emitting elements 104 using the optical sensor 50. Knowing those positions, the processor 22 can define the first frame and thus process data from the optical sensor 50 (regarding positions of optical marker 102) so that they are defined in this first frame. Determining the positions regarding the first frame which is the frame defined by the optical marker 102b of the second marking device 100b facilitates precise determination of the position of a bone fragment and / or surgical tool even if the body part moves. The first frame may differ depending on the steps of the method.

[0110] In embodiments, the first frame in this step is the frame of the optical marker 102b of the second marking device 100b, subsequently named the frame of the second marking device 100b. This identification of the frame may help to compensate for any unexpected movement of the body part. Moreover, with the frame defined in this way, the body part BP may be displaced during surgery, relative to the optical sensor 50, but the position of a bone fragment and / or surgical tool may still be determined.[oni] The second frame may be defined by the x-ray imaging device 30. In embodiments, the second frame is the frame of the x-ray picture and, from the second frame, the position of elements of the system 10 such as the x-ray marker 150, may be defined within the x-ray picture. The second frame associated to one x-ray picture may differ from the second frame associated to another different x-ray picture.

[0112] In embodiments, the position of the x-ray marker 150 in the second frame is determined automatically, using image processing. In some embodiments, the x-ray marker 150 includes radiopaque elements 110 which are viewable on the x-ray picture, and the processor 22 may be configured to detect the position of said radiopaque elements 110 on the x-ray picture.

[0113] In other embodiments, the position of the x-ray marker 150 in the second frame is determined manually. For example, an operator may select on the display 40 the positions of the radiopaque elements 110 on the x-ray picture. The selected positions are obtained by the processor 22.

[0114] In some embodiments, the position of the x-ray marker 150 in the second frame is determined partially automatically. For example, an operator may select on the display 40 the positions of the radiopaque elements 110 on the x-ray picture and the processor 22 corrects said positions using image processing.

[0115] In other embodiments, the position of the x-ray marker 150 in the second frame is determined automatically by the x-ray imaging device 30 which determines, while acquiring the x-ray picture, the position of the x-ray marker 150 in the second frame and sends saidposition to the processor 22. The processor 2 obtains, and thus determines, said position of the x-ray marker 150.

[0116] The position of the x-ray marker 150 in the first frame may be determined using the known relative position of the x-ray marker 150 with respect to the optical marker 102 in the first frame when the x-ray picture is acquired, as described previously. In embodiments, the x-ray marker 150 comprises at least three radiopaque elements 110a in the first marking device 100a and at least three radiopaque elements 110b in the second marking device 100b and, in this step, determining the position of the x-ray marker 1 in the first frame includes determining the position of each radiopaque elements 110 using (i) positions of the radiopaque elements 110a of the first marking device 100a in a frame defined by the optical marker 102a of the first marking device 100a, (ii) positions of the radiopaque elements 110b of the second marking device 100b in the frame defined by the optical marker 102b of the second marking device 100b which is the first frame, (iii) a transformation between the frame defined by the optical marker 102a of the first marking device 100a and the frame defined by the optical marker 102b of the second marking device 100b which is the first frame. Said transformation may be determined using data from the optical sensor 50, from which the positions of the optical markers 102a, 102b, and thus a transformation between the frames they define, can be determined.

[0117] In embodiments, the positions of the radiopaque elements 110a, 110b are determined using a known relationship between, for each marking device 100, its radiopaque elements 110 and its reflecting and / or emitting elements 104.

[0118] According to preferred embodiments, equation systems may be used to determine the positions of the radiopaque elements 110a, 110b. An equation system may include at least one equation. The positions of the radiopaque elements 110a of the first marking device 100a in a frame defined by the optical marker 102a of the first marking device 100a may be determined using a first equation system expressing a relation between said positions of the radiopaque elements 110a and the positions of the optical marker 102a of the first marking device 100a in the frame defined by said optical marker 102a. The positions of the radiopaque elements 110b of the second marking device 100b in the first frame are determined using a second equation system expressing a relation between said positions of the radiopaque elements 110b of the second marking device 100b and the positions of the optical marker 102b of the second marking device 100b in the first frame.

[0119] For the part of the x-ray marker 150 containing the second marking device 100b, the position of said part (i.e., the radiopaque elements 110b) in the first frame is determined using the position of the optical marker 102b in the first frame and the known relation between thispart of the x-ray marker 150 and the optical marker 102b. The position of the optical marker 102b in the first frame (i.e., the frame defined by the optical sensor 50) is computed by the processor 22 using the position of the optical marker 102b in the frame of the optical sensor 50 which is detected by the optical sensor 50.

[0120] For the part of the x-ray marker 150 containing the first marking device 100a, the position of said part (i.e., the radiopaque elements 106a) is first determined using the position of the optical marker 102a in the frame defined by said optical marker 102a and the known relationship between this part of the x-ray marker 150 and the optical marker 102a, which determines the position of the part of the x-ray marker 150 containing the first marking device 100a in the frame defined by the optical marker 102a of the first marking device 100a. The position of the optical marker 102a in the frame defined by said optical marker 102a is computed by the processor 22 using the position of the optical marker 102a in the frame of the optical sensor 50 which is detected by the optical sensor 50.

[0121] Then, using the position of the part of the x-ray marker 150 containing the first marking device 100a in the frame defined by the optical marker 102a of the first marking device 100a and the transformation between the frame defined by the optical marker 102a of the first marking device 100a and the first frame, the position of the part of the x-ray marker 150 containing the first marking device 100a may be determined in the first frame.

[0122] Thus, the positions of every part of the x-ray marker 150 (i.e., every radiopaque element 110) may determined in the first frame (i.e., the frame defined by the second marking device 100b). The positions are each obtained in the same frame, which is the frame of the second marking device 100b defined by the optical marker 102b of the second marking device 100b. At the end of this step, the processor 22 should have obtained the position of the x-ray marker 150 in the first frame and in the second frame.

[0123] In embodiments, this step is performed for each x-ray picture. Therefore, if two x-ray pictures are acquired, a first round is performed to determine a position of the x-ray marker 150 in the first frame and in a second frame associated with the first x-ray picture and a second round is performed to determine a position of the x-ray marker 150 in the first frame and in a second frame associated with the second x-ray picture.

[0124] The method next includes a step of determining a transformation between the first frame and the second frame using said positions of the x-ray marker 150 in the first frame and in the second frame. A transformation between two frames facilitates one in determining a correlation between two or more frames. That is, a transformation allows one to determine a position of any element in a frame based on a position of said element in another frame. Thetransformation may be a mathematical function or, more generally, a mathematical model. In preferred embodiments, the transformation between the first frame and the second frame allows one to determine a position of any element in the second frame based on a position of said element in the first frame, and conversely.

[0125] In embodiments, the first frame in this step is the frame of the optical marker 102b of the second marking device 100b, i.e., the frame of the second marking device 100b. In some embodiments, the transformation is expressed as the position of the x-ray imaging device 30 during the acquiring of the x-ray picture with respect to the first frame.

[0126] The transformation may be determined using the positions of the x-ray marker 150 in the first frame and in the second frame. For example, the transformation can be a rigid transformation that defines the position and orientation of the x-ray imaging device 30 with respect to the first fame combined with a projection transformation that defines 2D positions in the x-ray picture of 3D points whose positions are determined by the X-ray imaging device 30.

[0127] In some embodiments, a matching step is performed in order to match the positions of the radiopaque elements 106 of the x-ray marker 150 in the first frame with the positions of the radiopaque elements 106 of the x-ray marker 150 into the second frame. To this end, a perspective-n-point algorithm may be used. In some embodiments, the positions of the radiopaque elements 110 of the x-ray marker 150 in the first frame are 3D positions and the positions of the radiopaque elements 110 of the x-ray marker 150 in the second frame are 2D positions. The transformation may subsequently be determined using the results of the matching, and preferably the intrinsic parameters of the s-ray imaging device 30 such as the focal distance and center position.

[0128] If the radiopaque elements 110 are in different marking devices 100a, 100b, for example, the first marking device 100a and the second marking device 100b, the matching and the positions of the reflecting elements 104a, 104b allows one to determine in which marking device 100a, 100b a radiopaque element 110 is included.

[0129] At the end of this step, a transformation between the first frame and the second frame is obtained which will allows one to determine the corresponding displacement of any optical marker in the first frame in the second frame. In some embodiments, this step is performed for each x-ray picture.

[0130] In some embodiments, when the surgery includes cutting a bone, the method includes a step of displaying, on the x-ray picture, a cutting line CL, as shown in FIGS. 5A-5B and FIGS. 7A-7B, indicating where the main bone BM should be cut to separate said main bone BM into the first bone fragment BF1 and the second bone fragment BF2. In other words, thecutting line CL is configured to indicate how to cut the main bone to separate said main bone into the first bone fragment BF1 and the second bone fragment BF2. The cutting line CL may be drawn by an operator on the X-ray picture displayed by the screen 40, directly on the display screen 40 if said display 40 is touch-sensitive or using a computer mouse.

[0131] In some embodiments, this cutting line may be determined automatically by the processor 22 using a segmented main bone BM and information regarding the surgery to perform.

[0132] In some embodiments, or in addition to or combination with those previously described, the cutting line may be determined automatically using a position of a surgical tool 60 equipped with a fourth marking device lOOd. The surgical tool 60 may be a cutting tool. The surgical tool 60 may be oriented in the direction the surgeon wants to cut the main bone BM. Using this direction, the cutting line CL may be determined and displayed on the display screen 40. This direction is determined using a determined position of the surgical tool 60, and thus of its fourth marking device lOOd, as described in greater detail below.

[0133] In embodiments, once the cutting is performed, the contour of the main bone BM (i.e., the segmented main bone BM) is separated into at least two contours, for example the contour of the first bone fragment BF1 and the contour of the second bone fragment BF2. In some embodiments, the separation may be performed automatically, as a function of the cutting line CL, or the surgeon may input where the cut was made so the contour is cut into two contours in accordance with the surgeon input. The two contours may displayed on the screen 40, along with any other information relevant to the procedure.

[0134] The method further includes a step of determining a displacement of the optical marker 102a of the first marking device 100a (or fourth marking device lOOd) in the first frame, to any marking device attached to a bone fragment such as the third marking device 100c fixed to the second bone fragment BF2. While this step is described with respect to the position of the first bone fragment BF1 in the second frame, this step is equally applicable to any other bone fragment or surgical tool equipped with an optical marker 102. According to preferred embodiments, this step is performed using data sent by the optical sensor 50 to the processor 22.

[0135] In embodiments, this step includes tracking the position of the optical marker 102a of the first marking device 100a by the optical sensor 50 which, based on the tracking, sends to the processor 22 displacement data. The position of the optical marker 102a may be easily trackable, especially using the optical sensor 50, if the optical marker 102a reflects visible, ultraviolet, and / or infrared light.

[0136] The displacement of the optical marker 102a of the first marking device 100a may be determined using an updated position of the optical marker 102a of the first marking device 100a. The optical sensor 50 detects an updated position and then, with regards to a previous position, computes the corresponding displacement. In some embodiments, the optical sensor 50 only detects positions and send them to the processor 22 which computes the displacement based on the updated positions. For example, the processor 22 may receive data from the optical sensor 50, which allows the processor 22 to compute a displacement of the optical marker 102a of the first marking device 100a.

[0137] In some embodiments, the first frame in this step is the frame of the optical sensor 50. In other preferred embodiments, the first frame in this is the frame defined by the optical marker 102b of the second marking device 100b so that the displacement of the first marking device 100a may be determined even if the body part BP moves by determining the position of the optical marker 102a of the first marking device 100a in the frame defined by the optical sensor 50. Then, the position of the optical marker 102b of the second marking device 100b in the frame defined by the optical sensor 50 may be determined. Based on that determination, the processor 22 computes the updated position of the optical marker 102a of the first marking device 100a in the frame defined by the optical marker 102b of the second marking device 100b, which is the first frame in that case. Using the updated position of the optical marker 102a of the first marking device 100a in the first frame, the displacement of the optical marker 102a, and thus of the first bone fragment BF1, may be determined.

[0138] If several, for example two, marking devices 100a, 100c are fixed on the body part, this step may be performed for the optical markers 102a, 102c of each marking device 100a, 100c. This step may be performed for any marking device 100 fixed on the body part or on a surgical tool 60.

[0139] The method further includes a step of determining the position of the first bone fragment BF1 in the second frame as a function of the displacement of the optical marker 102a of the first marking device 100a and the transformation between the first frame and the second frame. While this step is described with respect to the position of the first bone fragment BF1 in the second frame, this step is equally applicable to any other bone fragment or surgical tool equipped with an optical marker 102.

[0140] In embodiments, the displacement of the optical marker 102a of the first marking device 100a in the first frame is mapped into the second frame, using the transformation of the first and second frames. The new position of the first marking device 100a, and of the first bone fragment BF1, is therefore represented in the second frame, i.e., in the x-ray picture.

[0141] According to preferred embodiments, this step includes generating, using the processor 22, an updated x-ray picture where the position of the bone fragment is updated. In other words, in these preferred embodiments, the x-ray picture shows the first bone fragment BF1 in its new position which corresponds to the new position of the first bone fragment BF1 without acquiring a new x-ray picture, which may be displayed on the screen 40. As shown in FIGS. 7A-7B, the segmented first bone fragment BF1 may be displaced and this displacement may be reflected in the corresponding x-ray image.

[0142] In some embodiments, this step is performed for each x-ray picture if several are obtained. This step may also be performed for each marking device attached to a bone fragment, especially for example the second bone fragment BF2, or on a surgical tool 60. Indeed, once the transformation between the first frame and the second frame is determined, the position of any marking device 100 may be determined in the second frame based on its position (more precisely the position of its optical marker 102) in the first frame. Therefore, in embodiments where the first marking device 100a and the second marking device 100b together form the x-ray marker 150, there is no need for the other marking devices 100 to also form a part of the s-ray marker 150.

[0143] With this step, the position of the marking device(s) 100 attached to the bone fragments and / or the surgical tool 60 is updated. These positions may be updated sequentially or simultaneously.

[0144] In embodiments, when this step includes determining the position of a surgical tool 60, this step may also include making a determination regarding the position of an elongated tip of the surgical tool 60 using the determined position of the surgical tool 60 in the second frame (i.e., the position of the fourth marking device lOOd in the second frame) and a known position of the elongated tip with regards to the fourth marking device lOOd. In some embodiments, the surgical tool 60 is displayed on the screen 40, in the x-ray picture, such that the surgeon can precisely control its movements while using the tool.

[0145] According to preferred embodiments, for each marking device 100, the steps of determining the displacement and position of the bone fragments and / or the surgical tool 60 are performed continuously to follow or monitor the displacement of the bone fragment(s) and / or surgical tool(s) in real time. That is, using the methods and systems described herein, a surgeon may be able to precisely monitor a percutaneous surgery without needing to continuously take x-rays throughout the duration of the surgery. Rather, using the optical marker(s) 102, the position of bone fragment(s) and / or surgical tool(s) may be determined without having toacquire x-ray pictures during the surgery. Only one x-ray picture, for each desired orientation of the body part, acquired at the beginning of the surgery is needed.Surgical Method for Hallux Valgus Treatment

[0146] Using the systems and methods described herein, a percutaneous method of performing a hallux valgus surgery may be performed. This surgical method may include attaching at least a first marking device 100a on the first bone fragment BF1, with the first bone fragment BF 1 being the distal end of the first metatarsal, and attaching a second marking device 100b on the reference bone BR, with the reference bone BR being the first cuneiform, the second cuneiform, or the base of the second metatarsal. In embodiments, a third marking device 100c is attached to a second bone fragment BF2, with the second bone fragment BF2 being the base of the first metatarsal. As shown in FIG. 8, the marking device(s) may be attached to the bone using a screwdriver SD, or any other suitable surgical tool.

[0147] In some embodiments, the surgical method may be performed without the second marking device 100b on the reference bone BR. However, use of the second marking device 100b is preferable so that a first frame of reference may be defined by the second marking device 100b, since the second marking device 100b on the reference bone BR is not moved during the hallux valgus procedure. Thus, when a second marking device 100b is used, the precise position of the first marking device 100a can be determined even if the body part BP to which the first marking device 100a is attached moves.

[0148] After the marking devices 100 are attached to the appropriate bones within the patient’s foot, as shown in FIG. 8, the main bone BM may be cut to separate said main bone BM into the first bone fragment BF1 and the second bone fragment BF2. After the main bone BM is cut, the first bone fragment BF1 and the second bone fragment BF2 may be displaced while monitoring the positions thereof using the marking devices 100a, 100b. In some embodiments where a third marking device 100c is used and attached to the second bone fragment BF2, the position of this marking device 100c is monitored as well while the bones are being displaced.

Claims

CLAIMS1. A method for determining a position of at least a first bone fragment in a body part or of at least one surgical tool during a surgery involving a displacement of the first bone fragment or the surgical tool relative to a reference bone, wherein a first marking device is fixed to the first bone fragment or to the surgical tool, the first marking device comprises an optical marker, said optical marker being optically detectable, and the method is performed by a processor, the method comprising the following steps:a) obtaining an x-ray picture of the body part displaying an x-ray marker, said x- ray marker being x-ray detectable, the x-ray picture having been acquired with an x-ray imaging device, the x-ray marker having a known relative position with respect to the optical marker of the first marking device in a first frame when the x-ray picture is acquired,b) determining a position of the x-ray marker in the first frame and a position of the X-ray marker in a second frame associated with the x-ray picture,c) determining a transformation between the first frame and the second frame using said positions of the x-ray marker in the first frame and the second frame, d) determining a displacement of the optical marker of the first marking device in the first frame,e) determining the position of the first bone fragment or of the surgical tool in the second frame as a function of said displacement of said optical marker and said transformation between the first frame and the second frame.

2. The method according to claim 1, wherein a second marking device is fixed to the reference bone, the second marking device comprises an optical marker being optically detectable and wherein the first frame is defined by the optical marker of the second marking device.

3. The method according to claim 1, wherein the x-ray marker comprises at least six radiopaque elements.

4. The method according to claim 3, wherein the radiopaque elements are spherical.

5. The method according to claim 2, wherein the x-ray marker comprises at least three radiopaque elements in the first marking device and at least three radiopaque elementsin the second marking device and wherein, in step b), determining the position of the x- ray marker in the first frame includes determining the position of each radiopaque elements using:- positions of the radiopaque elements of the first marking device in a frame defined by the optical marker of the first marking device,- positions of the radiopaque elements of the second marking device in the frame defined by the optical marker of the second marking device which is the first frame, - a transformation between the frame defined by the optical marker of the first marking device and the frame defined by the optical marker of the second marking device which is the first frame.

6. The method according to claim 5, wherein the positions of the radiopaque elements of the first marking device in a frame defined by the optical marker of the first marking device are determined using a first equation system expressing a relation between said positions of the radiopaque elements and the position of the optical marker of the first marking device in the frame defined by said optical marker and / or wherein the positions of the radiopaque elements of the second marking device in the first frame are determined using a second equation system expressing a relation between said positions of the radiopaque elements of the second marking device and the position of the optical marker of the second marking device in the first frame.

7. The method according to claim 1, comprising a step of segmenting at least the first bone fragment in the x-ray picture.

8. The method according to claim 1, comprising a step of acquiring the x-ray picture with the x-ray imaging device.

9. The method according to claim 1, wherein the optical marker of the first marking device is configured to reflect or to emit one or more of visible light, ultraviolet light, and infrared light and the displacement of the optical marker of the first marking device in the first frame is determined using displacement data obtained from an optical sensor connected to the processor.

10. The method according to claim 1, wherein the optical marker comprises at least three planar or spherical optically detectable elements configured to reflect or emit one or more of visible light, ultraviolet light, and infrared light.

11. The method according to claim 1, wherein step e) comprises generating an updated x- ray picture including an updated position of the first bone fragment or of the surgical tool.

12. The method according to claim 1, wherein a third marking device is fixed to a second bone fragment, and wherein step d) comprises determining a displacement of the optical marker of the third marking device in the first frame, and a step e) of determining the position of the second bone fragment in the second frame as a function of said displacement of said optical marker and said transformation between the first frame and the second frame.

13. The method according to claim 1, wherein the first bone fragment and a second bone fragment are parts of a main bone and the method includes a step of displaying on the x-ray picture a cutting line for cutting the main bone to separate said main bone into the first bone fragment and the second bone fragment.

14. The method according to claim 1, wherein, in step a), two x-ray pictures of the body part are obtained and each x-ray picture represents a different orientation of the body part.

15. The method according to claim 1, wherein the body part is a foot and the surgery corresponds to treatment of a hallux valgus.

16. A method for treating a hallux valgus percutaneously including a step x) of fixing a first marking device on a first bone fragment being a part of the first metatarsal and a second marking device on a reference bone being the first cuneiform, the second cuneiform or the base of the second metatarsal, a step y) of cutting the main bone to separate said main bone into the first bone fragment and a second bone fragment, a step z) of displacing the first bone fragment relative to the reference bone, said method comprisingmonitoring the position of the first bone fragment by performing the method according to claim 2.

17. A computing device comprising a processor configured to perform the method according to claim 1.

18. A non-transitory computer readable medium comprising a computer program product stored thereon comprising program code instructions for executing the method of claim 1.

19. An optical marker configured for use in performing the method according to claim 1.

20. A system for use in a surgery involving a displacement of at least one bone fragment in a body part of a patient, the system comprising:at least one marking device configured to be attached to the at least one bone fragment, the marking device comprising:at least one optical marker, the optical marker being optically detectable; andone or more radiopaque elements; andan x-ray marker formed of the one or more radiopaque elements of the at least one marking device, wherein the x-ray marker is configured to be x-ray detectable; and a processor;wherein the processor is configured to:receive an x-ray picture of a body part displaying the x-ray marker, the x-ray marker having a known relative position with respect to the optical marker; anddetermine a position of the x-ray marker and, using the position of the x- ray marker, determine a displacement of the optical marker,wherein the displacement of the optical marker is indicative of the displacement of the at least one marking device, indicating a position of the at least one bone fragment to which the at least one marking device is attached.

21. The system of claim 20, wherein the at least one marking device comprises three radiopaque elements, and wherein the x-ray marker comprises at least six radiopaque elements.

22. The system of claim 20, wherein the optical marker of the at least one marking device comprises one or more light reflecting elements.

23. The system of claim 20, wherein the at least one marking device has a surface area of about 12 cm2or less.

24. The system of claim 20, wherein the body part of the patient is the foot, and wherein the at least one bone fragment is a part of the first metatarsal, the second metatarsal, the first cuneiform, or the second cuneiform within the foot.

25. The system of claim 20, wherein the system is used in a surgical method for treating a hallux valgus.

Images