Surgery assistance system for joint laxity assessment

Abstract

A surgical assistance system may have a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining at least one radiographic image of a patient prior to a surgical procedure; image processing the at least on radiographic image of the patient using a parametrized machine learning module; and outputting implant parameter recommendations to an operator of the surgical procedure.

Description

SURGERY ASSISTANCE SYSTEMFOR JOINT LAXITY ASSESSMENTCROSS-REFERENCE TO RELATED APPLICATION

[0001] The present application claims the benefit of United States Patent Application No. 63 / 664,143, filed on June 25, 2024, the entire content of which is incorporated herein by reference.TECHNICAL FIELD

[0002] The present application relates to computer-assisted surgery, such as computer-assisted surgery systems used in orthopedic surgery and to robotic surgery systems, in which implants are implanted in patient joints.BACKGROUND OF THE ART

[0003] Computer-assisted surgery commonly provides an operator with navigation data during a surgical procedure. The navigation data may take various forms, including position and orientation data pertaining to bones and tools, predicted alterations, imaging, etc. The computer-assisted surgery systems may also include robotic apparatuses to perform some steps of surgical procedures.

[0004] In spite of computer assistance, one of the main sources of uncertainty in orthopedic joint surgery, despite careful planning, is joint laxity, i.e., the looseness of ligaments in a joint, the knee joint being a common example due to the prevalence of knee surgery among orthopedic joint surgery. If the ligaments are too loose, for example, the surgical plan may have to be changed live intra-operatively. A surgeon may be obligated to use thicker implants, this requires a change in bone cuts, inventory has to be available, the procedure takes more time, to name a few of the adjustments that may occur because of changes to the surgical plan. This may result in an increase in duration of a surgical procedure, delays, non-optimal implant selection, etc.SUMMARY

[0005] In accordance with a first aspect of the present disclosure, there is provided a surgical assistance system comprising: a processing unit; and a non-transitorycomputer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining at least one radiographic image of a joint of a patient prior to a surgical procedure; image processing the at least one radiographic image of the patient using a parametrized machine learning module; and outputting joint laxity data for the joint of the patient to an operator of the surgical procedure.

[0006] Further in accordance with the first aspect, for instance, implant parameter recommendations are output as associated with the joint laxity data.

[0007] Still further in accordance with the first aspect, for instance, bone cut parameters are output as associated with the implant parameters.

[0008] Still further in accordance with the first aspect, for instance, control data is output to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

[0009] Still further in accordance with the first aspect, for instance, a robot arm of the robotic computer-assisted surgery controller is driven based on bone cut parameters associated with the implant parameters.

[0010] Still further in accordance with the first aspect, for instance, the driving of the robot arm of the robotic computer-assisted surgery controller includes tracking the robot arm relative to at least one bone of the joint to position an end effector of the robot arm relative to the at least one bone as a function of the bone cut parameters.

[0011] Still further in accordance with the first aspect, for instance, the machine learning module is trained by performing image processing of radiographic images of past surgical procedures.

[0012] Still further in accordance with the first aspect, for instance, control data may be obtained from computer-assisted surgery controllers used during the past surgical procedures, and wherein the machine learning module is trained using said control data.

[0013] Still further in accordance with the first aspect, for instance, patient data associated with the radiographic images of the past surgical procedures may be obtained, the patient data including one or more of age, gender, race, ethnicity, genetics,height, weight, body mass index, congenital conditions, pathologies, medical history, and wherein the machine learning module is trained using said patient data.

[0014] Still further in accordance with the first aspect, for instance, tool data may be obtained from computer-assisted surgery controllers used during the part surgical procedures, and wherein the machine learning module is trained using the tool data.

[0015] Still further in accordance with the first aspect, for instance, post-operative assessment data may be obtained as associated with the past surgical procedures, and wherein the machine learning module is trained using said post-operative assessment data.

[0016] Still further in accordance with the first aspect, for instance, obtaining postoperative assessment data includes receiving quantitative post-operative assessment data.

[0017] Still further in accordance with the first aspect, for instance, obtaining postoperative assessment data includes receiving qualitative post-operative assessment data.

[0018] In accordance with a second aspect of the present disclosure, there is provided a surgical assistance system comprising: a processing unit; and a non- transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining radiographic images of joints of patients before surgical procedures to at least one bone of the joints; obtaining joint laxity data associated with the radiographic images of the joints of the patients during or pursuant to the surgical procedures; training a machine learning module using the radiographic images and the implant data; and outputting the machine learning module parametrized to output joint laxity data as a function of at least one radiographic image of a patient prior to a surgical procedure.

[0019] Further in accordance with the second aspect, for instance, the machine learning module is trained by performing image processing of the radiographic images of the surgical procedures.

[0020] Still further in accordance with the second aspect, for instance, control data may be from computer-assisted surgery controllers used during the surgical procedures, and wherein the machine learning module is trained using said control data.

[0021] Still further in accordance with the second aspect, for instance, patient data may be associated with the radiographic images, the patient data including one or more of age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, and wherein the machine learning module is trained using said patient data.

[0022] Still further in accordance with the second aspect, for instance, tool data obtaining from computer-assisted surgery controllers used during the surgical procedures, and wherein the machine learning module is trained using the tool data.

[0023] Still further in accordance with the second aspect, for instance, post-operative assessment data is obtained as associated with the surgical procedures, and wherein the machine learning module is trained using said post-operative assessment data.

[0024] Still further in accordance with the second aspect, for instance, obtaining postoperative assessment data includes receiving quantitative post-operative assessment data.

[0025] Still further in accordance with the second aspect, for instance, obtaining postoperative assessment data includes receiving qualitative post-operative assessment data.

[0026] Still further in accordance with the second aspect, for instance, implant data is obtained as associated with the radiographic images of the joints of the patients pursuant to the surgical procedures.

[0027] In accordance with a third aspect of the present disclosure, there is provided a system comprising: the surgical assistance as described above, a robotic computer- assisted surgery controller configured to be operated based on bone cut parameters associated implant parameters output by the surgical assistance system.

[0028] Further in accordance with the third aspect, for instance, the robotic computer- assisted surgery controller includes a robot arm configured to be driven based on the bone cut parameters associated with the implant parameters.

[0029] Still further in accordance with the third aspect, for instance, a tracking system may be included and configured to track the robot arm relative to at least one bone of the joint to position an end effector of the robot arm relative to the at least one bone as a function of the bone cut parameters.DESCRIPTION OF THE DRAWINGS

[0030] Fig. 1 is a schematic view of a surgery assistance system in accordance with a variant of the present disclosure, in training;

[0031] Fig. 2 is a block diagram of the surgery assistance system of Fig. 1 , as used in support of a surgical procedure performed using a computer-assisted surgery system, in accordance with another variant of the present disclosure; and

[0032] Fig. 3 is a schematic view of a robotic arm of a robot arm of the computer- assisted surgery system.DETAILED DESCRIPTION

[0033] Referring to Fig. 1, a surgery assistance (SA) system in accordance with the present disclosure is generally shown at 10. The SA system 10 may include an operating system for use during computer-assisted surgery, such as shown in Fig. 2, optionally including with a robot 20. As illustrated, the SA system 10 may for example be used in knee replacement surgery, whether the knee replacement surgery involve a full or partial tibial plateau implant and / or a femoral implant. The SA system 10 could be used for other body parts, including non-exhaustively hip joint, spine, and shoulder bones, in orthopedic surgery in which bones are altered to receive implants, or other types of surgery, typically of the type in which joints are operated in, such as by the placement of an implant at the articulation.

[0034] Referring to Fig. 1 , the SA system 10 may include a server 12 having one or more processing units 14, such as conventional central processing unit(s) (CPll(s)), and a non-transitory computer-readable memory 16 communicatively coupled to the one or more processing units 14. The memory 16 may store therein computer-readable instructions. The server 12 may be implemented in any suitable way. For example, in the present embodiment the server 12 is shown as a single entity, which may be a suitable computer for example with the associated communications hardware, input-output hardware, and controls hardware to give a few examples, all of which may be selected, for example from conventional hardware, to suit each particular embodiment of the other components of the SA system 10. In other embodiments, the server 12 may be implemented in a distributed manner, with for example parts thereof being physical parts while other parts may be simulated, such as via virtual machines for example, and / or yet other parts thereof may be remote. The server 12 may also be cloud-based.

[0035] The machine-learning module 18 is the learning module of the SA system 10 that receives data from surgical procedures to parametrize / train a machine-learning algorithm (MLA), which may be stored in the non-transitory memory 16 of the server 12, or in any other suitable non-transitory memory such as remote memory on the cloud, for example.

[0036] The assistance module 19 is the intervening module of the SA system 10 that provides machine-learned assistance during a surgical procedure based on the parametrized / trained machine-learning algorithm (MI_A).

[0037] Referring to Fig. 1 , the ML module 18 performs data acquisition for subsequent training of the MLA. The MLA may be trained to have the capacity of evaluating joint laxity, for optionally outputting implant parameters, as a function of one or more radiographic images of a patient obtained prior to a surgical procedure. In Fig. 1, sets of radiographic images A1 are shown, as illustratively part of a patient file A2. Therefore, according to an embodiment, the machine-learning module 18 receives radiographic images from past surgical procedures. The images may be in the form of an image feed and / or video feed from different types of radiography. For example, the images and / or video feed may be obtained from a standard X-ray machine, a fluoroscope, a CT scanner, a C-arm. It is also possible that the images be obtained from other imaging modalities, including magnetic-resonance imagery (MRI).

[0038] In Fig. 1, the radiographic images are shown as an example as being for the patient’s knee. The patient file A2 could have a single radiographic image. In the case of a knee, the radiographic images may be that of the knee in extension, such as in a load bearing condition, as this is a common occurrence. The radiographic images may be of other knee stances, including in flexion, in extension with a distraction force applied to the leg, as optional conditions of the leg in the radiographic images.Moreover, the radiographic images may include images from a lateral point of view and / or from an anterior point of view. It may be easier to generate radiographic images when applying a distraction force, i.e., pulling the tibia away from the femur, than to generate images in a load-bearing condition of the leg. Distraction forces may also be measured, such as if applied by a robot arm or by a human operator,

[0039] After data acquisition, the ML module 18 trains its machine-learning algorithm (MLA). In a variant, the training includes performing image processing of the radiographic images A1, in order to identify visual cues or aspects associated with the bones, enabling the ML module 18 to associate the visual aspects with joint laxity, and / or implant selection. In an embodiment, the image processing is done locally, in edge computing. There may be numerous visual cues or aspects that can be observed in the radiographic images, enabling the ML module 18 to correlate the radiographic image to implant parameters.

[0040] For example, the image processing by the ML module 18 may allow same to identify a bone contour, three-dimensional geometry, a contact surface between adjacent bones in an articulation, wear of the cartilage surfaces and / or of the bone surfaces at the articulation. The image processing by the ML module 18 may analyse bone thickness and / or density, cartilage thickness and / or density via the contrast of the radiographic image. More particularly, the ML module 18 may process the images to identify signs of wear in the articulation, whether it be wear to the bone and / or cartilage. For example, the image processing by the ML module 18 may also observe the presence of osteophytes or like bone spurs at the joints. The training performed by the ML module 18 may consider such osteophytes as an indication that the bone has worn, and thus that the ligaments have gained laxity and looseness due to the wear. As another example, the gap between the bones of the articulation may also be an indication of wear. More particularly, the radiographic images may not display cartilage with sufficient resolution, but a gap between bones being lower than standard gaps may indicate the wear of cartilage, and thus a gain of laxity and looseness in the joint due to the wear. Thus, some of these visual cues or aspects observed in the image processing by the ML module 18 may allow the ML module 18 to learn joint laxity values as a function of the visual cues or aspects, notably if the patient files A2 include values forjoint laxity. The patient files A2 used in the training of the MLA may be limited to the radiographic images A1 prior to the surgical procedure.

[0041] The visual cues or aspects, and assessed joint laxity may also be correlated to the implant selection. Moreover, as explained below, post-operative assessments may also be considered, to further train the ML module 18. Additional information may be used for the training, such as data on the implant that was selected for the patient. The implant that was selected may be in the form of a brand, a type, a model, a serial number, and may also include information such as dimensions, a three-dimensional model, elevation views, etc. Alternatively, the implant may be identified using image processing of radiographic images after the surgical procedure, as shown at A3, in the training of the MLA.

[0042] The data acquisition may take other forms as well. The ML module 18 may supplement radiographic images A1 and implant type with patient-related data, such as non-confidential data for privacy reasons. The data that is part of the patient file A2 obtained by the ML module 18 may include patient data such as age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, etc. The data acquired by the ML module 18 may also include surgical flow information from the past surgical procedures, as well as control data from a CAS system, such as from the CAS controller 50 (Fig. 2), that performed the past surgical procedure. This may include an identification of tools used, bones being altered, navigation data (e.g., position and / or orientation of tools relative to the bones in a referential system) the parameters of alteration (depth of cuts, orientation, navigation data), navigation of robot arm 20 if present.

[0043] In some instances, an assessment of the surgery is done post-operatively, and may be added to the patient file A2. The ML module 18 may access this information as part of data acquisition as well. The assessment of surgery may take various forms, including quantitative data. In the case of orthopedic surgery, the quantitative data may be joint laxity values, distance or length data, such as limb length discrepancy, cut depth. The quantitative data may be orientation data, such as varus / valgus, offset, tilt, etc. The quantitative data may be volumetric data, such as volume of bone removed, volume of resection. The assessment may also include qualitative data, with patientfeedback including pain level, perceived mobility, joint looseness, patient satisfaction score, etc. The assessment data may be acquired over a rehabilitation period, with post-operative patient follow ups and the use of wearable sensor technologies, for instance over an extended period of time.

[0044] The training of the ML algorithm may be based on training data acquired from multiple prior surgeries, in different locations, from different SA systems 10, and / or involving different surgeons. The training of the ML algorithm in the ML module 18 may include at least 100 surgical procedures, without an upper echelon of review. The machine learning algorithm may be trained with or without supervision by image processing radiographic images of surgeries for patients of different age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, etc, to train the ML algorithm with procedures covering a wide diversity of cases, including standard cases, and deviation cases. Age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, etc may have an impact on the type of implant selected in prior surgical procedures used as training data for the ML module 18. Training can also be done based on surgeon preferences.

[0045] As a consequence of the training of the ML algorithm, the learning module 18 may produce and output a parametrized ML algorithm. The ML algorithm may be selected from different supervised machine learning algorithms, such as neural networks, Bayesian networks, support vector machines, instance-based learning, decision trees, random forests, linear classifiers, quadratic classifiers, linear regression, logistic regression, k-nearest neighbor, hidden Markov models, or the like. The ML algorithm may be selected from different unsupervised machine learning algorithms, such as expectation-maximization algorithms, vector quantization, and information bottleneck method.

[0046] Thus, if available, the assessment of the surgery done post-operatively, such as in the form of the quantitative data and / or qualitative data, may be associated with the selection of implant and radiographic images to train the ML algorithm in evaluating the success of a surgical procedure as per the radiographic images of the bone and the implant selected, and its numerous parameters as a function of post-operativeassessment. There results a trained ML algorithm in the ML module 18. The trained ML algorithm may have the capacity of performing various functions through its training, including outputting values of joint laxity from an analysis of radiographic images of a joint, such as the knee joint. With the values or like assessment of joint laxity, the trained ML algorithm may provide implant recommendations.

[0047] Accordingly, the SA system 10 may be defined as being a surgical assistance system having: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining radiographic images of joints of patients before surgical procedures to at least one bone of the joints; obtaining implant data associated with the radiographic images of the joints of the patients pursuant to the surgical procedures; training a machine learning module using the radiographic images and the implant data; and outputting the machine learning module parametrized to output joint laxity data as a function of at least one radiographic image of a patient prior to a surgical procedure. Joint laxity data may be expressed as a shortest distance between points of contact between tibia and femur, from a medial position and from a lateral position. For example, joint laxity data may be expressed in millimetres, e.g., 6mm medial and 8mm lateral, for a knee in extension and for a knee in extension. The shortest distances may be between the medial condyles of the femur and tibia, and between the lateral condyles of the femur and tibia.

[0048] Referring to Figs. 1 and 2, the assistance module 19 has been updated with the parametrized machine learning algorithm. The assistance module 19 may be the part of the SA system 10 that is used pre-operatively, peri-operatively, or intraoperatively, to assist a user in assessing joint laxity prior to surgery. This joint laxity assessment may also be used to recommend an implant type. In an embodiment, the assistance module 19 accesses the server 12 in which the parametrized ML algorithm is located, for instance locally in edge computing, but in other embodiments the ML algorithm may be on the cloud for example. Pre-operatively, peri-operatively, and / or intraoperatively, data acquisition is performed by the assistance module 19, to output implant parameters (e.g., type, brand, dimensions, etc). The implant parameters may be with a goal to achieve a desired level of joint laxity. The assistance module 19 assisting the user atleast by obtaining and image processing one or more radiographic images of the patient, ahead of surgery.

[0049] Therefore, the assistance module 19 obtains one or more radiographic images ahead of a surgical procedure. The images may be in the form of an image feed and / or video feed from different types of radiography. For example, the images and / or video feed may be obtained from a standard X-ray machine, a fluoroscope, a CT scanner, a C- arm. It is also possible that the images be obtained from other imaging modalities, including magnetic-resonance imagery (MRI). In the case of a knee, the radiographic images may be that of the knee in extension, such as in a load bearing condition, though this is optional. Moreover, the radiographic images may include images from a lateral point of view and / or from an anterior point of view. This or these images are obtained ahead of surgery.

[0050] In similar fashion to the ML module 18, the assistance module 19 may perform image processing of the radiographic images A1 , in order to identify visual cues or aspects associated with the bones, enabling the ML module 18 to use the visual aspects to proceed to implant selection. In an embodiment, the image processing is done locally, in edge computing. There may be numerous visual cues or aspects that can be observed in the radiographic images, enabling the assistance module 19 to identify a suitable implant.

[0051] For example, the image processing by the assistance module 19 may allow same to identify a bone contour, three-dimensional geometry, a contact surface between adjacent bones in an articulation, wear to the bone and / or cartilage in the articular surface. The image processing by the assistance module 19 may analyse bone thickness and / or density via the contrast of the radiographic image. The image processing by the assistance module 19 may also observe the presence of osteophytes or like bone spurs at the joints. For example, the image processing by the assistance module 19 may also observe the presence of osteophytes or like bone spurs at the joints. The assistance module 19 may consider such osteophytes as an indication that the bone has worn, and thus that the ligaments have gained laxity and looseness due to the wear. As another example, the gap between the bones of the articulation may also be an indication of wear. More particularly, the radiographic images may not displaycartilage with sufficient resolution, but a gap between bones being lower than standard gaps may indicate the wear of cartilage, and thus a gain of laxity and looseness in the joint due to the wear. Thus, some of these visual cues or aspects observed in the image processing by the assistance module 19 may allow the assistance module 19 to output joint laxity values, or an assessment of joint laxity, as a function of the visual cues or aspects. The assistance module 19 of the SA system 10 can output laxity parameters, such as a distance value in mm, inches, for each compartment, such a medial value and a lateral value. The laxity data generated by the SA system 10 can later be used by a planning software (as in Fig. 2) to properly select the ideal implant size and position based on surgeon preferences.

[0052] Thus, some of these visual cues or aspects observed in the image processing by the assistance module 19 may allow the assistance module to predict joint laxity. The visual cues or aspects, and predicted joint laxity may be correlated to implant size and thus to implant selection as proposed by the assistance module 19, such as in the form of implant parameters. Thus, the SA system 10 can directly generate by its machinelearning module a size recommendation and position for each implant based on a surgeon preference card or based on historical positioning done by this surgeon, in addition to other parameters. The surgeon cases could be used to train a SA algorithm to identify the specific typical surgeon preferences.

[0053] Other forms of data acquisition may be used by the assistance module 19. This may also include accessing the patient files A2, with its content that may be as described above. For example, this may include patient data such as age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, etc.

[0054] Based on this, the assistance module 19 may output bone cut parameters associated with the implant parameters, with the bone cut parameters optionally provide to a CAS controller, such as that shown at 50 in Fig. 2. If the CAS system is of the type having a robot, such as robot arm 20 in Fig. 3, the assistance module 19 may output and provide control data to operate the robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters. Based on output from the parametrized machine learning algorithm, the assistance module 19 may propose asurgical procedure flow using the available data: patient profile, type of surgery, tool accessibility, nature of procedure.

[0055] Moreover, if the assistance module 19 is used during surgery, the assistance module 19 may have access to live video footage from a camera or tracker device 40, may communicate with the CAS system 50, including the robotized surgery controller 70, input from a surgeon, access to post-operative qualitative and / or quantitative assessment data, etc.

[0056] Referring to Fig. 2, the SA system 10 is shown as being part of CAS set up used during a surgical procedure. The CAS featuring the SA system 10 in intraoperative set up may or may not be robotized. The SA system 10 may operate in parallel with a computer-assisted surgery system CAS. The CAS may thus be robotized in a variant, and has, may have or may be used with a robot 20, optical trackers 30, a tracker device 40, a CAS controller 50 (also known as a super controller 50), a tracking module 60, and a robot controller 70 (also known as a robot driver), or any combination thereof, with the CAS optionally receiving guidance from the SA system 10 to guide its operations, for example after the SA system 10 provides its joint laxity assessment and optionally, its implant recommendations.

[0057] The SA system 10 as used with the CAS may thus be described as including a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining at least one radiographic image of a joint of a patient prior to a surgical procedure; image processing the at least one radiographic image of the patient using a parametrized machine learning module; and outputting joint laxity data for the joint of the patient to an operator of the surgical procedure. Optionally, the SA system 10 may output implant parameter recommendations associated with the joint laxity data. In the context of the CAS, the SA system 10 may outputting bone cut parameters associated with the implant parameters. The SA system 10 may output control data to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

[0058] The robot 20, shown by its robot arm in Fig. 3, may be present as the working end of the CAS, and may be used to perform or guide bone alterations as planned by anoperator and / or the CAS controller 50 and as controlled by the CAS controller 50. While a robot arm is displayed, the robot, robot system and robotic process may be implemented in the form of an orientation mechanism without a robotic arm. More specifically, interconnected links with joints that may be lockable supporting a tool may constitute a robot in accordance with the present disclosure. The robot arm 20 may also be configured for collaborative / cooperative mode in which the operator may manipulate the robot arm 20. For example, the tooling end, also known as end effector, may be manipulated by the operator while supported and oriented by the robot arm 20. The robot 20 may be the coordinate measuring machine (CMM) of the robotic system 10.

[0059] The optical trackers 30 are positioned on the robot 20, on patient tissue (e.g., bones B), and / or on the tool(s) T and surgical instruments, and provide tracking data for the robot 20, the patient and / or tools.

[0060] The tracking device 40, also known as a sensor device, apparatus, etc performs optical tracking of the optical trackers 30, so as to enable the tracking in space (a.k.a., navigation) of the robot 20, the patient and / or tools.

[0061] The optical trackers 30 and tracking device 40 are an optional tracking modality of the robotic system 10. Other camera(s) may be present, for instance as a complementary registration tool or as a primary tracking tool. The camera may for instance be mounted on the robot 20, such as on the robot arm, such that the point of view of the camera is known in the frame of reference, also known as the coordinate system. Tracking may also be done using the robot 20 and its joint tracking capacity, as an alternative, or in addition to the tracking modalities described above. Moreover, other tracking modalities may include the use of inertial sensors, etc.

[0062] The CAS controller 50, also known as the super controller, includes the processor(s) and appropriate hardware and software to run a computer-assisted surgery procedure in accordance with one or more workflows. The CAS controller 50 may include or operate the tracking device 40, the tracking module 60, and / or the robot controller 70. As described hereinafter, the CAS controller 50 may also drive the robot arm 20 through a planned surgical procedure. The CAS controller 50 may be operated based on the control data provided by the SA system 10 to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

[0063] The tracking module 60 is tasked with determining the position and / or orientation of the various relevant objects during the surgery procedure, such as the end effector of the robot arm 20, bone(s) B and tool(s) T, using data acquired by the tracking device 40 and by the robot 20, and / or obtained from the robot controller 70. The position and / or orientation may be used by the CAS controller 50 to control the robot arm 20.

[0064] The robot controller 70 is tasked with powering or controlling the various joints of the robot arm 20, based on operator demands or on surgery planning. The robot controller 70 may also optionally calculate robot movements of the robot arm 20, so as to control movements of the robot arm 20 autonomously in some instances, i.e., without intervention from the CAS controller 50. The robot controller 70 may be operated based on the control data provided by the SA system 10 to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

[0065] Other components, devices, systems, may be present, such as surgical instruments and tools T, interfaces l / F such as displays, screens, computer station, servers, and like etc. Secondary tracking systems may also be used for redundancy.

[0066] Referring to Fig. 3, the robot 20 may have the robot arm 20 may be secured to the OR table 80 in a variant of the present disclosure. The robot arm 20 may alternatively stand from a base, for instance in a fixed relation relative to supporting the patient, whether it is attached to or detached from the OR table 80. The robot arm 20 has a plurality of joints 21 and links 22, of any appropriate form, to support an end effector 23 that may interface with the patient, or may be used during surgery without interfacing with the patient. In Fig. 1 , merely as an example, the end effector 23 is shown as being a reamer, that is rotatably supported at the end of the robot arm 20 for rotating about its shaft axis. In the case of a knee surgery, the end effector 23 may be a cut guide, or cut blade, as examples among others. The end effector or tool head may optionally incorporate a force / torque sensor for collaborative / cooperative control mode, in which an operator manipulates the robot arm 20. The robot arm 20 is shown being a serial mechanism, arranged for the tool head 23 to be displaceable in a desired numberof degrees of freedom (DOF). The tool head 23 may for example be a support that is not actuated, the support being used to support a tool, with the robot arm 20 used to position the tool relative to the patient. For example, the robot arm 20 controls 6-DOF movements of the tool head 23, i.e., X, Y, Z in the coordinate system, and pitch, roll and yaw. Fewer or additional DOFs may be present, as shown below with reference to Fig. 3. For simplicity, only a fragmented illustration of the joints 21 and links 22 is provided, but more joints 21 of different types may be present to move the end effector 23 in the manner described above. The joints 21 are powered for the robot arm 20 to move as controlled by the CAS controller 50 (i.e., including the robot controller 70) in the six DOFs, and in such a way that the position and orientation of the end effector 23 in the coordinate system may be known, for instance by readings from encoders on the various joints 21. Therefore, the powering of the joints is such that the end effector 23 of the robot arm 20 may execute precise movements, such as moving along a single direction in one translation DOF, or being restricted to moving along a plane, among possibilities. Such robot arms 20 are known, for instance as described in United States Patent Application Serial no. 11 / 610,728, and incorporated herein by reference.

[0067] The end effector 23 of robot arm 20 may be defined by a chuck or like tool interface, typically actuatable in rotation. As a non-exhaustive example, numerous tools may be used as end effector for the robot arm 20, such tools including a registration pointer, a cut guide, a cut blade, a reamer (e.g., cylindrical, tapered), a reciprocating saw, a retractor, a camera, an ultrasound unit, a laser rangefinder or light-emitting device (e.g., the indicator device of US Patent No. 8,882,777), a laminar spreader, an instrument holder, or a cutting guide, depending on the nature of the surgery. The various tools may be part of a multi-mandible configuration or may be interchangeable, whether with human assistance, or as an automated process. The installation of a tool in the tool head may then require some calibration in order to track the installed tool in the X, Y, Z coordinate system of the robot arm 20.

[0068] The end effector 23 of the robot arm 20 may be positioned by the robot 20 relative to the surgical area in a desired orientation according to a surgical plan, such as a plan based on preoperative imaging. In order to position the end effector 23 of the robot arm 20 relative to the patient B, the CAS controller 50 can manipulate the robot arm 20 automatically (without human intervention), or in a collaborative mode by asurgeon manually operating the robot arm 20 (e.g. physically manipulating, via a remote controller through the interface l / F) to move the end effector 23 of the robot arm 20 to the desired location, e.g., a location called for by a surgical plan to align an instrument relative to the anatomy. Once aligned, a step of a surgical procedure can be performed, such as by using the end effector 23.

[0069] The CAS system may be without the robot arm 20, with the operator performing manual tasks. In such a scenario, the CAS system may only have the CAS controller 50, the tracking apparatus 60. In another embodiment, the CAS system is one used without robotic assistance, and assists an operator by way of surgical navigation, i.e. , tracking the surgical instrument(s) relative to the bone(s) in orthopedic surgery. The CAS system may also have non-actuated foot support 30 and thigh support 40 to secure the limb. When it operates the robot arm 20, the CAS system may drive the robot arm 20 autonomously, and / or as an assistive or collaborative tool for an operator (e.g., surgeon). The surgical workflow of the CAS system may be operated based on the control data provided by the SA system 10 to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

[0070] Referring to Fig. 2, the optional trackers 30 are shown secured to the bones B and at various locations on the robot 20, and may also or alternatively be on instruments. The trackers 30 may be known as trackable elements, markers, navigation markers, active sensors (e.g., wired or wireless) that may for example include infrared emitters. In a variant, the trackers 30 are passive retro- reflective elements, that reflect light. The trackers 30 have a known geometry so as to be recognizably through detection by the tracker device 40. For example, the trackers 30 may be retro-reflective lenses. Such trackers 30 may be hemispherical in shape, by way of a shield. The shield may be hollow and may cover a reflective membrane or surface. In an embodiment, the trackers 30 may be active emitters.

[0071] The tracker device 40 may be embodied by an image capture device, capable of illuminating its environment. In a variant, the tracker device 40 may have two (or more) points of view, such that triangulation can be used to determine the position of the tracker devices 30 in space, i.e., the coordinate system of the CAS. The tracker device40 may emit light, or use ambient light, to observe the trackers 30 from its points of view, so as to determine a position of the trackers 30 relative to itself. In an embodiment, the tracker device 40 is of the type known as the Polaris products by Northern Digital Inc. The tracker device 40 may form the complementary part of the CMM function of the CAS, with the trackers 30 on the robot base 20 for example.

[0072] Referring to Fig. 2, the CAS controller 50 is shown in greater detail relative to the other components of the robotic system 10. The CAS controller 50 has a processor unit 51 and a non-transitory computer-readable memory 52 communicatively coupled to the processing unit 51 and configured for executing computer-readable program instructions executable by the processing unit 51 to perform some functions, such as tracking the patient tissue and tools, using the position and orientation data from the robot 20 and the readings from the tracker device 40. Accordingly, as part of the operation of the CAS controller 50, the computer-readable program instructions may include an operating system that may be viewed by a user or operator as a GUI on one or more of the interfaces of the CAS. It is via this or these interfaces that the user or operator may interface with the robotic system, be guided by a surgical workflow, obtain navigation data, etc. The CAS controller 50 may also control the movement of the robot arm 20 via the robot controller module 70. The CAS may comprise various types of interfaces l / F, for the information to be provided to the operator. The interfaces l / F may include and / or screens including wireless portable devices (e.g., phones, tablets), audio guidance, LED displays, head-mounted display for virtual reality, augmented reality, mixed reality, among many other possibilities. For example, the interface l / F comprises a graphic-user interface (GUI) operated by the CAS. The CAS controller 50 may also display images captured pre-operatively, or using cameras associated with the procedure (e.g., 3D camera, laparoscopic cameras, tool mounted cameras), for instance to be used in the collaborative / cooperative control mode of the system 10, or for visual supervision by the operator of the CAS, with augmented reality for example. The CAS controller 50 may drive the robot arm 20, in performing the surgical procedure based on the surgery planning achieved pre-operatively, or in maintaining a given position and orientation to support a tool. The CAS controller 50 may run various modules, in the form of algorithms, code, non-transient executable instructions, etc, in order to operate the CAS in the manner described herein. The CAS controller 50 may be part of anysuitable processor unit, such as a personal computer or computers including laptops and desktops, tablets, server, etc.

[0073] The tracking module 60 may be a subpart of the CAS controller 50, or an independent module or system. The tracking module 60 receives the position and orientation data from the robot 20 and the readings from the tracker device 40. The tracking module 60 may hence determine the relative position of the objects relative to the robot arm 20 in a manner described below. The tracking module 60 may also be provided with models of the objects to be tracked. For example, the tracking module 60 may track bones and tools, and hence may use virtual bone models and tool models. The bone models may be acquired from pre-operative imaging (e.g., MRI, CT-scans), for example in 3D or in multiple 2D views, including with 2D X-ray to 3D bone model technologies. The virtual bone models may also include some image processing done preoperatively, for example to remove soft tissue or refine the surfaces that will be exposed and tracked. The virtual bone models may be of greater resolution at the parts of the bone that will be tracked during surgery, such as the knee articulation in knee surgery. The bone models may also carry additional orientation data, such as various axes (e.g., longitudinal axis, mechanical axis, etc). The bone models may therefore be patient specific. It is also considered to obtain bone models from a bone model library, with the data obtained from the video images used to match a generated 3D surface of the bone with a bone from the bone atlas. The virtual tool models may be provided by the tool manufacturer, or may also be generated in any appropriate way so as to be a virtual 3D representation of the tool(s).

[0074] Additional data may also be available, such as tool orientation (e.g., axis data and geometry). By having access to bone and tool models, the tracking module 60 may obtain additional information, such as the axes related to bones or tools.

[0075] Still referring to Fig. 2, the CAS controller 50 may have the robot controller 70 integrated therein. However, the robot controller 70 may be physically separated from the CAS controller 50, for instance by being integrated into the robot 20 (e.g., in the robot base 20B). The robot controller 70 is tasked with powering and / or controlling the various joints of the robot arm 20. The robot controller 70 may also optionally calculate robot movements of the robot arm 20, so as to control movements of the robot arm 20autonomously in some instances, i.e., without intervention from the CAS controller 50. There may be some force feedback provided by the robot arm 20, for instance via the force sensor 24 or other sensors 25 to avoid damaging the bones, to avoid impacting other parts of the patient or equipment and / or personnel. The robot controller 70 may perform actions based on a surgery planning, and on control data provided by the SA system 10 to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters. The surgery planning may be a module programmed specifically for any given patient, according to the parameters of surgery desired by an operator such as an engineer and / or surgeon. The parameters may include geometry of selected, planned bone cuts, planned cut depths, sequence or workflow of alterations with a sequence of surgical steps and tools, tools used, etc.

[0076] The SA system 10 therefore relies on common pre-operative imaging, namely the X-rays of the knee in extension such as in a load bearing condition, for its MLA to be trained and to predict joint laxity with more accuracy. Indeed, the MLA relies on the X- rays of a patient to analyze bone wear, osteophytes, and other cues that could be determinative of joint laxity. With the analysis of the X-rays, joint laxity may be accurately estimated. The SA system 10 may use the bone atlas of thousands of X-rays and associated intra-operative details such as implants used, joint laxity, patient details, etc. to train a machine-learning algorithm. The MLA may establish a correlation between the X-ray images and joint laxity.

[0077] Therefore, the present disclosure discloses a surgical assistance system that may include a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining radiographic images of joints of patients before surgical procedures to at least one bone of the joints; obtaining joint laxity data and optionally implant data associated with the radiographic images of the joints of the patients during or pursuant to the surgical procedures; training a machine learning module using the radiographic images and the implant data; and outputting the machine learning module parametrized to output joint laxity data as a function of at least one radiographic image of a patient prior to a surgical procedure.

[0078] According to another aspect, the surgical assistance system may have computer-readable program instructions executable by the processing unit for: obtaining at least one radiographic image of a joint of a patient prior to a surgical procedure, for instance with the at least one radiographic image being obtained pre-operatively, or before any resection of soft tissue associated with a surgical procedure of the joint; image processing the at least one radiographic image of the patient using a parametrized machine learning module; and outputting joint laxity data for the joint of the patient to an operator of the surgical procedure. The surgical assistance system may perform these steps before commencement of the resection of soft tissue associated with a surgical procedure of the joint, as an option. The computer-readable program instructions may be executable by the processing unit for: outputting implant parameter recommendations associated with the joint laxity data; outputting bone cut parameters associated with the implant parameters; outputting control data to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters; driving a robot arm of the robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters; tracking the robot arm relative to at least one bone of the joint to position an end effector of the robot arm relative to the at least one bone as a function of the bone cut parameters; performing image processing of radiographic images of past surgical procedures for training; obtaining control data from computer-assisted surgery controllers used during the past surgical procedures, and wherein the machine learning module is trained using said control data; obtaining patient data associated with the radiographic images of the past surgical procedures, the patient data including one or more of age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, and wherein the machine learning module is trained using said patient data; obtaining tool data from computer-assisted surgery controllers used during the part surgical procedures, and wherein the machine learning module is trained using the tool data; obtaining post-operative assessment data associated with the past surgical procedures, and wherein the machine learning module is trained using said post-operative assessment data; obtaining post-operative assessment data includes receiving quantitative post-operative assessment data; and / orobtaining post-operative assessment data includes receiving qualitative post-operative assessment data.

[0079] The surgical assistance system may also be part of a system with a robotic computer-assisted surgery controller configured to be operated based on bone cut parameters associated implant parameters output by the surgical assistance system. The robotic computer-assisted surgery controller includes a robot arm configured to be driven based on the bone cut parameters associated with the implant parameters. A tracking system may be configured to track the robot arm relative to at least one bone of the joint to position an end effector of the robot arm relative to the at least one bone as a function of the bone cut parameters

Claims

CLAIMS:

1. A surgical assistance system comprising: a processing unit; and a non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining at least one radiographic image of a joint of a patient prior to a surgical procedure; image processing the at least one radiographic image of the patient using a parametrized machine learning module; and outputting joint laxity data for the joint of the patient to an operator of the surgical procedure.

2. The surgical assistance system of claim 1 , further including outputting implant parameter recommendations associated with the joint laxity data.

3. The surgical assistance system of claim 2, further including outputting bone cut parameters associated with the implant parameters.

4. The surgical assistance system of claim 2, further including outputting control data to operate a robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

5. The surgical assistance system of claim 2, further including driving a robot arm of the robotic computer-assisted surgery controller based on bone cut parameters associated with the implant parameters.

6. The surgical assistance system of claim 5, wherein driving the robot arm of the robotic computer-assisted surgery controller includes tracking the robot arm relative to at least one bone of the joint to position an end effector of the robot arm relative to the at least one bone as a function of the bone cut parameters.

7. The surgical assistance system of any one of claims 1 to 6, wherein the machine learning module is trained by performing image processing of radiographic images of past surgical procedures.

8. The surgical assistance system of claim 7, further including obtaining control data from computer-assisted surgery controllers used during the past surgical procedures, and wherein the machine learning module is trained using said control data.

9. The surgical assistance system of any one of claims 7 and 8, further including obtaining patient data associated with the radiographic images of the past surgical procedures, the patient data including one or more of age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, and wherein the machine learning module is trained using said patient data.

10. The surgical assistance system of any one of claims 7 to 9, further including obtaining tool data from computer-assisted surgery controllers used during the part surgical procedures, and wherein the machine learning module is trained using the tool data.

11. The surgical assistance system of any one of claims 7 to 10, further including obtaining post-operative assessment data associated with the past surgical procedures, and wherein the machine learning module is trained using said post-operative assessment data.

12. The surgical assistance system of claim 11 , wherein obtaining post-operative assessment data includes receiving quantitative post-operative assessment data.

13. The surgical assistance system of claim 11 or claim 12, wherein obtaining postoperative assessment data includes receiving qualitative post-operative assessment data.

14. A surgical assistance system comprising: a processing unit; anda non-transitory computer-readable memory communicatively coupled to the processing unit and comprising computer-readable program instructions executable by the processing unit for: obtaining radiographic images of joints of patients before surgical procedures to at least one bone of the joints; obtaining joint laxity data associated with the radiographic images of the joints of the patients during or pursuant to the surgical procedures; training a machine learning module using the radiographic images and the implant data; and outputting the machine learning module parametrized to output joint laxity data as a function of at least one radiographic image of a patient prior to a surgical procedure.

15. The surgical assistance system of claim 14, wherein the machine learning module is trained by performing image processing of the radiographic images of the surgical procedures.

16. The surgical assistance system of any one of claims 14 to 15, further including obtaining control data from computer-assisted surgery controllers used during the surgical procedures, and wherein the machine learning module is trained using said control data.

17. The surgical assistance system of any one of claims 14 to 16, further including obtaining patient data associated with the radiographic images, the patient data including one or more of age, gender, race, ethnicity, genetics, height, weight, body mass index, congenital conditions, pathologies, medical history, and wherein the machine learning module is trained using said patient data.

18. The surgical assistance system of any one of claims 14 to 17, further including obtaining tool data from computer-assisted surgery controllers used during the surgical procedures, and wherein the machine learning module is trained using the tool data.

19. The surgical assistance system of any one of claims 14 to 18, further including obtaining post-operative assessment data associated with the surgical procedures, andwherein the machine learning module is trained using said post-operative assessment data.

20. The surgical assistance system of claim 19, wherein obtaining post-operative assessment data includes receiving quantitative post-operative assessment data.

21. The surgical assistance system of claim 19 or claim 20, wherein obtaining postoperative assessment data includes receiving qualitative post-operative assessment data.

22. The surgical assistance system of any one of claims 14 to 21 , further including obtaining implant data associated with the radiographic images of the joints of the patients pursuant to the surgical procedures.

23. A system comprising: the surgical assistance system according to any one of claims 1 to 13, a robotic computer-assisted surgery controller configured to be operated based on bone cut parameters associated implant parameters output by the surgical assistance system.

24. The system of claim 23, wherein the robotic computer-assisted surgery controller includes a robot arm configured to be driven based on the bone cut parameters associated with the implant parameters.

25. The system of claim 24, including a tracking system configured to track the robot arm relative to at least one bone of the joint to position an end effector of the robot arm relative to the at least one bone as a function of the bone cut parameters.

Images