Predicting movement of anatomical elements

Abstract

A method for predicting movement of an anatomical element. The method includes receiving a surgical plan including a predetermined trajectory of a robot and a predetermined position of a second anatomical element, receiving, from a tracking system, data representing movement of a first anatomical element, and determining, using a machine learning model, a predicted movement of the second anatomical element using the data including the movement of the first anatomical element. The method further includes determining, based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element, a predicted position of the second anatomical element, and when the predetermined position of the second anatomical element is different from the predicted position of the second anatomical element, updating the predetermined trajectory, the predetermined position of the second anatomical element, or both, based on the predicted position of the second anatomical element.

Description

Background Technology 1. Technical Field

[0002] This invention relates to the field of autonomous or semi-autonomous surgery, and more specifically to predicting the movement of anatomical elements in relation to surgery.

[0003] 2. Discussion in related fields

[0004] Typically, a surgical plan is received as part of performing autonomous or semi-autonomous surgery. A surgical plan may include an ordered list of actions to be performed on one or more anatomical elements and the locations associated with those one or more anatomical elements. Summary of the Invention

[0005] The systems and methods described herein relate to tracking anatomical elements during autonomous or semi-autonomous surgical procedures performed using electronic computing devices. One of the greatest challenges in performing navigation-assisted or robot-assisted spinal surgery is maintaining system registration with the anatomical structures. For example, performing a movement on a first anatomical element (e.g., a first vertebra) during the surgical procedure can cause movement on a second anatomical element (e.g., a second vertebra). If the surgery requires movements on the second anatomical element, this movement will create a discrepancy between the position of the second anatomical element recorded in the surgical plan and its actual position.

[0006] To overcome this challenge, existing systems and methods utilize tracking systems (e.g., optical tracking systems, electromagnetic tracking systems, inertial measurement unit tracking systems, etc.) that include an anatomical structure tracker (e.g., an optical instrument) placed on or near a first anatomical element. In some cases, the tracking system is configured to measure the movement of the first anatomical element. For example, the anatomical structure tracker may be placed on a first vertebra or sacrum. The tracking system can transmit data representing the movement of the first anatomical element to an electronic processor. Based on the movement of the first anatomical element, the electronic processor can determine the position of the first anatomical element on or near the anatomical structure tracker. It should be understood that, as described herein, the position of an anatomical element refers to its spatial location (e.g., relative to the x, y, and z coordinate planes) and its spatial orientation (e.g., pitch, roll, and yaw of the anatomical element). The electronic processor can also determine the position of a second anatomical element (e.g., a second vertebra) on which the tracker is not placed, based on the assumption that the movement of a second anatomical element matches the movement of a first anatomical element.

[0007] However, since the movement of the first anatomical element may differ from the movement of the second anatomical element, determining the position of the second anatomical element based on the movement of the first anatomical element may lead to an inaccurate determination of the position of the second anatomical element.

[0008] Therefore, the embodiments described herein provide systems and methods for predicting the movement of a second anatomical element using a machine learning model. The machine learning model can predict the movement of the second anatomical element based on patient data, historical surgical data, surgical plans, the movement of a first anatomical element, combinations thereof, etc. Using the predicted movement of the second anatomical element generated by the machine learning model to determine the position of the second anatomical element, rather than relying on the assumption that the movement of the second anatomical element is the same as the movement of the first anatomical element, allows for a more accurate determination of the position of the second anatomical element. Therefore, even when the electronic processor does not receive data on the movement of each anatomical element of the patient in relation to surgery (e.g., each vertebra of the patient's spine), the systems and methods described herein allow for accurate autonomous and semi-autonomous surgery on anatomical elements.

[0009] One embodiment provides a method for predicting the movement of an anatomical element. The method includes: receiving a surgical plan, the surgical plan including a predetermined trajectory of a robot and a predetermined position of a second anatomical element; receiving data representing the movement of a first anatomical element from a tracking system; and using the data including the movement of the first anatomical element to determine a predicted movement of the second anatomical element using a machine learning model. The method further includes: determining a predicted position of a second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element; and updating the predetermined trajectory of the robot included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both, based on the predicted position of the second anatomical element when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element.

[0010] Another implementation provides a method for training a machine learning model to generate predicted movements of anatomical elements. The method includes: receiving data representing movement of a first anatomical element from a training tracking system; receiving data representing actual movement of a second anatomical element from the training tracking system; and using the machine learning model to determine a predicted movement of the second anatomical element using the data including the movement of the first anatomical element. The method further includes: modifying the machine learning model based on the difference between the predicted movement of the second anatomical element and the actual movement of the second anatomical element when the difference is outside a predetermined range.

[0011] Another embodiment provides a system for predicting the movement of an anatomical element. The system includes a robot, a tracking system, and an electronic computing device including an electronic processor. The electronic processor is configured to: receive a surgical plan including a predetermined trajectory of the robot and a predetermined position of a second anatomical element; receive data representing the movement of the first anatomical element from the tracking system; and use the data including the movement of the first anatomical element to determine a predicted movement of the second anatomical element using a machine learning model. The electronic processor is further configured to: determine a predicted position of the second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element; and update the predetermined trajectory of the robot included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both, based on the predicted position of the second anatomical element when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element.

[0012] Another embodiment provides a system for predicting the movement of an anatomical element. The system includes a tracking system and an electronic computing device including an electronic processor. The electronic processor is configured to: receive a predetermined position of a second anatomical element; receive data representing the movement of a first anatomical element from the tracking system; and determine a predicted movement of the second anatomical element using the data including the movement of the first anatomical element via a machine learning model. The electronic processor is further configured to: determine a predicted position of the second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element; and update the predetermined position of the second anatomical element based on the predicted position of the second anatomical element when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element.

[0013] Other aspects, features, and implementations will become apparent upon consideration of the detailed description and accompanying drawings. Attached Figure Description

[0014] The same reference numerals in the accompanying drawings denote the same or similarly functional elements in the various views, and are incorporated in and form part of the specification together with the following detailed description, and are also used to illustrate various embodiments, examples, aspects and features including the claimed subject matter, and to explain the various principles and advantages of those embodiments, examples, aspects and features.

[0015] Figure 1 This is a block diagram of a system for predicting the movement of anatomical elements according to one embodiment.

[0016] Figure 2 yes Figure 1 An illustrative example of the position of the first tracker of the tracking system included in the system relative to the anatomical element.

[0017] Figure 3 It is for use Figure 1 An illustrative example of a flowchart of a method for predicting the movement of anatomical elements using a system.

[0018] Figure 4 It is for use Figure 1 Another illustrative example of a flowchart of a system for predicting the movement of anatomical elements.

[0019] Figure 5 This is a block diagram of a system for training a machine learning model to generate predicted movements of anatomical elements, according to one embodiment.

[0020] Figure 6 yes Figure 5 An illustrative example of the position of the first and second trackers relative to the anatomical element in the system.

[0021] Figure 7 It is for use Figure 5 An illustrative example of a flowchart illustrating a method for training a machine learning model to generate predicted movements of anatomical elements.

[0022] Figure 8 yes Figure 5 The machine learning models included in the system and the technologies applied to machine learning models Figure 7 An illustrative example of the method.

[0023] Those skilled in the art will understand that the elements in the accompanying drawings are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the dimensions of some elements in the drawings may be enlarged relative to other elements to aid in understanding the illustrated examples, aspects, and features.

[0024] In some instances, device and method components have been indicated by conventional symbols in the accompanying drawings where appropriate, so as to show only those specific details relevant to understanding the various embodiments, examples, aspects and features, so as not to confuse the contents of this disclosure with details that are readily apparent to those skilled in the art and have the benefit of the description herein. Detailed Implementation

[0025] For ease of description, some or all of the example systems presented herein are illustrated using a single example of each of their components. Some examples may not describe or illustrate all components of the system. Other exemplary implementations may include more or fewer of each of the illustrated components, may combine some components, or may include additional or alternative components.

[0026] It should be understood that although some of the accompanying drawings illustrate hardware and software located within a particular device, these depictions are for illustrative purposes only. In some embodiments, the illustrated components may be combined or divided into separate software, firmware, and / or hardware. For example, instead of being located within and executed by a single electronic processor, logic and processing may be distributed across multiple electronic processors. Regardless of how they are combined or divided, hardware and software components may reside on the same computing device or may be distributed among different computing devices connected by one or more networks or other suitable communication links.

[0027] Figure 1 An exemplary implementation of a system 100 for predicting the movement of anatomical elements is provided. The system includes a database 110, a robot 113, a tracking system 120, and an electronic computing device 130. The electronic computing device 130 includes an electronic processor 135 (e.g., a programmable microprocessor, microcontroller, or similar device), a memory 140 (e.g., a non-transitory computer or machine-readable storage), an input device 145, and an output device 150. The input device 145 may be, for example, a keypad, keyboard, mouse, touchscreen (e.g., as part of the output device 150), microphone, camera, universal serial bus (“USB”) port, etc. The output device 150 may be, for example, a speaker, touchscreen, liquid crystal display (“LCD”), light-emitting diode (“LED”) display, organic LED (“OLED”) display, electroluminescent display (“ELD”), etc. It should be understood that although the electronic computing device 130 is illustrated as including a single input device 145 and a single output device 150, the electronic computing device 130 may include multiple input devices and multiple output devices. Electronic processor 135 is communicatively connected to memory 140, input device 145, and output device 150. In some embodiments, electronic processor 135 cooperates with memory 140 and is configured to implement the methods described herein. In some embodiments, memory includes machine learning model 155. Machine learning model 155 may be a supervised learning model, such as a neural network, decision tree, random forest, Naive Bayes classifier, support vector machine, etc.

[0028] Database 110, robot 113, tracking system 120, and electronic computing device 130 can communicate via a wired or wireless communication network 160. The communication network may include, for example, one or more cables, a local area network (LAN), a wide area network (WAN), or a short-range wireless network (such as Bluetooth). ™Networks), combinations thereof, etc. For example, in one embodiment, the electronic computing device 130 may communicate with the database 110 via the Internet and with the robot 113 via one or more cables. In some embodiments, the electronic processor 135 may be configured to send and receive data from the database 110, the robot 113, and the tracking system 120 via a communication network 160 using one or more communication interfaces included in the electronic computing device 130. It should be understood that, although Figure 1 System 100 is illustrated as including only a single database 110, a single robot 113, and a single electronic computing device 130, but system 100 may alternatively include multiple databases, robots, and electronic computing devices. For example, system 100 may include multiple databases, and the functions described herein performed by database 110 may be partitioned among multiple databases. It should also be understood that while robot 113 and electronic computing device 130 are... Figure 1 The component is illustrated as a separate component in system 100, but the component illustrated as included in electronic computing device 130 may alternatively be included in robot 113.

[0029] In one implementation, database 110 is configured to store patient data associated with one or more patients. Patient data may include, for example, patient-associated age, patient-associated body mass index (“BMI”), patient-associated bone mineral density, patient-associated computed tomography scan, patient-associated sex, patient-associated implant history (e.g., date and placement location associated with implants such as screws, rods, retainers, prostheses, etc.), combinations thereof, etc. In some embodiments, patient data is associated with patients undergoing surgery, where the systems and methods described herein are utilized. In some embodiments, patient data may be stored in database 110 prior to surgery. In some embodiments, electronic processor 135 is configured to send a query to database 110 requesting patient data associated with a unique patient identifier, and to receive patient data associated with a unique patient identifier from database 110.

[0030] In some implementations, database 110 is configured to store one or more surgical plans. In one example, database 110 may receive surgical plans and unique plan identifiers associated with the surgical plans from an electronic computing device (e.g., electronic computing device 130 or another electronic computing device) via a communication network 160. In some implementations, electronic processor 135 is configured to send a query to database 110 requesting a surgical plan associated with a unique plan identifier, and to receive the surgical plan associated with the unique plan identifier from database 110.

[0031] In some embodiments, the tracking system 120 is an optical tracking system including a first tracker 122 and a position sensor 123 (e.g., a depth camera). In some embodiments, the first tracker 122 is an optical instrument. Although the tracking system 120 is illustrated in the figures and described throughout as an optical tracking system, in some embodiments, the tracking system may be an electromagnetic tracking system, an inertial measurement unit tracking system, etc. In some embodiments, the position sensor 123 may be included in the robot 113. In some embodiments, the first tracker 122 is positioned on or near a first anatomical element and is configured to send data about the movement of the first anatomical element to an electronic processor 135. In one example, the first tracker 122 is placed on the L1 vertebra of the spine, and the position sensor 123 uses the first tracker 122 to collect data about changes in the position of the L1 vertebra as measured in x, y, z, or Cartesian coordinates, and changes in the orientation of the L1 vertebra as measured in degrees. Figure 2 In another example shown, the first tracker 122 is placed on the sacrum 200 of the spine 205.

[0032] Robot 113 can be any surgical robot or surgical robot system. Robot 113 is or includes, for example, a Mazor XTM Stealth Robot Guidance System. Robot 113 includes robotic instruments 115, and in some embodiments, may include more than one robotic instrument. If robot 113 includes multiple robotic instruments, each robotic instrument may be positioned independently of each other robotic instrument included in robot 113. Robotic instruments may be controlled in a single shared coordinate space or in separate coordinate spaces. Robotic instrument 115 may be a robotic arm. Robot 113 may include one or more electronic processors configured to control the movement of robotic instrument 115. For example, robot 113 may include one or more electronic processors configured to control robotic instrument 115 to manipulate surgical instruments to perform or assist surgical tasks. In another example, robot 113 may include one or more electronic processors configured to control robotic instrument 115 to hold and / or manipulate anatomical elements during or in connection with surgical procedures. In some embodiments, the robotic device 115 includes one or more sensors that enable the one or more electronic processors included in the robot 113 to determine the precise position (location and orientation) of the robotic device (and any object or anatomical element held or fixed to the robotic device), the force applied by the robotic device 115, the torque applied by the robotic device 115, and combinations thereof.

[0033] The robot 113, together with the robotic instrument 115, may have, for example, one, two, three, four, five, six, seven, or more degrees of freedom. Furthermore, the robotic instrument 115 can be positioned at any location, plane, and / or focal point. Location includes position and orientation. Therefore, surgical instruments or other objects held by the robotic instrument 115 can be precisely positioned according to the surgical plan.

[0034] In some embodiments, the electronic processor included in robot 113 sends historical surgical data to electronic processor 135. The historical surgical data may include anatomical elements to which torque is applied and the amount of torque applied, anatomical elements to which force is applied and the amount of force applied, or both. For example, when robotic instrument 115 applies torque to the L1 vertebra, the electronic processor included in robot 113 sends data to electronic processor 135 indicating that torque is applied to the L1 vertebra and the amount of torque applied to the L1 vertebra.

[0035] Figure 3 This is a flowchart of a method 300 for predicting the movement of anatomical elements using system 100. In some embodiments, method 300 begins at step 305, where electronic processor 135 receives a surgical plan that includes a predetermined trajectory for a robot (e.g., robot 113) and a second anatomical element (e.g., Figure 2 The predetermined location of the second vertebra 210 in the spine 205 shown. In some embodiments, the electronic processor 135 receives the surgical plan from the database 110, as described above. In other embodiments, the surgical plan may be received via an input device 145. In some embodiments, the electronic processor 135 sends one or more commands to the robot 113 to move the robotic instrument 115 along a predetermined trajectory. In some embodiments, the predetermined trajectory includes a series of ordered actions to be performed by the robotic instrument 115. When the robot 113 includes multiple robotic instruments, each action included in the predetermined trajectory may include an indication of which of the multiple robotic instruments is to perform the action.

[0036] At step 310, the electronic processor 135 receives data from a tracking system (e.g., tracking system 120) representing movement of a first anatomical element (e.g., a first vertebra (or sacrum) 200 in the spine 205). In some embodiments, the electronic processor 135 uses the data received from the tracking system 120 to determine the movement of the first anatomical element. In some embodiments, the movement of the first anatomical element may occur due to patient movement, movement of the bed or operating table in which the patient is located, or movement of the patient due to actions of a human operator (such as a surgeon). In other embodiments, the movement of the first anatomical element is caused when the robotic instrument 115 performs an action. For example, the movement of the first anatomical element is caused when the robotic instrument 115 performs an action included in the surgical plan, which involves applying force or torque to a portion of the spine 205. In some embodiments, when the robotic instrument 115 performs an action, data about the action is collected by one or more sensors included in the robotic instrument 115, and this data is included in historical surgical data.

[0037] At step 315, the electronic processor 135 uses a machine learning model (e.g., machine learning model 155) to determine the predicted movement of the second anatomical element using data including the movement of the first anatomical element. In some embodiments, the data used or analyzed by the machine learning model 155 also includes patient-associated computed tomography scans and historical surgical data. In other embodiments, the data used or analyzed by the machine learning model 155 also includes patient data, historical surgical data, surgical plans, combinations thereof, etc.

[0038] At step 320, the electronic processor 135 determines the predicted position of the second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element. At step 325, when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element, the electronic processor 135 updates the predetermined trajectory of the robot 113 included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both, based on the predicted position of the second anatomical element. In some embodiments, steps 310 to 325 are repeated until each action included in the surgical plan is completed.

[0039] In some embodiments, the electronic processor 135 is configured to output an updated trajectory of the robot 113, an updated predetermined position of the second anatomical element, or both, via an output device (e.g., output device 150). For example, the electronic processor 135 may display the updated trajectory of the robot 113 via a display device. The electronic processor 135 may be configured to receive inputs confirming or rejecting the updated trajectory of the robot 113, the updated predetermined position of the second anatomical element, or both, via an input device (e.g., input device 145). When the input received by the electronic processor 135 confirms the updated trajectory of the robot 113, the updated predetermined position of the second anatomical element, or both, the electronic processor 135 may send a signal to move the robot 113 based on the updated trajectory of the robot 113, the updated predetermined position of the second anatomical element, or both.

[0040] When the input received by the electronic processor 135 rejects the updated trajectory of the robot 113, the updated predetermined position of the second anatomical element, or both, the electronic processor 135 may retrain the machine learning model 155. In some embodiments, when the input received by the electronic processor 135 rejects the updated trajectory of the robot 113, the updated predetermined position of the second anatomical element, or both, the electronic processor 135 may be configured to request manual execution of the surgical procedure. In some embodiments, when the input received by the electronic processor 135 rejects the updated trajectory of the robot 113, the updated predetermined position of the second anatomical element, or both, the electronic processor 135 may be configured to request, via input device 145, to receive, user-determined updated trajectory of the robot 113, user-determined updated predetermined position of the second anatomical element, or both, before executing the next action included in the surgical plan or predetermined trajectory.

[0041] Figure 4 It is for use Figure 1An illustrative example of a flowchart of a method 400 for predicting the movement of anatomical elements using a system. In some embodiments, method 400 begins at steps 405 and 410. In step 405, patient data is collected and stored, for example, in database 110, before or preoperatively, the surgery is performed. In step 410, a surgical plan is created and may be saved or stored in database 110 before or preoperatively, the surgery is performed. Steps 415 through 435 are performed during or during the surgery. At step 415, an optical instrument or positional anatomical tracker (e.g., first tracker 122) is positioned or placed by, for example, a medical professional, to contact a first anatomical element (e.g., a first vertebra or sacrum). At step 420, robotic instrument 115 is registered to the patient's anatomy. In other words, the coordinate system of robotic instrument 115 is aligned with the coordinate system of tracking system 120. If robot 113 comprises multiple robotic instruments, the coordinate system of each robotic instrument is aligned with the coordinate system of tracking system 120. At step 425, anatomical tracking is initialized. In other words, the position and orientation of the first tracker 122 are determined as the reference position.

[0042] At step 430, surgical planning begins. At step 430, as the robot 113 interacts with the patient's anatomy, the electronic processor 135 receives historical surgical data (including torques and forces applied to the patient's anatomical elements by the robotic instrument 115). At step 435, the electronic processor 135 calculates or determines the movement of a second anatomical element (e.g., a second vertebra). The electronic processor 135 is configured to determine the movement of the second anatomical element by executing a machine learning model 155. At step 440, the electronic processor 135 determines the movement of a first anatomical element based on movement data received from the tracking system 120. It should be understood that the movement of the first anatomical element is not determined by the machine learning model 155, but is input into the machine learning model. At step 445, the electronic processor 135 executes the machine learning model 155 to predict the movement of the second anatomical element by using the movement of the first anatomical element, patient data, historical surgical data, surgical plan, and combinations thereof as input.

[0043] Figure 5This is a block diagram of an exemplary example of a system 500 for training a machine learning model to generate predicted movements of a second anatomical element. System 500 may include the same or similar components and connections as system 100. Components included in system 500 may be configured to perform the same or similar functions as components included in system 100. However, system 500 includes a training tracking system 505 that includes a second tracker 515 in addition to the first tracker 122. In one embodiment, the second tracker 515 is a segmented tracker that includes an optical sphere 520 and an inertial sensor 525 (such as a gyroscope or accelerometer), with a position sensor 123 configured to capture the movement of the optical sphere. In some embodiments, the first tracker 122 may be physically larger than the second tracker 515. A description of using a segmented tracker to determine the movement and position or pose of an anatomical element can be found in U.S. Patent Application No. 2023 / 0270503, the contents of which are incorporated herein by reference. It should be understood that the training tracking system 505 described herein is merely an illustrative example, and other systems and methods for determining the movement of the second anatomical element may be used when performing the method 700 described below.

[0044] The second tracker 515 can be placed on or near the anatomical element. Although Figure 5 The training tracking system 505 shown includes only the first tracker 122 and the second tracker 515, but it should be understood that the training tracking system 505 may include a greater number of trackers.

[0045] Figure 6 This is an illustrative example of the position of the first tracker 122 and the second tracker 515 relative to the patient's anatomical elements. Figure 6 In the configuration, a first tracker 122 is placed on a first vertebra 610 (first anatomical element) included in the spine 605, and a second tracker 515 is placed on a second vertebra 615 (second anatomical element). A third tracker 620 is placed on a third vertebra 625, a fourth tracker 630 is placed on a fourth vertebra 635, and a fifth tracker 640 is placed on a fifth vertebra 645.

[0046] Figure 7 It is for use Figure 5An illustrative example of a flowchart of a method 700 for training a machine learning model (e.g., machine learning model 155) to generate predicted movements of anatomical elements. In some embodiments, method 700 begins at step 705, where electronic processor 135 receives data from a tracking system (e.g., tracking system 120) representing movement of a first anatomical element (e.g., first vertebra 610). At step 710, electronic processor 135 receives data from a training tracking system (e.g., training tracking system 505 including a second tracker 515) representing actual movement of a second anatomical element (e.g., second vertebra 615).

[0047] At step 715, the electronic processor 135 uses the machine learning model 155 to determine the predicted movement of the second anatomical element using data including the movement of the first anatomical element. In some embodiments, the data used or analyzed by the machine learning model 155 may also include patient-associated computed tomography scans and historical surgical data. In other embodiments, the data used or analyzed by the machine learning model 155 may also include patient data, historical surgical data, surgical plans, combinations thereof, etc. In some embodiments, the predicted movement of the second anatomical element, the actual movement of the second anatomical element, and the movement of the first anatomical element are represented as a four-by-four matrix depicting spatial rotation (change of orientation) and translation (change of position).

[0048] In some embodiments, the electronic processor 135 is configured to determine the difference between the predicted movement of the second anatomical element and the actual movement of the second anatomical element. For example, the electronic processor 135 may be configured to determine the rotational difference between the predicted movement and the actual movement of the second anatomical element, and the translational difference between the predicted movement and the actual movement of the second anatomical element.

[0049] At step 720, when the difference between the predicted movement and the actual movement of the second anatomical element is outside a predetermined range, the electronic processor 135 modifies the machine learning model 155 based on the difference. In one embodiment, the predetermined range may be -0.5 mm to 0.5 mm, -0.5 degrees to 0.5 degrees, or both. When the translation difference is greater than 0.5 mm or less than -0.5 mm, the rotation difference is greater than 0.5 degrees or less than 0.5 degrees, or both, the electronic processor 135 modifies the machine learning model 155 based on the difference. For example, when the machine learning model 155 is a machine learning model that performs weighted calculations (e.g., a deep neural network (“DNN”)), the electronic processor 135 can modify the machine learning model 155 by adjusting one or more weights in the machine learning model 155 based on the difference.

[0050] In some implementations, the electronic processor 135 may be configured to modify the machine learning model 155 based on the difference using a loss function. For example, the loss function could be... ,in This is the first constant value, which weights the translation difference and thus increases or decreases its influence on changes to the machine learning model 155. It is a function representing the translation determined using the training tracking system 505. It is a function representing the translation estimate determined by machine learning model 155. This is the second constant value, which weights the rotation difference and thus increases or decreases the influence of the rotation difference on changes to the machine learning model 155. It is a function representing the rotation determined using the training tracking system 505, and It is a function representing the rotation estimate determined by the machine learning model 155.

[0051] It should be understood that the machine learning model 155 can be trained to determine the movement of multiple anatomical elements (e.g., the third vertebra 625, the fourth vertebra 635, and the fifth vertebra 645) when the first tracker 122 is placed on the first anatomical element (e.g., the first vertebra 610). Additionally, the machine learning model 155 can be trained to determine the movement of the multiple anatomical elements when the tracking system 120 is placed on any one of the multiple anatomical elements.

[0052] Figure 8 This is an illustrative example of a machine learning model (e.g., machine learning model 155) and a method 700 for training machine learning model 155. Figure 8 In this model, the input of machine learning model 155 is included in box 800, and the output of machine learning model 155 is represented by box 805 and ellipse 810. Figure 8 In this context, machine learning model 155 is illustrated as a neural network comprising an input layer, hidden layers, and an output layer. The actual location of the second anatomical element is... Figure 8 The values ​​are illustrated in boxes 815 and ellipse 820. Box 825 represents the calculation of the loss function. Box 830 illustrates the modification of machine learning model 155.

[0053] It should be understood that the systems and methods described herein can be applied to non-robotic surgeries to assist human surgeons. For example, in some embodiments, a patient's CT scan can be displayed during surgery via a display device (e.g., output device 150). In some embodiments, an illustration of the surgeon's tools can be displayed over the patient's CT scan as the tools interact with the patient's anatomy. In some embodiments, a portion of the displayed CT scan can be updated based on the position of the surgeon's tools. By using a machine learning model 155 to predict the movement of a second anatomical element, an electronic processor 135 can display the second anatomical element at an updated position relative to the surgeon's tools in the CT scan via a display device (e.g., output device 150). In other words, a portion of the displayed CT scan, the orientation of the CT scan relative to the illustration of the surgical tools, or both can be updated based on the updated predetermined position of the second anatomical element.

[0054] Although the examples provided in this article describe anatomical elements as vertebrae in the spine, it should be understood that anatomical elements can be bones, tissues, organs, veins, parts thereof, or any other part of the human anatomy.

[0055] Regarding the processes, systems, methods, and inspirations described herein, it should be understood that although the steps of such processes are described as occurring according to an ordered sequence, such processes can be practiced using the described steps performed in an order other than that described herein. It should also be understood that some steps may be performed simultaneously, other steps may be added, or some steps described herein may be omitted. In other words, the description of processes herein is provided for the purpose of illustrating certain embodiments and should in no way be construed as limiting the claims.

[0056] Therefore, it should be understood that the above description is intended to be illustrative rather than restrictive. Many embodiments and applications beyond the examples provided will become apparent upon reading the above description. The scope should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of their equivalents. Future developments are anticipated and intended in the art discussed herein, and the disclosed systems and methods will be incorporated into such future embodiments. In conclusion, it should be understood that modifications and variations are possible in this application.

[0057] Unless expressly indicated otherwise herein, all terms used in the claims are intended to be given their broadest reasonable structure and their common meaning as understood by one skilled in the art described herein. In particular, the use of singular articles such as “a,” “the,” “the,” etc., should be understood to enumerate one or more of the elements shown, unless expressly limited to the contrary is set forth in the claims.

[0058] Unless otherwise explicitly stated, every value and range should be interpreted as an approximation, as if the words “approximately” or “about” preceded the value or range.

[0059] In this document, the reference to “an embodiment” or “implementation” means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this disclosure. The appearance of the phrase “in an embodiment” in various places in the specification does not necessarily refer to the same embodiment, nor is it necessarily a single or alternative embodiment that is mutually exclusive with other embodiments. The same applies to the term “implementation.”

[0060] Unless otherwise specified herein, the use of ordinal adjectives such as “first,” “second,” “third,” etc., to refer to one of a plurality of similar objects merely indicates that different instances of such similar objects are being referred to, and does not imply that the similar objects referred to in this way must have a corresponding order or sequence in time, space, ranking, or any other way.

[0061] Unless otherwise specified herein, the conjunction “if” may also be interpreted, in addition to its explicit meaning, as “when”, “in response to determination”, or “in response to detection”, depending on the specific context. For example, the phrase “if determination” or “if detection [the condition]” may be interpreted as “when determination” or “in response to determination”, “in response to detection [the condition or event]”, or “in response to detection [the condition or event]”.

[0062] Furthermore, for the purposes of this specification, the terms “coupled” and “connected” refer to any manner known in the art or subsequently developed, in which energy is allowed to be transferred between two or more elements, and the insertion of one or more additional elements is considered, but not required. Conversely, the terms “direct coupling,” “direct connection,” etc., imply the absence of such additional elements. The same type of distinction applies to the use of the terms “attachment” and “direct attachment” when applied to the description of physical structures. For example, such a “direct attachment” of two corresponding parts in such a physical structure may be implemented using a relatively thin layer of adhesive or other suitable bonding agent.

[0063] The described embodiments should be considered illustrative rather than limiting in all respects. In particular, the scope of this disclosure is indicated by the appended claims, and not by the description and drawings herein. All modifications falling within the meaning and scope of the equivalents of the claims are to be covered by the claims.

[0064] The functionality of the various elements shown in the accompanying figures, including any functional blocks labeled “processor” and / or “controller,” can be provided using dedicated hardware and hardware capable of executing software in association with appropriate software. When provided by a processor, the functionality can be provided by a single dedicated processor, a single shared processor, or multiple separate processors, some of which may be shared. Furthermore, the explicit use of the terms “processor” or “controller” should not be construed as referring only to hardware capable of executing software, and may implicitly include, but is not limited to, digital signal processor (DSP) hardware, network processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile memory. Other conventional and / or custom hardware may also be included. Similarly, any switches shown in the accompanying figures are conceptual only. Their functionality can be implemented through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, with the specific technology selectable by the implementer, as understood more specifically from the context.

[0065] As used in this application, the term "circuit" may refer to one or more or all of the following: (a) a hardware-only circuit implementation (such as an implementation in analog and / or digital circuits only); (b) a combination of hardware circuits and software, such as (if applicable): (i) a combination of analog and / or digital hardware circuits with software / firmware, and (ii) any part of a hardware processor with software (including digital signal processors), software, and memory, which work together to enable a device (such as a mobile phone or server) to perform various functions; and (c) hardware circuits and / or processors, such as microprocessors or parts thereof, which require software (e.g., firmware) to operate, but may be absent when the software is not required to operate. "The definition of 'circuit' applies to all uses of the term in this application, including in any claim. As a further example, as used in this application, the term 'circuit' also covers implementations of hardware circuitry or processors (or processors) or a portion thereof and their accompanying software and / or firmware. The term 'circuit' also covers (e.g., and if applicable to a particular claim element) baseband integrated circuits or processor integrated circuits for mobile devices or similar integrated circuits in servers, cellular network devices, or other computing or networking devices."

[0066] Those skilled in the art will understand that any block diagram herein represents a conceptual view of an exemplary circuit embodying the principles of this disclosure. Similarly, it should be understood that any flowchart, flow diagram, state transition diagram, pseudocode, etc., represents various processes that can be represented substantially in a computer-readable medium and executed by a computer or processor, whether or not such a computer or processor is explicitly shown.

[0067] The following paragraphs provide various embodiments and alternatives disclosed herein.

[0068] Example 1. A method for predicting the movement of an anatomical element, the method comprising: receiving a surgical plan, the surgical plan including a predetermined trajectory of a robot and a predetermined position of a second anatomical element; receiving data representing the movement of a first anatomical element from a tracking system; using the data including the movement of the first anatomical element to determine a predicted movement of the second anatomical element using a machine learning model; determining a predicted position of the second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element; and updating the predetermined trajectory of the robot included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both, based on the predicted position of the second anatomical element when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element.

[0069] Example 2. The method according to Example 1, wherein the data further includes patient-associated computed tomography scans and historical surgical data.

[0070] Example 3. The method according to Example 1, wherein the data further includes patient data, historical surgical data, and the surgical plan.

[0071] Example 4. The method according to Example 1, the method further includes: moving the robot based on the updated trajectory.

[0072] Example 5. The method according to Example 1, the method further comprising: outputting the updated trajectory of the robot, the updated predetermined position of the second anatomical element, or both, via an output device; receiving, via an input device, an input confirming or rejecting the updated trajectory of the robot, the updated predetermined position of the second anatomical element, or both; when the received input confirms the updated trajectory of the robot, the updated predetermined position of the second anatomical element, or both, moving the robot based on the updated trajectory of the robot, the updated predetermined position of the second anatomical element, or both; and when the received input rejects the updated trajectory of the robot, the updated predetermined position of the second anatomical element, or both, retraining the machine learning model.

[0073] Example 6. The method according to Example 3, wherein the patient data includes at least one selected from the group consisting of: patient-associated age, BMI, bone mineral density, computed tomography scan, implantation history, and sex.

[0074] Example 7. The method according to Example 2 or 3, wherein the historical surgical data includes anatomical elements to which torque is applied and the amount of torque applied, anatomical elements to which force is applied and the amount of force applied, or both.

[0075] Example 8. The method according to Example 1, the method further includes using the robot to perform an action, wherein the action causes the movement of the first anatomical element, and data about the action is included in the historical surgical data.

[0076] Example 9. The method according to Example 1, wherein the first anatomical element is a first vertebra or a sacrum in the spine, and the second anatomical element is a second vertebra in the spine.

[0077] Example 10. A method for training a machine learning model to generate predicted movement of an anatomical element, the method comprising: receiving data representing movement of a first anatomical element from a training tracking system; receiving data representing actual movement of a second anatomical element from the training tracking system; using the data including the movement of the first anatomical element to determine a predicted movement of the second anatomical element using a machine learning model; and modifying the machine learning model based on the difference when the difference between the predicted movement of the second anatomical element and the actual movement of the second anatomical element is outside a predetermined range.

[0078] Example 11. The method according to Example 10, wherein the data further includes patient-associated computed tomography scans and historical surgical data.

[0079] Example 12. The method according to Example 10, wherein the data further includes patient data, historical surgical data, and surgical plans.

[0080] Example 13. According to the method of Example 10, wherein the predicted movement of the second anatomical element, the actual movement of the second anatomical element, and the movement of the first anatomical element are represented as a four-by-four matrix depicting spatial rotation and translation.

[0081] Example 14. The method according to Example 10, wherein changing the machine learning model based on the difference includes changing the machine learning model based on a loss function, wherein the loss function is , It is the first constant value. It is a function representing the translation determined using the training tracking system. It is a function representing the translation estimate determined by the machine learning model. It is the second constant value. It is a function representing the rotation determined using the training tracking system, and It is a function representing the rotation estimate determined by the machine learning model.

[0082] Example 15. The method according to Example 10, wherein the machine learning model performs a weighted calculation, and changing the machine learning model includes adjusting the weights in the machine learning model based on the differences.

[0083] Example 16. The method according to Example 10, wherein the training tracking system includes a first tracker, a second tracker, and a position sensor, the first tracker being associated with a first anatomical element, the second tracker being associated with a second anatomical element, and the position sensor being configured to capture the movement of the first tracker and the second tracker.

[0084] Example 17. A system for predicting the movement of an anatomical element, the system comprising: a robot; a tracking system; and an electronic computing device including an electronic processor configured to: receive a surgical plan including a predetermined trajectory of the robot and a predetermined position of a second anatomical element; receive data representing the movement of a first anatomical element from the tracking system; determine a predicted movement of the second anatomical element using the data including the movement of the first anatomical element via a machine learning model; determine a predicted position of the second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element; and update the predetermined trajectory of the robot included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both, based on the predicted position of the second anatomical element when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element.

[0085] Example 18. The system according to Example 17, wherein the data further includes patient-associated computed tomography scans and historical surgical data.

[0086] Example 19. The system according to Example 17, wherein the data further includes patient data, historical surgical data, and the surgical plan.

[0087] Example 20. The system according to Example 17, wherein the electronic processor is further configured to send signals to move the robot based on the updated trajectory.

[0088] Example 21. A system for predicting the movement of an anatomical element, the system comprising: a tracking system; and an electronic computing device including an electronic processor configured to: receive a predetermined position of a second anatomical element; receive data representing the movement of a first anatomical element from the tracking system; determine a predicted movement of the second anatomical element using the data including the movement of the first anatomical element via a machine learning model; determine a predicted position of the second anatomical element based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element; and update the predetermined position of the second anatomical element based on the predicted position of the second anatomical element when the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element.

[0089] The various features and advantages of the embodiments presented herein are set forth in the appended claims.

Claims

1. A method (300) for predicting movement of anatomical elements, the method (300) comprising: Receive a surgical plan, which includes a predetermined trajectory of the robot (113) and predetermined positions of the second anatomical element; Receive data representing the movement of the first anatomical element from the tracking system (120); The predicted movement of the second anatomical element is determined using data including the movement of the first anatomical element by a machine learning model (155); Based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element, the predicted position of the second anatomical element is determined; as well as When the predetermined position of the second anatomical element is different from the predicted position of the second anatomical element, the predetermined trajectory (113) of the robot included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both are updated based on the predicted position of the second anatomical element.

2. The method (300) of claim 1, wherein the data further includes computed tomography scans and historical surgical data associated with the patient.

3. The method (300) according to claim 1, wherein the data further includes patient data, historical surgical data and the surgical plan.

4. The method (300) according to claim 1, wherein the method (300) further comprises: The robot (113) moves based on the updated trajectory.

5. The method (300) according to claim 1, wherein the method (300) further comprises: The updated trajectory of the robot (113), the updated predetermined position of the second anatomical element, or both are output via the output device (150). The input device (145) receives or rejects the updated trajectory of the robot (113), the updated predetermined position of the second anatomical element, or both. When the received input confirms the updated trajectory of the robot (113), the updated predetermined position of the second anatomical element, or both, the robot (113) is moved based on the updated trajectory of the robot (113), the updated predetermined position of the second anatomical element, or both; and The machine learning model (155) is retrained when the received input rejects the updated trajectory of the robot (113), the updated predetermined position of the second anatomical element, or both.

6. The method (300) of claim 3, wherein the patient data includes at least one selected from the group consisting of: age, BMI, bone mineral density, computed tomography scan, implantation history and sex associated with the patient.

7. The method (300) according to claim 2 or 3, wherein the historical surgical data includes anatomical elements subjected to torque and the amount of torque applied, anatomical elements subjected to force and the amount of force applied, or both.

8. The method (300) of claim 1, further comprising performing an action using the robot (113), wherein the action causes the movement of the first anatomical element, and data about the action is included in the historical surgical data.

9. The method (300) according to claim 1, wherein the first anatomical element is a first vertebra in the spine (205) or a sacrum (200) in the spine, and the second anatomical element is a second vertebra (210) in the spine (205).

10. A method (700) for training a machine learning model (155) to generate predicted movements of anatomical elements, the method (700) comprising: Receive data representing the movement of the first anatomical element from the training tracking system (505); Receive data representing the actual movement of the second anatomical element from the training tracking system (505); The predicted movement of the second anatomical element is determined using data including the movement of the first anatomical element by a machine learning model (155); as well as When the difference between the predicted movement of the second anatomical element and the actual movement of the second anatomical element is outside a predetermined range, the machine learning model (155) is modified based on the difference.

11. The method (700) of claim 10, wherein the data further includes patient-associated computed tomography scans and historical surgical data.

12. The method (700) of claim 10, wherein the data further includes patient data, historical surgical data, and surgical plans.

13. The method (700) of claim 10, wherein the training tracking system (505) includes a first tracker (122), a second tracker (515), and a position sensor (123), the first tracker being associated with a first anatomical element, the second tracker being associated with a second anatomical element, and the position sensor being configured to capture movement of the first tracker (122) and the second tracker (515).

14. A system (100) for predicting movement of anatomical elements, the system (100) comprising: Robot (113); Tracking system (120); and Electronic computing device (130), the electronic computing device including electronic processor (135), the electronic processor (135) being configured to: Receive a surgical plan, the surgical plan including a predetermined trajectory of the robot (113) and a predetermined position of a second anatomical element; Receive data representing the movement of the first anatomical element from the tracking system (120); The predicted movement of the second anatomical element is determined using data including the movement of the first anatomical element by a machine learning model (155); Based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element, the predicted position of the second anatomical element is determined; as well as When the predetermined position of the second anatomical element is different from the predicted position of the second anatomical element, the predetermined trajectory (113) of the robot included in the surgical plan, the predetermined position of the second anatomical element included in the surgical plan, or both are updated based on the predicted position of the second anatomical element.

15. A system (100) for predicting movement of anatomical elements, the system (100) comprising: Tracking system (120); and Electronic computing device (130), the electronic computing device including electronic processor (135), the electronic processor (135) being configured to: The predetermined position for receiving the second anatomical element; Receive data representing the movement of the first anatomical element from the tracking system (120); The predicted movement of the second anatomical element is determined using data including the movement of the first anatomical element by a machine learning model (155); Based on the predicted movement of the second anatomical element and the predetermined position of the second anatomical element, the predicted position of the second anatomical element is determined; as well as When the predetermined position of the second anatomical element differs from the predicted position of the second anatomical element, the predetermined position of the second anatomical element is updated based on the predicted position of the second anatomical element.

Images