Surgical plan assessment

Abstract

In some implementations, a device may obtain a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot, and wherein the surgical plan is associated with one or more anatomical elements. The device may determine, using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements. The device may generate, using configuration information associated with the robot and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure. The device may determine, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations. The device may perform, based on the feedback information, one or more actions.

Description

SURGICAL PLAN ASSESSMENTBACKGROUND

[0001] In some surgical ecosystems, the convergence of connected technology and intelligent implants has enabled a shift toward more predictable procedures and improved patient outcomes in medical interventions. Through interconnected systems, surgical devices and instruments may communicate seamlessly, offering real-time data exchange and precise control over surgical procedures. The convergence of, and connectivity between, multiple components in a connected surgical ecosystem enables precise navigation and imaging, providing surgeons with improved insight into anatomical structures and facilitating improved planning and execution of surgical steps. This connected technology may enhance surgical precision and enable a more proactive approach to patient care, allowing for early detection of complications and personalized medical interventions. The integrated surgical ecosystem enables safer, more efficient procedures while providing surgeons with improved insights into patient physiology.

[0002] In some examples, the connected surgical ecosystem may include one or more surgical robots that facilitate robotically-assisted procedures, such as spinal procedures or other procedures. Surgical robots may assist a surgeon or other medical provider in carrying out a robotically-assisted procedure, and / or may complete one or more surgical procedures autonomously. Providing controllable linked articulating members allows a surgical robot to reach areas of a patient anatomy during various medical procedures.SUMMARY

[0003] Some implementations described herein relate to a system. The system may include a robot including one or more robotic arms. The system may include one or more memories and one or more processors communicatively coupled to the one or more memories. The one or more processors may be configured to obtain a surgical plan that is associated with a robotically-assisted procedure that is associated with the robot, wherein the surgical plan indicates one or more planned operations of the one or more robotic arms for the robotically- assisted procedure, and wherein the surgical plan is associated with one or more anatomical elements. The one or more processors may be configured to determine, using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements. The one or more processors may be configured to generate, using configuration information associated with the robot and theanatomical information, a modeling of a surgical field associated with the robotically-assisted procedure. The one or more processors may be configured to determine, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations. The one or more processors may be configured to perform, based on the feedback information, one or more actions.

[0004] Some implementations described herein relate to a method. The method may include obtaining, by a device, a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot, and wherein the surgical plan is associated with one or more anatomical elements. The method may include obtaining, by the device, configuration information associated with the robot. The method may include determining, by the device and using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements. The method may include determining, by the device, feedback information for the surgical plan based on the configuration information and the anatomical information. The method may include performing, by the device and based on the feedback information, one or more actions.

[0005] Some implementations described herein relate to a non-transitory computer-readable medium that stores a set of instructions. The set of instructions, when executed by one or more processors of a device, may cause the device to obtain a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot for the robotically- assisted procedure, and wherein the surgical plan is associated with one or more anatomical elements. The set of instructions, when executed by one or more processors of the device, may cause the device to determine, using medical imaging data of a patient of the robotically- assisted procedure, anatomical information of the patient associated with the one or more anatomical elements. The set of instructions, when executed by one or more processors of the device, may cause the device to generate, using configuration information associated with the robot and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure. The set of instructions, when executed by one or more processors of the device, may cause the device to determine, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations. The set of instructions, when executed by one or more processors of the device, may cause the device to perform, based on the feedback information, one or more actions.BRIEF DESCRIPTION OF THE DRAWINGS

[0001] For a better understanding of embodiments of the invention and to show how the same may be carried into effect, reference will now be made, purely by way of example, to the accompanying drawings in which like numerals designate corresponding elements or sections throughout. In the accompanying drawings:

[0006] Figs. 1A-1C are diagrams of an example implementation associated with surgical plan assessments.

[0007] Fig. 2 is a diagram of an example environment in which systems and / or methods described herein may be implemented.

[0008] Fig. 3 is a diagram of example components of a device associated with surgical plan assessments.

[0009] Fig. 4 is a flowchart of an example process associated with surgical plan assessments.

[0010] Fig. 5 is a flowchart of an example process associated with surgical plan assessments.DETAILED DESCRIPTION

[0011] The following detailed description of example aspects refers to the accompanying drawings. The same reference numbers in different drawings may identify the same or similar elements.

[0012] Some surgical procedures may be robotically-assisted procedures. In robotically- assisted procedures, a robotic system (e.g., that includes one or more robots, where each robot includes one or more robotic arms) may be used to enhance surgical precision and dexterity. Utilizing high-resolution imaging, real-time feedback mechanisms, and / or articulated robotic arms equipped with specialized instruments, surgeons may be enabled to navigate intricate anatomy with submillimeter accuracy, performing tasks such as vertebral fusion or disc replacement, among other examples. Through the integration of haptic feedback, computer vision, and / or machine learning algorithms, a robotic system may enable minimally invasive approaches, reduced patient trauma, and / or optimized surgical outcomes through precise instrument manipulation and anatomical visualization.

[0013] However, the use of robots for robotically-assisted procedures increases the difficulty and / or complexity and may reduce the accuracy and reliability of preoperative planning and intraoperative decision making for the robotically-assisted procedures. For example, the robots used for the robotically-assisted procedures may have complex robotic kinematics. A robotic arm may have many degrees of freedom and intricate joint articulation. The complex robotickinematics increases the difficulty and / or complexity of precisely predicting the maneuverability of a robotic arm within a unique anatomical context of a patient, thereby reducing the accuracy and / or reliability of preoperative planning and intraoperative decision making. Further, the robots used for the robotically-assisted procedures may be associated with ultra-high precision robotic instruments which may increase the risk of inadvertent proximity or contact with sensitive structures of a patient, such as nerves or blood vessels (e.g., because the ultra-high precision robotic instruments may have increased sensitivity to even slight movements or changes and / or reduced margins for error resulting in minor deviations or miscalculations in position bringing the robotic instruments near or in contact with sensitive structures of the patient). The reduced accuracy and / or reliability of preoperative planning and intraoperative decision making may increase the risk of potential intraoperative complications caused by inadvertent proximity or contact with sensitive structures of the patient.

[0014] As another example, the utilization of coordinate systems for robotic control (e.g., that may not align with conventional human comprehension) in combination with the complex robotic kinematics may impede seamless understanding and / or collaboration between surgeons and robotic systems. The incompatibility between human-centric anatomical references and robot-based coordinate frames increases the complexity and / or difficulty of the interpretation of spatial relationships between a position or pose of a robot and / or a robotic arm and anatomical elements in the unique anatomical context of a patient. As a result, an accuracy and / or reliability of preoperative planning and / or intraoperative decision-making may be reduced because a surgeon may be unable to interpret the coordinate frames and / or navigation mechanism of the robot and / or robotic arm. Further, in some examples, a surgeon may have limited visibility and restricted access to a surgical site (e.g., in minimally invasive procedures), impeding an ability of the surgeon to directly observe and / or promptly react to movements or malfunctions of the robot and / or robotic arm. This increases the risk of potential errors or adverse events because the surgeon may not have real-time insight into actions or feedback of the robot and / or robotic arm.

[0015] Additionally, the challenges described above may be increased because of the inherent variability of patient anatomies. For example, the effectiveness of robotic movements or actions may be based on individualized geometric configurations and properties of an anatomy of a given patient. This variability introduces complexity in preoperative planning and control for robotically-assisted procedures because generic (or template-based) robotic trajectories or manipulations may not universally accommodate diverse anatomical structures or pathologies of different patients. As a result, a surgeon may initiate a robotically-assisted procedure that isassociated with preoperative planning that is inaccurate and / or unreliable. This may consume resources (e.g., computing resources, processing resources, and / or energy resources) associated with performing a robotically-assisted procedure that has reduced effectiveness and / or efficacy for a given patient. Additionally, this increases the risk of undesired or negative outcomes for the patient. Further, due to the challenges described above, intraoperative decision making by the surgeon may be degraded, further reducing the effectiveness and / or efficacy of the robotically-assisted procedure and / or increasing the risk of undesired or negative outcomes for the patient of the robotically-assisted procedure.

[0016] Some aspects described herein enable surgical plan assessment for a robotically- assisted procedure. In some aspects, a planning device may be configured to provide patientspecific feedback information for a surgical plan of a robotically-assisted procedure for a given patient using a data-driven approach. For example, the planning device may obtain the surgical plan for the robotically-assisted procedure. The planning device may determine, using medical imaging data of a patient and / or other information, anatomical information of the patient, such as anatomical information associated with one or more anatomical elements that are associated with the robotically-assisted procedure. The planning device may determine the feedback information based on the anatomical information of the patient and configuration information of a robot associated with the robotically-assisted procedure.

[0017] In some aspects, the planning device may generate, using the configuration information and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure. The surgical field may be an area or environment in which the robotically-assisted procedure is to be performed, including one or more devices or components, the patient, one or more anatomical elements of the patient, and / or one or more surgeons or clinicians, among other examples. The planning device may determine, using the modeling, the feedback information based on a simulation of one or more planned operations indicated by the surgical plan for the robotically-assisted procedure. For example, the planning device may convert placement information (e.g., indicating placement or locations in the surgical field) of the one or more devices or components, the patient, one or more anatomical elements of the patient, and / or one or more surgeons or clinicians, among other examples, into a coordinate system used by the robot associated with the robotically-assisted procedure. Using the one or more devices or components, the patient, one or more anatomical elements of the patient, and / or one or more surgeons or clinicians, among other examples, modeled in the coordinate system, the planning device may simulate one or more planned operations by a robot for the robotically-assisted procedure. The planning device may determine the feedbackinformation based on the simulation of the one or more planned operations. For example, the simulation may indicate whether one or more robotic arms can safely reach intended anatomical elements during the one or more planned operations, a path of the robotic arm(s) relative to one or more anatomical elements of the patient, and / or a path of the robotic arm(s) relative to other device(s) or people (e.g., surgeons or other clinicians) in the surgical field, among other examples. The planning device may perform the modeling and / or simulation using an algorithmic approach, a heuristics-based approach, and / or one or more machine learning models, among other examples.

[0018] The feedback information may be based on one or more simulations of the one or more planned operations. For example, the feedback information may include qualitative feedback (e.g., indicating whether a surgical plan is achievable and / or whether there are one or more risks for the surgical plan associated with a given patient), semi-quantitative feedback (e.g., indicating a quantitative range associated with the one or more risks for the surgical plan, such as an implement of a robotic arm coming within X millimeters of a sensitive structure of the patient during a planned operation), and / or quantitative feedback (e.g., indicating quantitative information for one or more risks for the surgical plan, such as an implement of a robotic arm being Y millimeters from a sensitive structure of the patient during a planned operation). Additionally, or alternatively, the feedback information may indicate an estimated amount of time to perform the surgical plan for the robotically-assisted procedure. Additionally, or alternatively, the feedback information may indicate an estimated risk associated with the surgical plan based on the anatomical information of the patient. Additionally, or alternatively, the feedback information may include one or more work parameters indicating an estimated amount of work to be manually performed for the surgical plan, such as an amount of work to be performed to remove a targeted section of bone, among other examples.

[0019] The planning device may perform one or more actions based on determining the feedback information. In some aspects, the planning device may provide the feedback information for display (e.g., which may include instructions for user-robot interactions and / or instructions for action(s) to be performed by the user). Additionally, or alternatively, the planning device may transmit, to a navigation system of the robot, instructions associated with an execution of the one or more planned operations based on the feedback information indicating that the one or more planned operations are achievable. For example, the one or more instructions, when executed by the robot and / or the navigation system, may cause the robot to perform the one or more planned operations for the surgical plan.

[0020] As a result, preoperative planning for robotically-assisted procedures may be improved. For example, by enabling the planning device to determine feedback information using the anatomical information of the patient and the configuration information of the robot, the planning device may model or simulate a surgical field for the robotically-assisted procedure using information that is specific to a given patient and the current configuration of the robot. This enables the planning device to model or simulate planned operations of the robot in relation to anatomical information of the patient and / or other devices or people (e.g., surgeons) in the surgical field. The planning device may convert positioning information of the anatomical information of the patient and / or other devices or people (e.g., surgeons) in the surgical field into a coordinate system used to navigate the robot and / or one or more robotic arms to enable the planning device to obtain an accurate estimation or simulation of one or more operations of the robot and / or the robotic arm(s) as indicated by the surgical plan. By enabling the planning device to accurately estimate or simulate the one or more operations of the robot and / or the robotic arm(s) (e.g., in relation to one or more anatomical elements of a given patient and / or other devices or people (e.g., surgeons) in the surgical field), the planning device may determine whether the surgical plan is achievable and / or estimate any risks associated with the surgical plan with improved accuracy. The more accurate and / or reliable feedback information for the surgical plan may reduce a likelihood that a robotically-assisted procedure is initiated in which the robotically-assisted procedure cannot be successfully performed (e.g., because of patient anatomy or other factors) and / or that will result in unintended or negative outcomes for the patient, thereby conserving resources (e.g., processing resources, computing resources, memory resources, and / or power resources) that would have otherwise been associated with performing the robotically-assisted procedure. Further, this reduces the likelihood of unintended or negative outcomes for the patient of the robotically- assisted procedure that may have otherwise occurred with less accurate and / or reliable preoperative planning.

[0021] Figs. 1A-1C are diagrams of an example implementation 100 associated with surgical plan assessments. As shown in Figs. 1A-1C, example implementation 100 includes a planning device, an image database, a robot (e.g., that includes one or more robotic arms), a navigation system, and / or a display device. These devices are described in more detail below in connection with Fig. 2 and Fig. 3.

[0022] As shown in Fig. 1A, and by reference number 105, the planning device may obtain a surgical plan for a robotically-assisted procedure. The surgical plan may include one or more steps for performing the robotically-assisted procedure and / or one or more expected thresholdsfor monitoring one or more parameters during the robotically-assisted procedure, among other examples. The robotically-assisted procedure may be associated with the robot. For example, the robotically-assisted procedure may be associated with the robot in that the robot (and / or the one or more robotic arms) are to perform one or more operations during the robotically-assisted procedure. In some examples, the robotically-assisted procedure may be a spinal procedure (e.g., a spinal alignment, installing implants, osteotomy, fusion, and / or any other spinal procedure).

[0023] For example, the surgical plan may indicate one or more planned operations of the one or more robotic arms for the robotically-assisted procedure. The planned operation may be a movement, a path, a trajectory, and / or a surgical action (e.g., a cut, an incision, a tissue or bone removal, among other examples), among other examples, to be performed by the robot (e.g., by a robotic arm and / or by an implement or surgical tool configured for a robotic arm). The planning device may obtain the surgical plan via a user input. For example, a user (e.g., a surgeon or another user) may input the surgical plan to the planning device for assessment, as described in more detail elsewhere herein.

[0024] In some aspects, the surgical plan may indicate a type or category of procedure (e.g., a type or category of the robotically-assisted procedure) to be performed. For example, the surgical plan may indicate that the robotically-assisted procedure is a spinal alignment procedure, an implant installation procedure, osteotomy procedure, a fusion procedure, and / or another type of procedure. Additionally, or alternatively, the surgical plan may indicate one or more anatomical elements associated with the robotically-assisted procedure. For example, the surgical plan may indicate which anatomical elements are to be operated on, interacted with, or are otherwise involved with the robotically-assisted procedure. For example, for a spinal procedure, the surgical plan may indicate which lumbar vertebrae (e.g., the LI vertebrae, the L2 vertebrae, the L3 vertebrae, and / or another vertebrae) are to be operated on as part of the robotically-assisted procedure.

[0025] In some aspects, the planning device may determine the one or more planned operations of the one or more robotic arms for the robotically-assisted procedure based on, or otherwise associated with, the type or category of the robotically-assisted procedure and / or the one or more anatomical elements associated with the robotically-assisted procedure, among other examples. For example, the surgical plan may be a template or generic plan that is configured or designed to accomplish the type or category of procedure and for the one or more anatomical elements.

[0026] In some aspects, the template or generic plan may be associated with a surgeon or clinician who will be performing the robotically-assisted procedure. For example, the template or generic plan may be specific to the surgeon or the clinician. In other words, the surgeon or the clinician may design or configure the surgical plan (e.g., the template or generic plan) in accordance with one or more preferences or techniques desired by the surgeon or the clinician. In such examples, the planning device may obtain the template or generic plan from a profile of the surgeon or the clinician. For example, an input to the planning device may include an identifier of the surgeon or the clinician (e.g., a name, account identifier, or other identifier). The planning device may obtain the profile for the surgeon or the clinician using the identifier. The planning device may obtain the surgical plan from, or based on, the profile for the surgeon or the clinician.

[0027] As shown by reference number 110, the planning device may obtain configuration information associated with the robot. The configuration information may indicate one or more configurations, capabilities, and / or other information for the robot and / or for the one or more robotic arms. For example, the configuration information may indicate one or more parameters that define one or more capabilities of the robot, one or more operational characteristics of the robot, and / or one or more spatial constrains of robot, among other examples. For example, the configuration information may indicate a range of motion (e.g., of one or more robotic arms), a precision of movement supported by the robot, and / or a compatibility with one or more surgical tools or implements, among other examples.

[0028] In some aspects, the configuration information may include configurations for respective robotic arms of the one or more robotic arms. For example, the configuration information may include a quantity of the one or more robotic arms, a kinematic structure of the one or more robotic arms, and / or one or more degrees of freedom of the one or more robotic arms, among other examples. In some aspects, the configuration information may indicate a spatial boundary in which the robot (and / or the one or more robotic arms) can safely operate. The spatial boundary may be referred to herein as a safety volume (e.g., indicating a volume of space in which the robot can safely operate). The safety volume may indicate an area in which the one or more robotic arms can safely move (e.g., an area that is reachable by the one or more robotic arms and that does not cause the robot and / or the one or more robotic arms to contact a human or another device). In some aspects, the spatial boundary may define or indicate a reachable workspace of the one or more robotic arms and / or any constraints that may limit a movement and / or availability of the one or more robotic arms, such as for given anatomical regions or elements.

[0029] In some aspects, the configuration information may include location information of the robot. The location information may indicate a position of the robot within the surgical field. For example, the location information may indicate a position of the robot and / or the one or more robotic arms relative to other devices, components, and / or objects in the surgical field. For example, the location information may indicate a position of the robot and / or the one or more robotic arms relative to a bed or platform where the patient is expected to be located (e.g., a bed of the patient), one or more imaging devices, the navigation system, and / or expected positions of a surgical team (e.g., one or more surgeons, doctors, nurses, clinicians, or other members of the surgical team).

[0030] The location information may be indicated via one or more coordinates and / or reference points. For example, the location information may be coordinates and / or orientations relative to one or more fixed reference points in the surgical field. This may enable the positioning device to determine a position and / or alignment of the robot, the robotic arm(s), and / or other objects within the surgical field.

[0031] As shown by reference number 115, the planning device may obtain medical imaging data of a patient. The patient may be associated with the robotically-assisted procedure. In some aspects, the planning device may obtain the medical imaging data via the image database (e.g., as shown in Fig. 1A). Additionally, or alternatively, the planning device may obtain the medical imaging data via one or more imaging devices. The medical imaging data may include X-ray image data, X-ray-based image data (e.g., obtained via a fluoroscope, a computed tomography (CT) scanner, or other X-ray machine), magnetic resonance imaging (MRI) scanner-based image data, optical coherence tomography (OCT) scanner-based image data, positron emission tomography (PET) scanner-based date, ultrasound image data, endoscope image data, microscope image data, optical camera image data, thermographic camera (e.g., an infrared camera) image data, radar system (which may include a transmitter, a receiver, a processor, and one or more antennas) image data, and / or other image data. In some aspects, the medical imaging data may include one or more images, one or more videos, a feed or stream of images or video, and / or other image data. In some aspects, the medical imaging data may include two-dimensional (2D) image data, three-dimensional (3D) image data, and / or fourdimensional (4D) (e.g., 3D + time) image data.

[0032] For example, the surgical plan or another input to the planning device may identify the patient. For example, an input to the planning device may include an identifier (e.g., a name, a patient identifier, or another identifier) of the patient. The planning device may use the identifier to obtain the medical imaging data of the patient. For example, the planning devicemay perform a lookup operation via the image database using the identifier. In some other aspects, the medical imaging data may be provided to the planning device as an input (e.g., with, or in connection with, the surgical plan). The medical imaging data may enable the planning device to provide patient-specific feedback for the surgical plan, as described in more detail elsewhere herein.

[0033] As shown in Fig. IB, and by reference number 120, the planning device may analyze the medical imaging data to obtain anatomical information of the patient. For example, the planning device may determine, using medical imaging data of a patient of the robotically- assisted procedure, anatomical information of the patient. The anatomical information may include information of one or more anatomical elements associated with the robotically-assisted procedure. In some aspects, the planning device may generate the anatomical information using the medical imaging data. In some aspects, the planning device may extract the anatomical information from the medical imaging data.

[0034] In some aspects, the planning device may using a machine learning-based technique to analyze the medical imaging data. For example, the planning device may determine, generate, and / or obtain the anatomical information using a computer vision model or another model trained or configured to identify, determine, segment, or otherwise output anatomical information using medical imaging data. For example, the planning device may perform segmentation to partition medical imaging data into distinct regions corresponding to different anatomical elements (e.g., anatomical structures or tissue types). The segmentation may be based on intensity, texture, or shape characteristics, allowing for the planning device to identify organs, bones, blood vessels, and / or other relevant anatomical elements from the medical imaging data.

[0035] In some aspects, the planning device may perform anatomical localization to identify or determine locations of one or more anatomical elements in the anatomy of the patient. For example, the planning device may align the medical imaging data with preoperative or intraoperative reference frames. The planning device may detect and / or match corresponding anatomical features or markers between different image modalities or across sequential imaging time points, enabling accurate spatial alignment and combination of multiple imaging datasets. Additionally, the planning device may extract quantitative anatomical measurements and geometric parameters from the medical imaging data. The anatomical measurements and geometric parameters may provide insights into anatomical morphology, dimensions, and / or spatial relationships of one or more anatomical elements of the patient. The planning device may perform segmentation and / or identification of landmarks included in the one or moreanatomical elements using one or more image processing operations (e.g., thresholding, active contouring, manual tracing, and / or geometric heuristics), and / or machine learning or deep learning models, among other examples.

[0036] For example, the anatomical information may include a size or volume of area occupied by the patient within the surgical field. For example, the anatomical information may include spatial orientation and / or anatomical coverage of the patient. In some aspects, the anatomical information may include spatial orientation and / or anatomical coverage of the patient relative to a surgical bed or table or another reference point. In some aspects, the anatomical information may be based on the surgical plan in that the orientation of the patient (e.g., a supine position, a prone position, a lateral position, a lithotomy position, or another position) may be based on the surgical plan. In some aspects, the spatial orientation and / or anatomical coverage of the patient may be based on, or in relation to, a region of interest for the robotically-assisted procedure. For example, the spatial orientation and / or anatomical coverage of the patient may be based on, or in relation to, a one or more vertebral levels in the spine (e.g., the T10 to L5 vertebral levels).

[0037] The anatomical information may include information for one or more anatomical elements. For example, the anatomical information may include location information of respective anatomical elements within the anatomy of the patient. The location information may indicate a position, size, and / or orientation, among other examples, of the anatomical elements. The one or more anatomical elements may include bone, tissue, vertebrae, ligaments, muscles, tendons, cartilage, joints, organs, blood vessels, and / or other anatomical elements. In some aspects, the anatomical information may include a bone density, a body mass index, a muscle mass, a body composition, an organ size and / or shape, vascular anatomy information, and / or neural anatomy information.

[0038] In some aspects, the anatomical information may include information for one or more sensitive structures of the patient. The one or more sensitive structures may be structures or elements in the anatomy of the patient that are vulnerable to damage or disruption and / or which may result in functional impairment or negative outcomes for the patient if damaged. For example, sensitive structures may have a delicate structure, a high degree of innervation, and / or critical physiological functions, among other examples. The one or more sensitive structures may include nerves, blood vessels, organs, tissue, and / or sensory receptors, among other examples. The anatomical information may include location information of the one or more sensitive structures. In some aspects, the location information of the one or more sensitivestructures relative to one or more anatomical elements that are to be interacted with or removed during the robotically-assisted procedure.

[0039] In some aspects, the planning device may determine or obtain at least a portion of the anatomical information from one or more other sources (e.g., other than the medical imaging data). For example, the planning device may obtain an input indicating at least a portion of the anatomical information, such as a body mass index, a height, a gender, and / or a weight, among other examples, of the patient. The planning device may determine at least a portion of the anatomical information based on, using, or otherwise associated with information of the patient that is input to the planning device.

[0040] As shown by reference number 125, the planning device may generate a modeling of the surgical field for the robotically-assisted procedure. The planning device may generate the modeling based on, using, or otherwise associated with the anatomical information (e.g., of the patient), the configuration information (e.g., of the robot and / or the robotic arm(s)), and / or the surgical plan.

[0041] Generating the modeling may include setting values of respective parameters for a model that is configured to model, simulate, or represent (e.g., in a digital space) a physical space (e.g., the surgical field). The one or more parameters of the model may be used to define dimensions, boundaries, obstacles, and / or other features of the surgical field. The model may be a heuristics-based model (e.g., an algorithmic approach that utilizes heuristics) that uses one or more defined or configured rules to model, simulate, and / or otherwise obtain information associated with operations performed via the one or more robotic arms in the modeled surgical field. Additionally, or alternatively, the model may include a physics-based model that is configured to simulate the behavior of objects in accordance with the laws of physics. Additionally, or alternatively, the model may include an agent-based model that is configured to model or simulate autonomous agents that follow a set of rules to interact with an environment (e.g., the surgical field) in accordance with one or more defined behaviors (e.g., where one or more agents may include a surgeon, a clinician, and / or a robotic arm). Additionally, or alternatively, the model may include a probabilistic model that is configured to use probabilistic techniques, such as stochastic techniques, to represent uncertainty and variability within the surgical field.

[0042] Additionally, or alternatively, the model may include one or more machine learning models or artificial intelligence models that are trained to simulate and / or model one or more operations of robotic arms in surgical fields. For example, the one or more machine learning models or artificial intelligence models may be trained or configured to perform trajectoryprediction (e.g., of a robotic arm). The one or more machine learning models or artificial intelligence models may include one or more deep learning models configured to represent complex spatial relationships from information, such as the anatomical information, the configuration information (e.g., of the robot), and / or the medical imaging data, among other examples.

[0043] In some aspects, the one or more parameters of the model may include parameters for respective objects within the surgical field. The objects may include the robot, the one or more robotic arms, an implement or surgical tool configured for the robot, the patient, one or more anatomical elements of the patient, one or more surgical team members (e.g., a surgeon, clinician, nurse, or other surgical team member), a surgical bed or table, and / or other objects (e.g., devices, systems, and / or people) expected to be located in or near the surgical field. The one or more parameters may include position parameters for respective objects within the surgical field. A position parameter may define a position of an object using coordinate relative to a reference point or coordinate system.

[0044] In some aspects, the coordinate system may be the coordinate system used by the robot and / or the navigation system. For example, the planning device may generate coordinate locations of one or more objects (e.g., the robot, the one or more robotic arms, an implement or surgical tool configured for the robot, the patient, one or more anatomical elements of the patient, one or more surgical team members, a surgical bed or table, and / or other objects) using the coordinate system used by the robot and / or the navigation system. For example, the planning device may determine a location of anatomical elements of the patient using the anatomical information. The planning device may convert the location of the anatomical elements into the coordinate system used by the robot and / or the navigation system. This enables the planning device to accurately model, simulate, and / or evaluate the planned operations of the robot and / or the one or more robotic arms within the context of patient-specific anatomy, as described in more detail elsewhere herein.

[0045] In some aspects, the one or more parameters may include an orientation of respective objects included in the surgical field. In some aspects, the one or more parameters may include one or more boundary constraints. For example, the one or more boundary constraints may be based on the configuration information of the robot and / or the anatomical information of the patient. The one or more boundary constraints may include a patient-specific safety volume for the robot and / or the one or more robotic arms. For example, as described elsewhere herein, the configuration information may indicate a safety volume in which the robot (and / or the one or more robotic arms) can safely operate or move. The planning device may modify the safetyvolume based on the anatomical information of the patient. For example, the area or volume in which the robot (and / or the one or more robotic arms) can safely operate or move may consider the patient-specific anatomy. For example, one or anatomical elements of the patient (e.g., that are not to be contacted, cut, or otherwise interacted with during the robotically- assisted procedure) may extend into the safety volume indicated by the configuration information.

[0046] The planning device may modify the safety volume to a modified safety volume that does not include the one or anatomical elements of the patient (e.g., that are not to be contacted, cut, or otherwise interacted with during the robotically-assisted procedure). Additionally, or alternatively, the planning device may modify the safety volume (e.g., to the modified safety volume) based on, or using, location information of other objects in the surgical field, such as a surgical table or platform, one or more members of the surgical team, and / or other objects or devices. The planning device may define a boundary constraint for the model using the modified safety volume.

[0047] In some aspects, the one or more parameters may be based on the surgical plan. For example, the one or more parameters may define or configure the one or more planned operations of the robot and / or the one or more robotic arms. For example, the surgical plan may indicate one or more steps, surgical actions, movements, and / or other operations of the robot and / or the one or more robotic arms for the robotically-assisted procedure. For example, the one or more parameters may include operation parameters for respective planned operations of the one or more planned operations indicated by the surgical plan. An operation parameter may indicate, define, or configure, a path of motion, a trajectory, a speed of motion (e.g., a velocity), an acceleration, and / or an orientation (e.g., of a robotic arm and / or an implement or surgical tool configured for a robotic arm), among other examples. In some aspects, the one or more operation parameters may be based on, or may indicate or configure, robotic kinematics of the robot and / or the one or more robotic arms. For example, the one or more operation parameters may be based on, or may indicate or configure, degrees of freedom, joints, and / or movements that the robotic arm(s) are capable of, among other examples. The one or more operation parameters may enable the planning device to model or simulate the planned operations (e.g., steps, surgical actions, movements, and / or other operations) of the robot and / or of the one or more robotic arms during the robotically-assisted procedure (e.g., as defined, configured, or otherwise indicated by the surgical plan).

[0048] As shown by reference number 130, the planning device may determine, using the modeling (e.g., the model with one or more parameters set or configured as described above),feedback information for the surgical plan. For example, the planning device may determine, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations (e.g., performed via the model with one or more parameters set or configured as described above). The feedback information may be patient-specific feedback information for the surgical plan of the robotically-assisted procedure. For example, as described elsewhere herein, one or more parameters of the model may be based on the anatomical information of the patient, thereby enabling the planning device to model or simulate the one or more planned operations (and / or one or more alternative operations) in context of the unique anatomy of a given patient.

[0049] The feedback information may be based on one or more rules or thresholds. For example, as shown by reference number 135, the planning device may determine whether the surgical plan is achievable. The feedback information may indicate whether the one or more planned operations (e.g., of the robot and / or the one or more robotic arms) are achievable based on the simulation and / or the modeling. “Achievable” may refer to the simulation and / or the modeling indicating that the one or more planned operations can be performed in accordance with one or more rules or thresholds. For example, a rule may indicate that the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) are to remain within a patient-specific safety volume (e.g., the modified safety volume, as described elsewhere herein) during the one or more planned operations. If the simulation and / or model indicate that the one or more robotic arms do not remain in the patient-specific safety volume during the one or more planned operations, then the planning device may determine that the surgical plan is not achievable. As another example, the rule may be associated with a distance threshold. The distance threshold may be a value to incorporate a safety margin for the operation(s) of the robot. For example, the planning device may determine, based on the model and / or simulation, whether a distance between the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and an edge or boundary of the patient-specific safety volume satisfies the distance threshold. If the distance satisfies (e.g., is less than or equal to) the distance threshold, then the planning device may determine that the surgical plan is not achievable. For example, this rule and / or distance threshold may enable the planning device to accurately and reliably determine whether the one or more robotic arms can safely maneuver in the surgical field to perform the one or more planned operations indicated by the surgical plan.

[0050] As another example, a rule or threshold may be based on a proximity of the one or more robotic arms (and / or an implement or surgical tool configured for the one or more roboticarms) to one or more sensitive structures of the patient as modeled or simulated for the one or more planned operations indicated by the surgical plan. For example, a sensitive structure rule may indicate that if the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) is expected to contact a sensitive structure of the patient (e.g., as modeled or simulated for the one or more planned operations indicated by the surgical plan), then the surgical plan is not achievable. As another example, the sensitive structure rule may be associated with a distance threshold (e.g., to incorporate a safety margin between the robotic arm(s) and / or surgical tools and any sensitive structures of the patient). The distance threshold may be the same as the distance threshold described above or may be a different distance threshold. The planning device may determine, based on the model and / or simulation, whether a distance between the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and a sensitive structure of the patient satisfies the distance threshold. If the distance satisfies (e.g., is less than or equal to) the distance threshold, then the planning device may determine that the surgical plan is not achievable (e.g., because the risk of contacting or damaging the sensitive structure is high).

[0051] The rules and / or thresholds described herein are provided as examples. Other rules and / or thresholds may be configured and / or used by the planning device in a similar manner to determine whether the surgical plan is achievable. The feedback information may include qualitative information, such as whether the surgical plan is achievable. Additionally, or alternatively, the qualitative information may include whether the surgical plan has a level of risk. For example, if the model or simulation of the one or more planned operations indicates that a distance between the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and a sensitive structure of the patient satisfies (e.g., is less than or equal to) the distance threshold during a planned operation (but does not contact the sensitive structure), then the planning device may determine that the feedback information is to indicate that the surgical plan (or a step of the surgical plan) has a high level of risk. Other metrics may be used by the planning device to determine whether the surgical plan is achievable in a similar manner as described in connection with distance. For example, the other metrics may include an angle (e.g., between one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and an anatomical element), a bone volume, and / or a bone surface, among other examples.

[0052] In some aspects, the planning device may determine risk scores for respective planned operations of the one or more planned operations. A risk score may indicate a likelihood that a planned operation is achievable based on the model and / or simulation. For example, theplanning device may determine a risk score for a planned operation based on a distance between the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and one or more objects or areas during the planned operation, such as the patient-specific safety volume, and / or one or more sensitive structures, among other examples. For example, if the distance is a larger distance, then the planning device may determine that the risk score is to indicate a lower level of risk. If the distance is a smaller distance, then the planning device may determine that the risk score is to indicate a higher level of risk. The planning device may determine the feedback information based on whether the risk scores satisfy one or more risk thresholds. For example, if the risk score for a planned operation satisfies a risk threshold, then the planning device may determine that the surgical plan is not achievable. As another example, if the risk score for a planned operation satisfies a risk threshold, then the planning device may determine that the feedback information is to indicate that the surgical plan and / or a given planned operation or step (e.g., that is associated with the risk score that satisfies the risk threshold) has a high level of risk. In some aspects, if a risk or other information is associated with a sensitive structure, then the feedback information may identify the sensitive structure (e.g., may identify which nerve, tissue, blood vessel, or other sensitive structure that is associated with the information included in the feedback information).

[0053] In some aspects, as shown by reference number 140, the planning device may determine quantitative information to be included in the feedback information. The quantitative information may include one or more values, ranges, or other numeric information that is determined based on, or using, the model and / or simulation of the one or more planned operations. For example, the quantitative feedback may include one or more distances between the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and one or more objects or areas during the one or more planned operations. For example, if a distance during a planned operation satisfies (e.g., is less than or equal to) one or more distance thresholds, then the planning device may determine that the distance (or a range including the distance) is to be included in the feedback information. As described elsewhere herein, the one or more distance thresholds may be based on the structure, element, or object that the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) is near during the planned operation (e.g., a sensitive structure may be associated with a first distance threshold, an anatomical element may be associated with a second distance threshold, a device or object in the surgical field may be associated with a third distance threshold, and so on).

[0054] As an example, if a distance during a planned operation A satisfies (e.g., is less than or equal to) a distance threshold for a sensitive structure, then the planning device may determine that a range that includes the distance is to be included by the feedback information. For example, the distance threshold may be X millimeters. In such examples, the feedback information may include that during the planned operation A, the robotic arm (and / or implement or surgical tool) will be within X millimeters of the sensitive structure (e.g., without specifying the exact distance between the robotic arm (and / or implement or surgical tool) and the sensitive structure).

[0055] As another example, the feedback information may indicate estimated distances between the one or more robotic arms (and / or an implement or surgical tool configured for the one or more robotic arms) and one or more objects or areas during the one or more planned operations. For example, if a distance during a planned operation B satisfies (e.g., is less than or equal to) a distance threshold for a sensitive structure, then the planning device may determine or estimate the expected distance between the robotic arm (and / or implement or surgical tool) and the sensitive structure. The planning device may determine or estimate the distance using the parameter(s) set for the model, as described above. For example, the planning device may determine or estimate the distance using the location of the robotic arm (and / or implement or surgical tool) and the sensitive structure in the coordinate system used by the robot and / or the navigation system. For example, the determined or estimated distance between the robotic arm (and / or implement or surgical tool) and the sensitive structure may be A millimeters. In such examples, the feedback information may include that during the planned operation B, the robotic arm (and / or implement or surgical tool) will be N millimeters from the sensitive structure.

[0056] For example, the feedback information may include proximity information of an implement of the robot for the one or more planned operations. The proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by positioning information included in the anatomical information, of the one or more sensitive structures of the patient. In some aspects, the feedback information may indicate that the proximity information is associated with a planned operation of the one or more planned operations (e.g., to provide an indication as to in which step, cut, surgical action, or other operation the implement is coming close to a sensitive structure or other object). This may enable a surgeon and / or the planning device to identify when an implement (e.g., a surgical tool) is expected to come within a proximity (e.g., near) a sensitive structure of the patient during the robotically-assisted procedure.

[0057] The quantitative information described herein is provided as examples. The planning device may determine other quantitative information in a similar manner as described herein. The quantitative information may provide improved insights for a surgeon for the achievability, risks, and / or level of difficulty associated with the surgical plan for the robotically-assisted procedure. For example, because the feedback information may provide a quantitative information for one or more planned operations of the robot and / or the robotic arm(s), a surgeon make improve preoperative decisions based on the achievability, risks, and / or level of difficulty associated with the surgical plan for the robotically-assisted procedure indicated by the quantitative information.

[0058] In some aspects, as shown by reference number 145, the feedback information may include one or more recommended or suggested modifications to the surgical plan. For example, if the planning device determines that the surgical plan is not achievable (e.g., in a similar manner as described herein), then the planning device may determine one or more recommended or suggested modifications that would cause the surgical plan to be achievable. For example, the planning device may modify a path, trajectory, and / or orientation of a robotic arm and / or of an implement (or surgical tool) for a planned operation that the planning device determined is not achievable (e.g., to cause the planned operation to be achievable). Additionally, or alternatively, if the planning device determines that a planned operation is associated with a risk score that satisfies a risk threshold, then the planning device may determine one or more recommended or suggested modifications that would cause the planned operation to have an updated risk score that does not satisfy the risk threshold, such as by cause a robotic arm or implement to be positioned or to move further away from a sensitive structure of the patient.

[0059] In some aspects, as shown by reference number 150, the feedback information may include an estimated time to complete the surgical plan. For example, the planning device may determine an estimated amount of time to perform the surgical plan based on the simulation and / or the model. In some aspects, the planning device may determine the estimated amount of time for the robot to perform the one or more planned operations. For example, based on the anatomical information of the patient (e.g., bone density, amount of tissue or other biological material to be removed, or another anatomical information), then planning device may determine how much time it is estimated to take for the robot to perform the one or more planned operations specific to the patient. In some aspects, the feedback information may include the estimated time.

[0060] Additionally, or alternatively, the planning device may use the estimated amount of time in one or more other determinations described herein. For example, the planning device may determine whether the surgical plan is achievable and / or appropriate for the patient based on the estimated amount of time. For example, the robotically-assisted procedure may be associated with a standard of care time indicating an allowable amount of time for which the robotically-assisted procedure should take to reduce the likelihood of negative or unintended effects for the patient. If the estimated amount of time is greater than or equal to the standard of care time, then the planning device may determine that the surgical plan is not achievable and / or appropriate for the patient. Additionally, or alternatively, the planning device may determine a risk score for the surgical plan and / or a planned operation based on the estimated amount of time (e.g., if the one or more planned operations are estimated to take more time, then the level of risk for the surgical plan and / or the planned operation may be higher).

[0061] In some aspects, as shown by reference number 155, the feedback information may include one or more suggested orders of operations of the one or more planned operations. For example, the one or more suggested orders may include orders of operations for respective optimized parameters (e.g., different orders of operations may optimize different parameters, such as the quantity of cuts, and / or the amount of time for the robotically-assisted procedure, among other examples). In some examples, the suggested order of operations may be based on an amount of effort involved in the one or more planned operations for a surgeon or other clinician performing the robotically-assisted procedure. For example, the planning device may determine, based on the simulation, effort scores for respective planned operations of the one or more planned operations. The effort scores may indicate a level of effort for performing the respective planned operations. For example, an effort score may indicate a level of effort for a surgeon to perform a planned operation. An effort score may be based on an estimated amount of time for the surgeon to perform one or more operations or steps for the planned operation. Additionally, or alternatively, the effort score may be based on a quantity of operations or steps for the planned operation to be performed by the surgeon. Additionally, or alternatively, the effort score may be based a proximity (e.g., a distance) to one or more sensitive structures for the operations or steps for the planned operation to be performed by the surgeon (e.g., because operations or steps closer to a sensitive structure may require more care or attention from the surgeon).

[0062] An effort score may be a measure of an amount of work to be performed by the surgeon. For example, the effort score may indicate an amount of work to be performed by the surgeon to finish one or more surgical actions (e.g., one or more bone cuts) after one or morerobotically-assisted surgical actions (e.g., after robotically-assisted bone tissue removal). For example, the effort score may indicate an amount of effort and / or an amount of time that the surgeon is estimated to use (e.g., using one or more surgical tools or implements, such as a Kerrison rongeur) to perform or finish one or more surgical actions for the surgical plan.

[0063] The planning device may determine one or more recommendations to improve an execution efficiency of the surgical plan based on the feedback information. For example, the planning device may determine an order of the one or more planned operations based on the effort scores. For example, if two or more planned operations could be performed in any order relative to each other, then the planning device may order the two or more planned operations in order of decreasing effort scores (e.g., recommending that the planned operation taking more effort from the surgeon be performed first). For example, the planning device may determine that a planned operation taking more effort from the surgeon be performed first because the surgeon may have a lower level of fatigue earlier in the robotically-assisted procedure. In some aspects, one or more planned operations may be prerequisite operations to be performed before other planned operations. In such example, the prerequisite operation(s) may be ordered before the other planned operations (e.g., even if the other planned operations have higher effort scores). The feedback information may include the suggested order of operations for the surgical plan.

[0064] In some aspects, the planning device may determine the feedback information based on historical information associated with the surgical plan. For example, the planning device may obtain the historical information (e.g., from one or more databases or other devices). The historical information may indicate anatomical results of one or more historical procedures. An anatomical result may indicate whether a historical procedure was successful, whether any complications occurred, a functional outcome (e.g., a range of motion, pain score, or other metric), and / or a cosmetic outcome, among other examples. The historical information may be specific to a surgeon. For example, the planning device may obtain the historical information from a profile of the surgeon.

[0065] The planning device may determine the feedback information based on historical information, such as by setting values of one or more thresholds or values used to evaluate one or more rules based on the historical information. For example, if the historical information indicates that the surgical plan has a historically low success rate and / or a high rate of complications or issues, then the planning device may set values of one or more thresholds or values used to evaluate one or more rules to a more conservative level (e.g., by setting the value(s) higher or lower depending on the context) to ensure that the determination of thefeedback information takes into account the relatively higher risk associated with the surgical plan. As an example, a more conservative value in the context of a distance threshold for evaluating a proximity to a sensitive structure may be a higher value (e.g., to cause the threshold to be satisfied by larger distances).

[0066] As another example, if the historical information indicates that the surgeon has little or no experience performing the surgical plan (e.g., if the quantity of previously performed procedures by the surgeon for the surgical plan is less than or equal to an experience threshold), then the planning device may set values of one or more thresholds or values used to evaluate one or more rules to a more conservative level (e.g., by setting the value(s) higher or lower depending on the context) to ensure that the determination of the feedback information takes into account the relative inexperience of the surgeon for this surgical plan. As another example, if the historical information indicates that the surgeon has successful experience performing the surgical plan (e.g., if the quantity of previously performed procedures that were successful as performed by the surgeon for the surgical plan is greater than or equal to an experience threshold), then the planning device may set values of one or more thresholds or values used to evaluate one or more rules to a more aggressive level (e.g., by setting the value(s) higher or lower depending on the context) to ensure that the determination of the feedback information takes into account the experience of the surgeon for this surgical plan. As an example, a more aggressive value in the context of a distance threshold for evaluating a proximity to a sensitive structure may be a lower value (e.g., to cause the threshold to be satisfied by smaller distances).

[0067] In some aspects, the feedback information may include a recommended approach for the robotically-assisted procedure. The recommended approach may be a minimally invasive approach, an open approach, a combination of a minimally invasive and open approach (e.g., a mini-open approach), and / or a laparoscopy approach, among other examples. The planning device may determine the recommended approach based on one or more desired surgical outcomes indicated by the surgical plan and / or based on the simulation or model of the one or more planned operations. For example, if the simulation or model of the one or more planned operations indicates that the surgical plan is associated with a high level of complexity (e.g., due to the proximity to sensitive structures or other factors), then the recommended approach may be an open approach. As another example, if the simulation or model of the one or more planned operations indicates that a minimally invasive approach is achievable for the patientspecific anatomy, then the recommended approach may be a minimally invasive approach. In some aspects, the planning device may determine the recommended approach based on the anatomical information. For example, the anatomical information (e.g., obesity, sensitivestructure location, prior procedures or surgeries, or other medical conditions) may indicate one or more constraints that increase the risk or prevent the use of one or more types of approaches.

[0068] The feedback information described herein is provided as an example. The planning device may determine other feedback information for the surgical plan in a similar manner as described herein.

[0069] As shown in Fig. 1C, the planning device may perform one or more actions based on, or using the feedback information. For example, as shown by reference number 160, the planning device may transmit, and the display device may receive, the feedback information. For example, the planning device may provide, for display (e.g., via the display device), the feedback information. For example, the planning device may transmit, to the display device, display information that is configured to cause the display device to display or output the feedback information. For example, as shown by reference number 165, the display device may display the feedback information (e.g., based on receiving the display information). The display device may display the feedback information via a user interface. The user interface may include one or more interactive elements to enable a user (e.g., a surgeon or other user) to view feedback information (e.g., specific to a planned operation, step, or surgical action included in the surgical plan), modify the surgical plan (e.g., modify a planned operation, step, or surgical action included in the surgical plan or select between two or more options for the surgical plan), and / or accept or approve the surgical plan, among other examples. For example, the planning device (and / or another device, such as the navigation system) may obtain an indication that the surgical plan is approved via a user input to the user interface of the display device.

[0070] In some aspects, the planning device may receive, from the display device, an indication of one or more modifications to the surgical plan. In some examples, the indication of the one or more modifications may include an indication of one or more alternative options that may be selectable by a user (e.g., a surgeon). A user (e.g., a surgeon) may view the feedback information and modify the surgical plan based on the feedback information. In some aspects, the planning device may determine updated feedback information based on the modified surgical plan. For example, the planning device may determine the updated feedback information in a similar manner as described herein using the updated or modified surgical plan. In such examples, the planning device may provide, for display (e.g., via the display device), the updated feedback information for the updated or modified surgical plan. In this way, a surgeon may view feedback for a surgical plan of a robotically-assisted procedure and may iteratively update the surgical plan. This improves preoperative planning for robotically-assisted procedures and ensures that initiated robotically-assisted procedures have accurate and reliable preoperative planning.

[0071] Additionally, or alternatively, as shown by reference number 170, the planning device may transmit, to the navigation system of the robot, instructions associated with an execution of the one or more planned operations based on the feedback information (e.g., an execution of a selected option or alternative for the surgical plan). For example, the planning device may transmit the instruction based on the feedback information indicating that the one or more planned operations are achievable and / or associated with risk scores that indicate acceptable or low levels of risk. Additionally, or alternatively, the planning device may transmit the instructions based on, in response to, or otherwise associated with obtaining an approval of the surgical plan via a user input (e.g., from a surgeon or another user) to the display device or another device. The instructions may indicate the one or more planned operations. In some aspects, a surgeon or other surgical team member may control the operation of the robot during the robotically-assisted procedure and the instructions may facilitate the control by provided (e.g., pre-loading) the planned operations for the surgical plan (e.g., thereby enabling the robot and / or robotic arms to perform one or more automated operations and / or enabling the navigation system to provide enhanced instruction and / or feedback to the surgeon or other surgical team member).

[0072] As shown by reference number 175, the navigation system may transmit, and the robot may receive, navigation instructions that are based on the instructions received from the planning device. For example, the navigation instructions may be configured to cause the one or more robotic arms to perform the one or more planned operations for the surgical plan. As shown by reference number 180, the robot may perform one or operations for the surgical plan based on, or in response to, receiving the navigation instructions. For example, the planning device may enable the robot to perform operations during the robotically-assisted procedure based on, or in response to, the feedback information indicating that the surgical plan is achievable for a given patient and / or associated with risk scores that indicate acceptable or low levels of risk for the given patient. As described elsewhere herein, movement and / or operations of the robot may be initiated and / or controlled by a surgeon and the one or operations for the surgical plan may include automated or semi-automated operations that are based on the one or more planned operations, thereby facilitating and / or improving the accuracy of the user- controlled (e.g., surgeon controlled) operation of the robot. In some aspects, the robot may autonomously (e.g., without or with limited surgeon control or input) perform at least one (or all) of the one or more planned operations during the robotically-assisted procedure.

[0073] As indicated above, Figs. 1A-1C are provided as an example. Other examples may differ from what is described with regard to Figs. 1A-1C. The number and arrangement of devices shown in Figs. 1A-1C are provided as an example. In practice, there may be additional devices, fewer devices, different devices, or differently arranged devices than those shown in Figs. 1A-1C. Furthermore, two or more devices shown in Figs. 1A-1C may be implemented within a single device, or a single device shown in Figs. 1A-1C may be implemented as multiple, distributed devices. Additionally, or alternatively, a set of devices (e.g., one or more devices) shown in Figs. 1A-1C may perform one or more functions described as being performed by another set of devices shown in Figs. 1A-1C.

[0074] Fig. 2 is a diagram of an example environment 200 in which systems and / or methods described herein may be implemented. As shown in Fig. 2, environment 200 may include a planning device 210, an image database 220, a robot 230 (e.g., that includes one or more robotic arms 235), a navigation system 240, a display device 250, and a network 260. Devices of environment 200 may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections. In some examples, the environment 200 may be a surgical environment, a surgical field, a surgical space, a connected surgical operating room, or a connected surgical ecosystem, among other examples.

[0075] The planning device 210 may include one or more devices capable of receiving, generating, storing, processing, providing, and / or routing information associated with surgical plan assessments, as described elsewhere herein. The planning device 210 may include a communication device and / or a computing device. For example, the planning device 210 may include a server, such as an application server, a client server, a web server, a database server, a host server, a proxy server, a virtual server (e.g., executing on computing hardware), or a server in a cloud computing system. In some implementations, the planning device 210 may include computing hardware used in a cloud computing environment. In some examples, the planning device 210 may include a surgical navigation system. In some examples, the planning device 210 may be included in another device in the environment 200, such as the navigation system 240.

[0076] The image database 220 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with surgical plan assessments, as described elsewhere herein. The image database 220 may include a communication device and / or a computing device. For example, the image database 220 may include a data structure, a database, a data source, a server, a database server, an application server, a client server, a web server, a host server, a proxy server, a virtual server (e.g., executingon computing hardware), a server in a cloud computing system, a device that includes computing hardware used in a cloud computing environment, or a similar type of device. As an example, the image database 220 may store medical imaging data, as described elsewhere herein.

[0077] In some implementations, the medical imaging data may be generated by, captured by, and / or provided by (e.g., to the image database 220) one or more imaging devices. The imaging device may be operable to image anatomical element(s) (e.g., a bone, veins, tissue, and / or other anatomical elements) and / or other aspects of patient anatomy to output medical imaging data (e.g., image data depicting or corresponding to a bone, veins, and / or tissue). “Image data” or “imaging data” as used herein refers to the data generated or captured by an imaging device, including in a machine-readable form, a graphical / visual form, and in any other form. In some examples, medical imaging data may include data corresponding to an anatomical feature of a patient, or to a portion thereof. The image data may be or include a preoperative image, an intraoperative image, a postoperative image, or an image taken independently of any surgical procedure. In some examples, a first imaging device may be used to obtain first image data (e.g., a first image) at a first time, and a second imaging device may be used to obtain second image data (e.g., a second image) at a second time after the first time. The imaging device may be capable of taking a 2D image or a 3D image to yield the image data. The imaging device may be or comprise, for example, an ultrasound scanner (which may comprise, for example, a physically separate transducer and receiver, or a single ultrasound transceiver), an 0-arm, a C-arm, a G-arm, or any other device utilizing X-ray -based imaging (e.g., a fluoroscope, a CT scanner, or other X-ray machine), a MRI scanner, an OCT scanner, an endoscope, a microscope, an optical camera, a thermographic camera (e.g., an infrared camera), a radar system (which may include a transmitter, a receiver, a processor, and one or more antennas), or any other imaging device suitable for obtaining images of an anatomical feature of a patient. In some examples, medical imaging data, as described herein, may be considered to be continuous and / or provided as an image data stream (e.g., to the image database 220 via one or more imaging devices), such as if the medical image data represents two or more frames per second.

[0078] The robot 230 may be any surgical robot or surgical robotic system. The robot 230 may include a robotic guidance system. The robot may be configured to position, for example, a retraction assembly and / or an end unit at one or more precise position(s) and orientation(s), and / or to return a retraction assembly and / or an end unit to the same position(s) and orientation(s) at a later point in time. The end unit may include a surgical instrument, a surgicaltool, and / or an implement, among other examples. The robot 230 may additionally or alternatively be configured to manipulate a surgical tool or implement (whether based on guidance from the navigation system 240 or not) to accomplish or to assist with a surgical task. In some embodiments, the robot 230 may be configured to hold and / or manipulate an anatomical element during or in connection with a surgical procedure. The robot 230 may comprise one or more robotic arms 235. In some embodiments, the robotic arm 235 may comprise a first robotic arm and a second robotic arm, though the robot 230 may comprise more than two robotic arms. In some embodiments, one or more of the robotic arms 235 may be used to hold and / or maneuver a retraction assembly and / or an end unit. In embodiments where two retraction assemblies and end units are used, one robotic arm 235 may hold one set, and another robotic arm 235 may hold another set. Each robotic arm 235 may be positioned or moved independently of the other robotic arm(s). The robotic arms 235 may be controlled in a single, shared coordinate space, or in separate coordinate spaces.

[0079] The robot 230, together with the robotic arm(s) 235, may have one, two, three, four, five, six, seven, or more degrees of freedom. Further, the robotic arm 235 may be positioned in any pose, plane, and / or focal point. The pose includes a position and an orientation. As a result, the retraction assembly, the end unit, a surgical tool, or other object held by the robot 230 (or by the robotic arm 235) may be precisely positioned in one or more needed and specific positions and orientations. The robotic arm(s) 235 may comprise one or more sensors that enable one or more processors (or one or more processors of the robot 230) to determine a precise pose in space of the robotic arm (as well as any object or element held by or secured to the robotic arm). The one or more sensors may also measure a force at an end of the robotic arm 235. In some implementations, the sensors may measure a force at an end unit disposed at the end of the robotic arm 235. Sensors may be used in any other component in a similar manner. For example, a manually operated surgical tool may comprise a retraction assembly and a sensor for measuring a force at the surgical tool. The retraction assembly may be triggered when the sensor measures a force at the surgical tool that satisfies a force threshold.

[0080] In some examples, reference markers (e.g., navigation markers) may be placed on the robot 230 (including, e.g., on the robotic arm 235), an imaging device, a retraction assembly, an end unit or any other object, device, or component in a surgical space. The reference markers may be tracked by the navigation system 240, and the results of the tracking may be used by the robot 230 and / or by an operator. In some examples, the navigation system 240 may be used to track other components in the environment 200 (e.g., the retraction assembly, the end unit, one or more people, and / or one or more objects) and the one or more components can operatewithout the use of the robot 230 (e.g., with the surgeon manually manipulating a retraction assembly and an end unit and / or one or more surgical tools, based on information and / or instructions generated by the navigation system 240).

[0081] The navigation system 240 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with surgical plan assessments, as described elsewhere herein. The navigation system 240 may include a communication device and / or a computing device. The navigation system 240 may provide navigation for a surgeon and / or a surgical robot during a surgical procedure. The navigation system 240 may include one or more cameras or other sensor(s) for tracking one or more reference markers, navigated trackers, or other objects within the operating room or other room in which some or all of the environment 200 is located. The one or more cameras may be optical cameras, endoscopic cameras, infrared cameras, or other cameras. In some embodiments, the navigation system 240 may include one or more electromagnetic sensors. In some examples, the navigation system 240 may be used to track a position and orientation (e.g., a pose) of an imaging device, a retraction assembly, an end unit, the robot 230 and / or robotic arm 235, and / or one or more surgical tools (e.g., to track a pose of a navigated tracker attached, directly or indirectly, in fixed relation to the one or more of the examples described above). The navigation system 240 may include a display for displaying one or more images from an external source or for displaying an image and / or video stream from the one or more cameras or other sensors of the navigation system 240. In some examples, the display may be, or may include, the display device 250. The navigation system 240 may be configured to provide guidance to a surgeon or other user in the environment 200, to the robot 230, or to any other device or component of the environment 200. The guidance provided by the navigation system 240 may include a pose of one or more anatomical elements, whether or not a tool is in the proper trajectory, and / or how to move a tool into the proper trajectory to carry out a surgical task according to a preoperative or other surgical plan, among other examples.

[0082] The display device 250 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information associated with surgical plan assessments, as described elsewhere herein. The display device 250 may include a communication device and / or a computing device. The display device 250 may be any device capable of, or configured to, provide, display, or otherwise output (e.g., via a visual output and / or an audio output) information. For example, the display device 250 may include a wireless communication device, a mobile phone, a user equipment, a laptop computer, a tablet computer, a desktop computer, a wearable communication device (e.g., a smart wristwatch, apair of smart eyeglasses, a head mounted display, or a virtual reality headset), a keyboard, mouse, trackball, monitor, television, screen, touch screen, a speaker, and / or any other device for receiving information from a user and / or for providing information to a user. In some example, the display device 250 may be included in one or more devices or components of the environment 200. For example, the display device 250 may be a component of the planning device 210, and / or the navigation system 240, among other examples. In some examples, multiple devices or components of the environment 200 may include display devices 250. For example, the planning device 210 may include a first display device 250 and the navigation system 240 may include a second display device 250.

[0083] The network 260 may include one or more wired and / or wireless networks. For example, the network 260 may include a wireless wide area network (e.g., a cellular network or a public land mobile network), a local area network (e.g., a wired local area network or a wireless local area network (WLAN), such as a Wi-Fi network), a personal area network (e.g., a Bluetooth network), a near-field communication network, a telephone network, a private network, the Internet, and / or a combination of these or other types of networks. The network 260 enables communication among the devices of environment 200.

[0084] The number and arrangement of devices and networks shown in Fig. 2 are provided as an example. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown in Fig. 2. Furthermore, two or more devices shown in Fig. 2 may be implemented within a single device, or a single device shown in Fig. 2 may be implemented as multiple, distributed devices. Additionally, or alternatively, a set of devices (e.g., one or more devices) of environment 200 may perform one or more functions described as being performed by another set of devices of environment 200.

[0085] Fig. 3 is a diagram of example components of a device 300 associated with surgical plan assessments. The device 300 may correspond to the planning device 210, the image database 220, the robot 230, the robotic arm 235, the navigation system 240, and / or the display device 250. In some implementations, the planning device 210, the image database 220, the robot 230, the robotic arm 235, the navigation system 240, and / or the display device 250 may include one or more devices 300 and / or one or more components of the device 300. As shown in Fig. 3, the device 300 may include a bus 310, a processor 320, a memory 330, an input component 340, an output component 350, and / or a communication component 360.

[0086] The bus 310 may include one or more components that enable wired and / or wireless communication among the components of the device 300. The bus 310 may couple togethertwo or more components of Fig. 3, such as via operative coupling, communicative coupling, electronic coupling, and / or electric coupling. For example, the bus 310 may include an electrical connection (e.g., a wire, a trace, and / or a lead) and / or a wireless bus. The processor 320 may include a central processing unit, a graphics processing unit, a microprocessor, a controller, a microcontroller, a digital signal processor, a field-programmable gate array, an application-specific integrated circuit, and / or another type of processing component. The processor 320 may be implemented in hardware, firmware, or a combination of hardware and software. In some implementations, the processor 320 may include one or more processors capable of being programmed to perform one or more operations or processes described elsewhere herein.

[0087] The memory 330 may include volatile and / or nonvolatile memory. For example, the memory 330 may include random access memory (RAM), read only memory (ROM), a hard disk drive, and / or another type of memory (e.g., a flash memory, a magnetic memory, and / or an optical memory). The memory 330 may include internal memory (e.g., RAM, ROM, or a hard disk drive) and / or removable memory (e.g., removable via a universal serial bus connection). The memory 330 may be a non-transitory computer-readable medium. The memory 330 may store information, one or more instructions, and / or software (e.g., one or more software applications) related to the operation of the device 300. In some implementations, the memory 330 may include one or more memories that are coupled (e.g., communicatively coupled) to one or more processors (e.g., processor 320), such as via the bus 310. Communicative coupling between a processor 320 and a memory 330 may enable the processor 320 to read and / or process information stored in the memory 330 and / or to store information in the memory 330.

[0088] The input component 340 may enable the device 300 to receive input, such as user input and / or sensed input. For example, the input component 340 may include a touch screen, a keyboard, a keypad, a mouse, a button, a microphone, a switch, a sensor, a global positioning system sensor, a global navigation satellite system sensor, an accelerometer, a gyroscope, and / or an actuator. The output component 350 may enable the device 300 to provide output, such as via a display, a speaker, and / or a light-emitting diode. The communication component 360 may enable the device 300 to communicate with other devices via a wired connection and / or a wireless connection. For example, the communication component 360 may include a receiver, a transmitter, a transceiver, a modem, a network interface card, and / or an antenna.

[0089] The device 300 may perform one or more operations or processes described herein. For example, a non-transitory computer-readable medium (e.g., memory 330) may store a setof instructions (e.g., one or more instructions or code) for execution by the processor 320. The processor 320 may execute the set of instructions to perform one or more operations or processes described herein. In some implementations, execution of the set of instructions, by one or more processors 320, causes the one or more processors 320 and / or the device 300 to perform one or more operations or processes described herein. In some implementations, hardwired circuitry may be used instead of or in combination with the instructions to perform one or more operations or processes described herein. Additionally, or alternatively, the processor 320 may be configured to perform one or more operations or processes described herein. Thus, implementations described herein are not limited to any specific combination of hardware circuitry and software.

[0090] The number and arrangement of components shown in Fig. 3 are provided as an example. The device 300 may include additional components, fewer components, different components, or differently arranged components than those shown in Fig. 3. Additionally, or alternatively, a set of components (e.g., one or more components) of the device 300 may perform one or more functions described as being performed by another set of components of the device 300.

[0091] Fig. 4 is a flowchart of an example process 400 associated with surgical plan assessments. In some aspects, one or more process blocks of Fig. 4 are performed by a planning device (e.g., planning device 210). In some aspects, one or more process blocks of Fig. 4 are performed by another device or a group of devices separate from or including the planning device, such as the image database 220, the robot 230, one or more robotic arms 235, the navigation system 240, and / or the display device 250. Additionally, or alternatively, one or more process blocks of Fig. 4 may be performed by one or more components of device 300, such as processor 320, memory 330, input component 340, output component 350, and / or communication component 360.

[0092] As shown in Fig. 4, process 400 may include obtaining a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot for the robotically-assisted procedure, and wherein the surgical plan is associated with one or more anatomical elements (block 410). For example, the planning device may obtain a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot for the robotically-assisted procedure, and wherein the surgical plan is associated with one or more anatomical elements, as described above. In some aspects, the surgical planindicates one or more planned operations of one or more robotic arms of the robot for the robotically-assisted procedure. In some aspects, the surgical plan is associated with one or more anatomical elements.

[0093] As further shown in Fig. 4, process 400 may optionally include obtaining configuration information associated with the robot (block 420). For example, the planning device may obtain configuration information associated with the robot, as described above.

[0094] As further shown in Fig. 4, process 400 may include determining, using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements (block 430). For example, the planning device may determine, using medical imaging data of a patient of the robotically- assisted procedure, anatomical information of the patient associated with the one or more anatomical elements, as described above.

[0095] As further shown in Fig. 4, process 400 may include generating, using configuration information associated with the robot and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure (block 440). For example, the planning device may generate, using configuration information associated with the robot and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure, as described above.

[0096] As further shown in Fig. 4, process 400 may include determining, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations (block 450). For example, the planning device may determine, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations, as described above.

[0097] As further shown in Fig. 4, process 400 may include performing, based on the feedback information, one or more actions (block 460). For example, the planning device may perform, based on the feedback information, one or more actions, as described above.

[0098] Process 400 may include additional aspects, such as any single aspect or any combination of aspects described below and / or in connection with one or more other processes described elsewhere herein.

[0099] In a first aspect, performing the one or more actions includes transmitting, to a navigation system of the robot, instructions associated with an execution of the one or more planned operations based on the feedback information indicating that the one or more planned operations are achievable.

[0100] In a second aspect, alone or in combination with the first aspect, performing the one or more actions includes providing, for display, the feedback information.

[0101] In a third aspect, alone or in combination with one or more of the first and second aspects, the anatomical information indicates positioning information of one or more sensitive structures of the patient, the feedback information includes proximity information of an implement of the robot for the one or more planned operations, and the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

[0102] In a fourth aspect, alone or in combination with one or more of the first through third aspects, the feedback information indicates that the proximity information is associated with a planned operation of the one or more planned operations.

[0103] In a fifth aspect, alone or in combination with one or more of the first through fourth aspects, determining the feedback information includes determining risk scores for respective planned operations of the one or more planned operations, where the risk scores indicate a likelihood that the respective planned operations are achievable based on the simulation, and determining the feedback information based on whether the risk scores satisfy one or more thresholds.

[0104] In a sixth aspect, alone or in combination with one or more of the first through fifth aspects, the feedback information indicates whether the one or more planned operations are achievable based on the simulation.

[0105] In a seventh aspect, alone or in combination with one or more of the first through sixth aspects, the feedback information indicates an estimated amount of time to perform the surgical plan based on the simulation.

[0106] In an eighth aspect, alone or in combination with one or more of the first through seventh aspects, the configuration information indicates a safety volume indicating an area in which the one or more robotic arms can safely move, and generating the modeling of the surgical field includes modifying the safety volume based on the anatomical information.

[0107] In a ninth aspect, alone or in combination with one or more of the first through eighth aspects, the feedback information indicates a suggested modification to the surgical plan based on the simulation.

[0108] In a tenth aspect, alone or in combination with one or more of the first through ninth aspects, determining the feedback information includes determining, based on the simulation, effort scores for respective planned operations of the one or more planned operations, wherethe effort scores indicate a level of effort for performing the respective planned operations, and determining an order of the one or more planned operations based on the effort scores.

[0109] In an eleventh aspect, alone or in combination with one or more of the first through tenth aspects, process 400 includes obtaining historical information for one or more historical procedures associated with the surgical plan, the historical information indicates anatomical results of the one or more historical procedures, and the feedback information is based on the historical information.

[0110] Although Fig. 4 shows example blocks of process 400, in some aspects, process 400 includes additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in Fig. 4. Additionally, or alternatively, two or more of the blocks of process 400 may be performed in parallel.

[0111] Fig. 5 is a flowchart of an example process 500 associated with surgical plan assessments. In some aspects, one or more process blocks of Fig. 5 are performed by a planning device (e.g., planning device 210). In some aspects, one or more process blocks of Fig. 5 are performed by another device or a group of devices separate from or including the planning device, such as the image database 220, the robot 230, one or more robotic arms 235, the navigation system 240, and / or the display device 250. Additionally, or alternatively, one or more process blocks of Fig. 5 may be performed by one or more components of device 300, such as processor 320, memory 330, input component 340, output component 350, and / or communication component 360.

[0112] As shown in Fig. 5, process 500 may include obtaining a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot, and wherein the surgical plan is associated with one or more anatomical elements (block 510). For example, the planning device may obtain a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot, and wherein the surgical plan is associated with one or more anatomical elements, as described above. In some aspects, the surgical plan indicates one or more planned operations of one or more robotic arms of the robot. In some aspects, the surgical plan is associated with one or more anatomical elements.

[0113] As further shown in Fig. 5, process 500 may optionally include obtaining configuration information associated with the robot (block 520). For example, the planning device may obtain configuration information associated with the robot, as described above.

[0114] As further shown in Fig. 5, process 500 may include determining, using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements (block 530). For example, the planning device may determine, using medical imaging data of a patient of the robotically- assisted procedure, anatomical information of the patient associated with the one or more anatomical elements, as described above.

[0115] As further shown in Fig. 5, process 500 may include determining feedback information for the surgical plan based on the configuration information and the anatomical information (block 540). For example, the planning device may determine feedback information for the surgical plan based on the configuration information and the anatomical information, as described above.

[0116] As further shown in Fig. 5, process 500 may include performing, based on the feedback information, one or more actions (block 550). For example, the planning device may perform, based on the feedback information, one or more actions, as described above.

[0117] Process 500 may include additional aspects, such as any single aspect or any combination of aspects described below and / or in connection with one or more other processes described elsewhere herein.

[0118] In a first aspect, process 500 includes generating, using the configuration information and the anatomical information, a modeling of a surgical field associated with the robotically- assisted procedure, where the feedback information is based on a simulation of the one or more planned operations that uses the modeling.

[0119] In a second aspect, alone or in combination with the first aspect, performing the one or more actions includes transmitting, to a navigation system of the robot, instructions associated with an execution of the one or more planned operations based on the feedback information indicating that the one or more planned operations are achievable.

[0120] In a third aspect, alone or in combination with one or more of the first and second aspects, performing the one or more actions includes providing, for display, the feedback information.

[0121] In a fourth aspect, alone or in combination with one or more of the first through third aspects, the anatomical information indicates positioning information of one or more sensitive structures of the patient, the feedback information includes proximity information of an implement of the robot for the one or more planned operations, and the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

[0122] In a fifth aspect, alone or in combination with one or more of the first through fourth aspects, the feedback information indicates that the proximity information is associated with a planned operation of the one or more planned operations.

[0123] In a sixth aspect, alone or in combination with one or more of the first through fifth aspects, determining the feedback information includes determining risk scores for respective planned operations of the one or more planned operations, where the risk scores indicate a likelihood that the respective planned operations are achievable based on the configuration information and the anatomical information, and determining the feedback information based on whether the risk scores satisfy one or more thresholds.

[0124] In a seventh aspect, alone or in combination with one or more of the first through sixth aspects, the feedback information indicates whether the one or more planned operations are achievable based on the configuration information and the anatomical information.

[0125] In an eighth aspect, alone or in combination with one or more of the first through seventh aspects, the feedback information indicates an estimated amount of time to perform the surgical plan based on the configuration information and the anatomical information.

[0126] In a ninth aspect, alone or in combination with one or more of the first through eighth aspects, the configuration information indicates a safety volume indicating an area in which the one or more robotic arms can safely move, and process 500 includes modifying the safety volume based on the anatomical information.

[0127] In a tenth aspect, alone or in combination with one or more of the first through ninth aspects, the feedback information indicates a suggested modification to the surgical plan based on the configuration information and the anatomical information.

[0128] In an eleventh aspect, alone or in combination with one or more of the first through tenth aspects, determining the feedback information includes determining, based on the configuration information and the anatomical information, effort scores for respective planned operations of the one or more planned operations, where the effort scores indicate a level of effort for performing the respective planned operations, and determining an order of the one or more planned operations based on the effort scores.

[0129] In a twelfth aspect, alone or in combination with one or more of the first through eleventh aspects, process 500 includes obtaining historical information for one or more historical procedures associated with the surgical plan, the historical information indicates anatomical results of the one or more historical procedures, and the feedback information is based on the historical information.

[0130] Although Fig. 5 shows example blocks of process 500, in some aspects, process 500 includes additional blocks, fewer blocks, different blocks, or differently arranged blocks than those depicted in Fig. 5. Additionally, or alternatively, two or more of the blocks of process 500 may be performed in parallel.

[0131] The following provides non-limiting Examples of the present invention:

[0132] Example 1 : A method, comprising: obtaining, by a device, a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot for the robotically-assisted procedure, and wherein the surgical plan is associated with one or more anatomical elements; determining, using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements; generating, using configuration information associated with the robot and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure; determining, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations; and performing, based on the feedback information, one or more actions.

[0133] Example 2: The method of Example 1, wherein performing the one or more actions comprises: transmitting, to a navigation system of the robot, instructions associated with an execution of the one or more planned operations based on the feedback information indicating that the one or more planned operations are achievable.

[0134] Example 3: The method of any of Examples 1-2, wherein performing the one or more actions comprises: providing, for display, the feedback information.

[0135] Example 4: The method of any of Examples 1-3, wherein the anatomical information indicates positioning information of one or more sensitive structures of the patient, wherein the feedback information includes proximity information of an implement of the robot for the one or more planned operations, and wherein the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

[0136] Example 5: The method of Example 4, wherein the feedback information indicates that the proximity information is associated with a planned operation of the one or more planned operations.

[0137] Example 6: The method of any of Examples 1-5, wherein determining the feedback information comprises: determining risk scores for respective planned operations of the one or more planned operations, wherein the risk scores indicate a likelihood that the respectiveplanned operations are achievable based on the simulation; and determining the feedback information based on whether the risk scores satisfy one or more thresholds.

[0138] Example 7: The method of any of Examples 1-6, wherein the feedback information indicates whether the one or more planned operations are achievable based on the simulation.

[0139] Example 8: The method of any of Examples 1-7, wherein the feedback information indicates an estimated amount of time to perform the surgical plan based on the simulation.

[0140] Example 9 : The method of any of Examples 1-8, wherein the configuration information indicates a safety volume indicating an area in which the one or more robotic arms can safely move, and wherein generating the modeling of the surgical field comprises: modifying the safety volume based on the anatomical information.

[0141] Example 10: The method of any of Examples 1-9, wherein the feedback information indicates a suggested modification to the surgical plan based on the simulation.

[0142] Example 11: The method of any of Examples 1-10, wherein determining the feedback information comprises: determining, based on the simulation, effort scores for respective planned operations of the one or more planned operations, wherein the effort scores indicate a level of effort for performing the respective planned operations; and determining an order of the one or more planned operations based on the effort scores.

[0143] Example 12: The method of any of Examples 1-11, further comprising: obtaining historical information for one or more historical procedures associated with the surgical plan, wherein the historical information indicates anatomical results of the one or more historical procedures, and wherein the feedback information is based on the historical information.

[0144] Example 13: A method, comprising: obtaining, by a device, a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot, and wherein the surgical plan is associated with one or more anatomical elements; obtaining, by the device, configuration information associated with the robot; determining, by the device and using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements; determining, by the device, feedback information for the surgical plan based on the configuration information and the anatomical information; and performing, by the device and based on the feedback information, one or more actions.

[0145] Example 14: The method of Example 13, further comprising: generating, using the configuration information and the anatomical information, a modeling of a surgical fieldassociated with the robotically-assisted procedure, wherein the feedback information is based on a simulation of the one or more planned operations that uses the modeling.

[0146] Example 15: The method of any of Examples 13-14, wherein performing the one or more actions comprises: transmitting, to a navigation system of the robot, instructions associated with an execution of the one or more planned operations based on the feedback information indicating that the one or more planned operations are achievable.

[0147] Example 16: The method of any of Examples 13-15, wherein performing the one or more actions comprises: providing, for display, the feedback information.

[0148] Example 17: The method of any of Examples 13-16, wherein the anatomical information indicates positioning information of one or more sensitive structures of the patient, wherein the feedback information includes proximity information of an implement of the robot for the one or more planned operations, and wherein the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

[0149] Example 18: The method of Example 17, wherein the feedback information indicates that the proximity information is associated with a planned operation of the one or more planned operations.

[0150] Example 19: The method of any of Examples 13-18, wherein determining the feedback information comprises: determining risk scores for respective planned operations of the one or more planned operations, wherein the risk scores indicate a likelihood that the respective planned operations are achievable based on the configuration information and the anatomical information; and determining the feedback information based on whether the risk scores satisfy one or more thresholds.

[0151] Example 20: The method of any of Examples 13-19, wherein the feedback information indicates whether the one or more planned operations are achievable based on the configuration information and the anatomical information.

[0152] Example 21: The method of any of Examples 13-20, wherein the feedback information indicates an estimated amount of time to perform the surgical plan based on the configuration information and the anatomical information.

[0153] Example 22: The method of any of Examples 13-21, wherein the configuration information indicates a safety volume indicating an area in which the one or more robotic arms can safely move, and the method further comprising: modifying the safety volume based on the anatomical information.

[0154] Example 23: The method of any of Examples 13-22, wherein the feedback information indicates a suggested modification to the surgical plan based on the configuration information and the anatomical information.

[0155] Example 24: The method of any of Examples 13-23, wherein determining the feedback information comprises: determining, based on the configuration information and the anatomical information, effort scores for respective planned operations of the one or more planned operations, wherein the effort scores indicate a level of effort for performing the respective planned operations; and determining an order of the one or more planned operations based on the effort scores.

[0156] Example 25: The method of any of Examples 13-24, further comprising: obtaining historical information for one or more historical procedures associated with the surgical plan, wherein the historical information indicates anatomical results of the one or more historical procedures, and wherein the feedback information is based on the historical information.

[0157] Example 26: A system configured to perform one or more operations recited in one or more of Examples 1-25.

[0158] Example 27: An apparatus comprising means for performing one or more operations recited in one or more of Examples 1-25.

[0159] Example 28: A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising one or more instructions that, when executed by a device, cause the device to perform one or more operations recited in one or more of Examples 1-25.

[0160] Example 29: A computer program product comprising instructions or code for executing one or more operations recited in one or more of Examples 1-25.

[0161] Example 30: A system comprising a robot and one or more processors configured to perform one or more operations recited in one or more of Examples 1-25.

[0162] Example 31 : A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising: one or more instructions that, when executed by one or more processors of a device, cause the device to: obtain a surgical plan that is associated with a robotically-assisted procedure that is associated with a robot, wherein the surgical plan indicates one or more planned operations of one or more robotic arms of the robot for the robotically- assisted procedure, and wherein the surgical plan is associated with one or more anatomical elements; determine, using medical imaging data of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements; generate, using configuration information associated with the robot and the anatomical information, a modeling of a surgical field associated with the robotically-assistedprocedure; determine, using the modeling, feedback information for the surgical plan based on a simulation of the one or more planned operations; and perform, based on the feedback information, one or more actions.

[0163] Example 32: The non-transitory computer-readable medium of claim 13, wherein the one or more instructions, that cause the device to perform the one or more actions, cause the device to provide, for display, the feedback information.

[0164] Example 33: The non-transitory computer-readable medium of any of claims 13-19, wherein the anatomical information indicates positioning information of one or more sensitive structures of the patient, wherein the feedback information includes proximity information of an implement of the robot for the one or more planned operations, and wherein the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

[0165] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the aspects described herein to the precise forms that are described. Modifications and variations may be made in light of the above description or may be acquired from practice of the aspects described herein.

[0166] As used herein, the term “component” is intended to be broadly construed as hardware, firmware, and / or a combination of hardware and software. It will be apparent that systems and / or methods described herein may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not limiting of the aspects described herein. Thus, the operation and behavior of the systems and / or methods are described herein without reference to specific software code — it being understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0167] As used herein, satisfying a threshold may, depending on the context, refer to a value being greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, or the like.

[0168] Even though particular combinations of features are recited in the claims and / or described in the specification, these combinations are not intended to limit the aspects described herein. In fact, many of these features may be combined in ways not specifically recited in the claims and / or described in the specification. Although each dependent claim listed below may directly depend on only one claim, the description includes each dependent claim in combination with every other claim in the claim set. As used herein, a phrase referring to “atleast one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiple of the same item.

[0169] When “a component” or “one or more components” (or another element, such as “a controller” or “one or more controllers”) is described or claimed (within a single claim or across multiple claims) as performing multiple operations or being configured to perform multiple operations, this language is intended to broadly cover a variety of architectures and environments. For example, unless explicitly claimed otherwise (e.g., via the use of “first component” and “second component” or other language that differentiates components in the claims), this language is intended to cover a single component performing or being configured to perform all of the operations, a group of components collectively performing or being configured to perform all of the operations, a first component performing or being configured to perform a first operation and a second component performing or being configured to perform a second operation, or any combination of components performing or being configured to perform the operations. For example, when a claim has the form “one or more components configured to: perform X; perform Y ; and perform Z,” that claim should be interpreted to mean “one or more components configured to perform X; one or more (possibly different) components configured to perform Y ; and one or more (also possibly different) components configured to perform Z.”

[0170] No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items, and may be used interchangeably with “one or more.” Further, as used herein, the article “the” is intended to include one or more items referenced in connection with the article “the” and may be used interchangeably with “the one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, or a combination of related and unrelated items,), and may be used interchangeably with “one or more.” Where only one item is intended, the phrase “only one” or similar language is used. Also, as used herein, the terms “has,” “have,” “having,” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise. Also, as used herein, the term “or” is intended to be inclusive when used in a series and may be used interchangeably with “and / or,” unless explicitly stated otherwise (e.g., if used in combination with “either” or “only one of’).

Claims

CLAIMSWhat is claimed is:

1. A system (200, 300), comprising: a robot (230) including one or more robotic arms (235); one or more memories (330); and one or more processors (320), communicatively coupled to the one or more memories (330), configured to: obtain a surgical plan (105) that is associated with a robotically-assisted procedure that is associated with the robot (230), wherein the surgical plan (105) indicates one or more planned operations of the one or more robotic arms (235) for the robotically-assisted procedure, and wherein the surgical plan (105) is associated with one or more anatomical elements; determine, using medical imaging data (115) of a patient of the robotically- assisted procedure, anatomical information of the patient associated with the one or more anatomical elements; generate, using configuration information (110) associated with the robot (230) and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure; determine, using the modeling, feedback information (130) for the surgical plan (105) based on a simulation of the one or more planned operations; and perform, based on the feedback information (130), one or more actions.

2. The system (200, 300) of claim 1, wherein the one or more processors (320), to perform the one or more actions, are configured to: transmit, to a navigation system (240) of the robot (230), instructions associated with an execution of the one or more planned operations based on the feedback information (130) indicating that the one or more planned operations are achievable.

3. The system (200, 300) of any of claims 1-2, wherein the one or more processors (320), to perform the one or more actions, are configured to: provide, for display, the feedback information (130).

4. The system (200, 300) of any of claims 1-3, wherein the anatomical information indicates positioning information of one or more sensitive structures of the patient, wherein the feedback information (130) includes proximity information of an implement of the robot (230) for the one or more planned operations, and wherein the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

5. The system (200, 300) of any of claims 1-4, wherein the one or more processors (320), to determine the feedback information (130), are configured to: determine risk scores for respective planned operations of the one or more planned operations, wherein the risk scores indicate a likelihood that the respective planned operations are achievable based on the simulation; and determine the feedback information (130) based on whether the risk scores satisfy one or more thresholds.

6. The system (200, 300) of any of claims 1-5, wherein the configuration information (110) indicates a safety volume indicating an area in which the one or more robotic arms (235) can safely move, and wherein the one or more processors (320), to generate the modeling of the surgical field, are configured to: modify the safety volume based on the anatomical information.

7. The system (200, 300) of any of claims 1-6, wherein the one or more processors (320), to determine the feedback information (130), are configured to: determine, based on the simulation, effort scores for respective planned operations of the one or more planned operations, wherein the effort scores indicate a level of effort for performing the respective planned operations; and determine an order of the one or more planned operations based on the effort scores.

8. A method, comprising:obtaining, by a device (210), a surgical plan (105) that is associated with a robotically- assisted procedure that is associated with a robot (230), wherein the surgical plan (105) indicates one or more planned operations of one or more robotic arms (235) of the robot (230), and wherein the surgical plan (105) is associated with one or more anatomical elements; obtaining, by the device (210), configuration information (110) associated with the robot; determining, by the device (210) and using medical imaging data (115) of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements; determining, by the device (210), feedback information (130) for the surgical plan (105) based on the configuration information (110) and the anatomical information; and performing, by the device (210) and based on the feedback information (130), one or more actions.

9. The method of claim 8, further comprising: generating, using the configuration information (110) and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure, wherein the feedback information (130) is based on a simulation of the one or more planned operations that uses the modeling.

10. The method of any of claims 8-9, wherein performing the one or more actions comprises: transmitting, to a navigation system (240) of the robot (230), instructions associated with an execution of the one or more planned operations based on the feedback information (130) indicating that the one or more planned operations are achievable.

11. The method of any of claims 8-10, wherein the anatomical information indicates positioning information of one or more sensitive structures of the patient, wherein the feedback information (130) includes proximity information of an implement of the robot (230) for the one or more planned operations, andwherein the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

12. The method of claim 11, wherein the feedback information (130) indicates that the proximity information is associated with a planned operation of the one or more planned operations.

13. An apparatus (210, 300), comprising: means for obtaining (210, 320, 330, 340, 360) a surgical plan (105) that is associated with a robotically-assisted procedure that is associated with a robot (230), wherein the surgical plan (105) indicates one or more planned operations of one or more robotic arms (235) of the robot (230) for the robotically-assisted procedure, and wherein the surgical plan (105) is associated with one or more anatomical elements; means for determining (210, 320, 330), using medical imaging data (115) of a patient of the robotically-assisted procedure, anatomical information of the patient associated with the one or more anatomical elements; means for generating (210, 320, 330), using configuration information (110) associated with the robot (230) and the anatomical information, a modeling of a surgical field associated with the robotically-assisted procedure; means for determining (210, 320, 330), using the modeling, feedback information (130) for the surgical plan (105) based on a simulation of the one or more planned operations; and means for performing (210, 320, 330, 340, 350360), based on the feedback information (130), one or more actions.

14. The apparatus (210, 300) of claim 13, wherein the means for performing the one or more actions comprise: means for providing (210, 320, 330, 350 360), for display, the feedback information (130).

15. The apparatus (210, 300) of any of claims 13-14, wherein the anatomical information indicates positioning information of one or more sensitive structures of the patient,wherein the feedback information (130) includes proximity information of an implement of the robot (230) for the one or more planned operations, and wherein the proximity information indicates a first position of the implement relative to a second position of a sensitive structure, indicated by the positioning information, of the one or more sensitive structures.

Images