System and method for controlling a robotic arm

Abstract

The present invention provides a control method for managing a surgical robot having an arm configured to hold a medical tool. The method includes receiving a set of sensor signals from sensors on the arm, generating a profile based on the set of sensor signals, comparing the profile to at least one signature, determining a state of the arm based on the result of the comparison, and changing the operation of the arm based on the state of the arm. The state of the arm indicates a predetermined condition that has occurred or is highly likely to occur. Changing the operation of the arm prevents the condition from occurring or continuing. The robotic arm can hold a medical tool used during a surgical operation or another medical procedure.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein relate generally to controlling robotic devices, and more particularly to systems and methods for controlling a robotic arm to perform a procedure. Background Art

[0002] Robotic arms have been developed for manufacturing automobiles, fabricating integrated circuit chips, extracting hazardous materials, and performing medical mechanical procedures. In the latter case, it is believed that robotic surgery will prove to be more accurate and able to access many areas of the body that are not easily accessible to human surgeons. Therefore, many institutions have turned their attention to the development of surgical robotic arms.

[0003] However, surgical robotic arms developed to date have many significant deficiencies that make them a safety hazard to patients and therefore virtually unusable. For example, while improved accuracy is a primary goal of robotic surgery, existing robotic arms are inaccurate due to malfunctioning sensors, errors in the control software, vibrations, and other inconsistencies that adversely affect the stability, positioning, and accuracy of movement. These inconsistencies can have catastrophic consequences, especially when the robotic arms are used to perform amputations, brain surgeries, and / or other operations that require delicate levels of precision. Summary of the invention

[0004] According to one or more embodiments, a method for managing a mechanical device includes receiving a set of sensor signals from sensors on an arm, generating a profile based on the set of sensor signals, comparing the profile to at least one signature, determining a state of the arm based on the result of the comparison, and changing the operation of the mechanical arm based on the state of the arm, wherein the state of the arm indicates a predetermined condition that has occurred or is highly likely to occur, and wherein changing the operation of the arm prevents the condition from occurring or continuing. The set of sensor signals may include one or more sensor signals, and the set of sensor signals may be generated by strain gauges or other types of sensors as described herein.

[0005] The set of sensor signals may include one or more strain signals. The set of sensor signals may include one or more accelerometer signals, gyroscope signals, potentiometer signals, or linear variable differential transformer and transformer (LVDT) signals. The at least one signature may include a vibration profile. When the profile matches at least one signature by a predetermined percentage, an operation of changing the arm may be performed. The at least one signature may include at least one margin value, and the state of the arm may correspond to at least one sensor signal in the set of sensor signals having a value within a predetermined range, the value is less than the sensor value corresponding to the condition, and the margin value may correspond to a lower value of the predetermined range.

[0006] The at least one signature may include a value of the first strain signal at a first time, comparing the profile to the at least one signature includes comparing a value of the first strain signal at a second time with a value of the first strain signal at the first time, and determining the state of the arm includes determining that a difference between the value of the first strain signal at the first time and the value of the first strain signal at the second time is greater than a predetermined difference.

[0007] Changing the operation of the arm may include disabling operation of the arm for at least a temporary period of time. The condition may correspond to a force on the arm causing the arm to deviate in an unintended direction. The condition may correspond to a force on the arm caused by a collision or vibration. The condition may correspond to an improper or unsafe locking state of the robotic arm during operation.

[0008] According to one or more other embodiments, a system for managing a surgical robotic device having an arm configured to hold a medical tool includes an interface configured to receive a set of sensor signals; a memory configured to store one or more algorithms; and a processor configured to execute the one or more algorithms to: control an arm including a sensor that generates a set of sensor signals, the one or more algorithms generating a profile based on the set of sensor signals; comparing the profile to at least one signature; determining a state of the arm based on a result of the comparison; and generating a signal to change the operation of the arm based on the state of the robotic arm, wherein the state of the arm indicates a predetermined condition that has occurred or is highly likely to occur, and wherein changing the operation of the arm prevents the condition from occurring or continuing.

[0009] The set of sensor signals may include a set of strain signals. The set of sensor signals may include one or more accelerometer signals, gyroscope signals, potentiometer signals, or linear variable differential transformer and transformer (LVDT) signals. The at least one signature may include a vibration profile. When the profile matches the at least one signature by a predetermined percentage, the operation of changing the arm may be performed.

[0010] The at least one signature may correspond to at least one margin value, the state of the arm may correspond to at least one sensor signal in a set of sensor signals having a value within a predetermined range that is less than a sensor value corresponding to the condition; and the margin value may correspond to a lower value of the predetermined range. The at least one signature may include a value of a first strain signal at a first time; and one or more algorithms may cause the processor to compare the value of the first strain signal at a second time with the value of the first strain signal at the first time and determine that the difference between the value of the first strain signal at the first time and the value of the first strain signal at the second time is greater than a predetermined difference. The one or more algorithms may cause the processor to disable operation of the robotic arm for at least a temporary period of time. The condition may correspond to a force on the arm causing the arm to deviate in an unintended direction. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Additional objects and features of the present invention will become more apparent in the following detailed description and the appended claims taken in conjunction with the accompanying drawings. Although several exemplary embodiments are described, similar reference numerals represent similar components in each of the drawings, wherein:

[0012] Figure 1 An embodiment of a system for controlling a robotic arm is shown;

[0013] Figure 2 An embodiment of attaching a mechanical device including a mechanical arm is shown;

[0014] Figure 3 An embodiment of a method for controlling a robotic arm is shown;

[0015] Figure 4 An example of the arrangement structure of the robot arm sensor is shown;

[0016] Figure 5 An example of sensors arranged on a body part is shown;

[0017] FIG. 6A to FIG. 6C Detection of the state of a robot arm according to one embodiment is shown;

[0018] Figure 7 Detection of the state of a robot arm according to another embodiment is shown;

[0019] Figure 8 Detection of the state of a robot arm according to another embodiment is shown;

[0020] Fig. 9 Another example of the arrangement structure of sensors on the robot arm is shown; and

[0021] Fig.10 Examples showing how operations can be performed using robotic devices and robotic arms. DETAILED DESCRIPTION

[0022] The specification and the accompanying drawings illustrate the principles of various exemplary embodiments. Therefore, it will be understood that those skilled in the art will be able to design various layouts, which, although not explicitly described or shown herein, embody the principles of the present invention and are included in the scope of the present invention. In addition, all examples set forth herein are clearly intended in principle for teaching purposes to assist readers in understanding the principles of the present invention and the concepts proposed by the inventors for promoting the prior art, and should be understood as not being limited to these specifically set forth examples and conditions. In addition, unless otherwise specified (e.g., "or" or "or alternatively"), the term "or" used herein refers to non-exclusive (i.e., and / or). In addition, the various exemplary embodiments described herein are not necessarily mutually exclusive, because some exemplary embodiments can be combined with one or more other exemplary embodiments to form new exemplary embodiments. Descriptions such as "first", "second", "third", etc. are not intended to limit the order of the elements discussed, but are used to distinguish one element from the next element, and are generally interchangeable. Values ​​such as maximum or minimum values ​​can be predetermined and can be set to different values ​​based on the application.

[0023] Figure 1 A system for controlling a holding arm of a tool used to perform various medical procedures is shown. This is accomplished by processing information to determine patterns that may cause anomalies that affect the operation and stability of the holding arm. By identifying these patterns, the system is able to take proactive measures to avoid anomalies before they occur, thereby ensuring patient safety and the likelihood of successful operation. The medical tool held and guided by the arm can be a drill, scalpel, saw, probe, sensor, cautery electrode, catheter, syringe, endoscopy, or another type of tool used to perform a patient procedure.

[0024] refer to Figure 1 , the system includes a processor 10, a memory 20, a database 30, and a display 40, which can also be used as a user interface (IF), for example. The processor 10 executes one or more control algorithms stored in the memory 20 to identify patterns that may cause anomalies related to the robot holding arm 50, as described above. The processor performs these functions based at least in part on information received from one or more sensors mounted on or near the robot holding arm. For example, the information can be received through an interface 65.

[0025] exist Figure 1In the example of , four sensors S1 to S4 are shown, but in another embodiment, a different number of sensors may be included. The processor 10 can generate control signals for selected ones of the sensors, for example, to activate them and / or change their operating mode. The processor 10 can also receive images from at least one camera 80 mounted on the robot holding arm and / or on or near the medical tool 70 held and controlled by the arm. For illustrative purposes, the medical tool will be described as a bone saw for performing a total knee arthroplasty procedure.

[0026] Based on the patterns determined to exist based on the information collected from the sensors and / or cameras, the processor 10 may generate one or more control signals 75 for controlling the operation of the robot holding arm 50. In one embodiment, some of these control signals 85 may be output to the medical tool 70. Using these signals, the processor 10 may take proactive measures to avoid a fault or other anomaly that has at least a predetermined probability of occurrence given the patterns determined to exist. In order to determine these patterns, the database 30 may store statistical and / or other data that can be compared to the sensor signals. As will be explained in more detail below, in one embodiment, the patterns may include signatures from strain gauges, accelerometers, potentiometers in joints, gyroscopes, optical tracking sensors, linear variable differential transformers and converters (LVDTs), load sensors, torque sensors, pressure sensors, and / or other types of sensors that can be used as a basis for predicting abnormal conditions.

[0027] The system for controlling the holding arm may be configured in a variety of ways.

[0028] Figure 2 Shows an example of how the system can be configured. Figure 2 , the system includes a mechanical device 220, which is held in and transported on a cart 210. The cart 210 may be located in a hospital, a clinic, a doctor's office, an outpatient or urgent care center, or another medical facility. In actual applications, the cart can be pushed to the patient's location when an operation is to be performed. The mechanical device 220 may be equipped with a servo motor and / or other mechanical drives and supports for controlling a mechanical device holding arm 50, which supports and controls the movement of a bone saw or other type of medical tool 70. The cart 210 may also be equipped with a device 215 for controlling the operation of the mechanical arm and / or processing signals received from the mechanical arm. In one embodiment, the mechanical device 220 may include a processor 10, a memory 20, and / or a database 30 for executing the algorithm described herein.

[0029] The mechanical device 220 can be carried by the cart on the support 225 until the cart reaches the patient position. At this point, the mechanical device 220 can be mechanically removed from the cart, but the mechanical device can still communicate with the device 215 through one or more wires 235. The patient position can be a bed, an operating table in an operating room, or another patient position. For the purpose of illustration, the patient position can be described as an operating room including an operating table 245. When released, the mechanical device 220 can be placed in a position close to the operating table for operating on the patient. In one embodiment, the mechanical device 220 can be attached to the edge of the operating table by a clamp or other fastener 255.

[0030] Mechanical equipment holding arm control

[0031] Figure 3 An embodiment of a method for controlling the movement and operation of a robot holding arm 50 in order to perform an operation using a bone saw 70 is shown. The method may be implemented at least in part based on one or more control algorithms stored in a system memory 20, which may be included in the robot 220, for example, together with a processor 10 for executing these algorithms. In one embodiment, the control algorithm may be responsive to input received by a graphical user interface of a display. The input may be entered by a technician, a surgeon / doctor, or both, in order to set up the robot and / or move and control the arm and bone saw to perform an operation on a patient. The display may include a touch screen for receiving input. In addition to the control screen for operating the robot arm, the display 40 may output other types of information including the patient's vital signs.

[0032] In one embodiment, the control program used to control the robotic arm is matched to the specific structure and arrangement of sensors on the robotic arm. For example, the robotic arm may include one or more segments and / or pivot points that are controlled to allow movement along three axes to allow for proper positioning of a medical tool (e.g., a bone saw) relative to the body part of a patient being operated on. In one embodiment (e.g., Figure 4 ), the robotic arm 50 has two sections 51 and 52 and four pivot points 53, 54 and 55 for extending and rotating the bone saw 70 in various directions. In another embodiment, the robotic arm 50 may have a different number of sections and / or pivot points.

[0033] For example, depending on the structure of the robot arm, the arrangement structure of the sensors on the robot arm (e.g. Figure 1The sensors may include strain gauges that detect strain at predetermined points along the arm. The strain values ​​detected at these points may be processed by an algorithm to generate a strain signature that may be used as a basis for determining the probability of an abnormality occurring before or while the abnormality is occurring. Actions may then be automatically taken based on the algorithm to preemptively address such abnormalities, thereby protecting the patient and ensuring a successful operation.

[0034] Figure 4 An embodiment of how sensors S1 to S4 may be arranged relative to the robotic arm 50 is shown. In this embodiment, the sensors may be strain gauges. Both strain gauges S1 and S3 may be arranged on opposite sides of the segment 51 of the robotic arm, and another two strain gauges S2 and S4 may be arranged on opposite sides of the segment 52 of the robotic arm. At these locations, the strain gauges (and / or other types of sensors as previously indicated) are able to detect loads, vibrations, and / or other forces that may affect the position, movement, and operation of the medical tool 70. Figure 4 The types, quantities, and arrangements of the sensors in the embodiment are used as examples only. In other embodiments, different types, quantities, and / or arrangements of sensors may be used to determine the forces and loads on the robotic arm.

[0035] refer to Figure 3 , the method includes turning on the power and setting up the system at 1105. This may involve operating room staff transporting the mechanical equipment (e.g., manually, by cart 210, or otherwise) to the location of the patient to be operated on. Depending on the type of surgery to be performed and the medical tools 70 to be used, the patient location may be an operating room including an operating table, a bed, or another location.

[0036] A set of baseline strain values ​​is established for the robotic arm at 1110. For example, the baseline strain values ​​may be acquired when the robotic arm is at rest in order to define the noise floor and initial offset of the strain readings when the system is in an initial configuration.

[0037] At 1115, a homing procedure is performed, which may involve calibrating the robot arm. During the homing procedure, the robot arm may move along each of the three axes, wherein the movement along each axis is performed through a predetermined (e.g., full) range of motion to establish a corresponding end stop position. Measuring strain through this activity may generate a signal that can be used to determine an initial position of a joint (e.g., a pivot point) of the robot arm.

[0038] At 1120, a strain signature of the robotic arm is generated using the strain gauge signals generated during the homing procedure. The strain signature may be generated in a variety of ways. For example, a strain signature may be generated using a transfer function that maps the load of the holding arm to a specific configuration of the horizontal links, such as Figure 4As shown. The specific configuration may include one or more joint angles of the three rotating joints shown in the figure. The transfer function can be estimated analytically or empirically. In addition, strain gauges can be used to check the specific total static load of system components. The combination of sensor values ​​at any given time can represent a strain profile that indicates how the mechanical device is positioned at that moment under assumed static or quasi-static conditions. In one embodiment, the strain signature can be a predefined time variation in the strain profile observed at a reasonable resolution.

[0039] Examples of types of strain signatures that can be generated include point signatures, profile signatures, and template signatures. A point signature may describe an aspect or state of a particular component or element of a system. Point signatures may be calculated analytically or measured empirically, and may consider only a single point in time. For example, if sensor x reads greater than a predetermined value y, then the component to which sensor x is attached is close to yield.

[0040] The profile signature may describe the static pose of the holding arm and mechanical device. In this case, the profile signature may be calculated analytically or empirically based on a set of points. In one embodiment, multiple signals may be used to calculate the profile signature at a single point in time. For example, if the sensor reads values ​​W, X, Y, Z in a static or quasi-static condition, then it may be determined that the horizontal link has configuration C.

[0041] Template signatures can describe system-level and subsystem-level state changes that could lead to a hazardous situation. This type of signature can be measured empirically and can be based on a template that includes a set of profiles. The information in these profiles can be based on any number of sensor signals, and in some cases, not even on any sensor signals, which will provide information about the system (e.g., a certain fault). The number of time samples in the measured time series can be at least T / (1 / f), where T is the time length for generating the template and f is the sampling frequency of the processor. So, for example, if the shape of the curve matches the shape of the measured time series, the template signature will indicate a high probability that the measured event matches a known event established empirically by the template.

[0042] The strain signature can be used to initialize one or more pre-computed models of a mechanical device. For example, the strain signature can be used to calibrate the pre-computed model to determine the loading condition of the arm during operation. Once calibrated, the pre-computed model can be used as a basis for predicting abnormalities or other types of events that occur when operating on a patient. The prediction can involve comparing the occasional strain signal captured by the sensor to the model, as described in more detail below.

[0043] At 1125, the mechanical device 220 is positioned relative to the patient, for example, according to any of the embodiments described herein, for example, by attaching the mechanical device to an operating table or bed. For example, this can be performed by using any of the clamping configurations previously described and adjusting the height and position of the mechanical device to allow the medical tool 70 to be correctly positioned relative to the body part of the patient to be operated on. When the mechanical device 220 is transported in the cart 210, operation 1125 may also include separating the mechanical device 220 from the cart 210. In one embodiment, the workflow operations to be followed when positioning the mechanical device 220 can be controlled by software. In order to guide the technician or surgeon to correctly perform these operations, text and / or graphics corresponding to these operations can be output on the display 40.

[0044] Once the robot is in place and attached to the operating table or bed, a registration procedure may be performed based on the type of surgery that will be performed on the patient. When the surgery involves amputating the patient's foot using a bone saw, the registration procedure may include acquisition and imageless registration of bony landmarks using a tracking array used by the vision system to position the femur, tibia, robot, and bone saw in a global coordinate system.

[0045] In one embodiment, the visual system may include one or more motion capture sensors strategically placed on the patient's body in areas including and adjacent to the amputation site, such as the calf, angle, and foot. The motion capture sensors may include, for example, any combination of photoelectric sensors, angle sensors, infrared sensors, and optical sensors. A plurality of additional sensors (e.g., accelerometers, inertial sensors, magnetic bearing sensors, etc.) may also be placed on the patient's body. Once the sensors are in place, one or more cameras may capture position and orientation data corresponding to the site of the amputation.

[0046] Figure 5 An example of one of these cameras is shown in the form of camera 80, which is also Figure 1 . The field of view of camera 80 includes light emitted or reflected from fiducial markers 1331, 1332 and 1333, which are arranged in and around the global variation amputation of the total knee arthroplasty site (marked by dashed line 1335). Additional cameras can be added, for example, at different angles or fields of view to provide additional data. Once the registration procedure and surgical plan are completed, the surgeon can begin to perform the amputation using a bone saw. In some cases, this may require repositioning the mechanical equipment to establish the first cutting plane.

[0047] At 1130, the method may include capturing a new reference state by the system to establish a new baseline against which future measurements are compared. For example, this operation may be performed at the beginning of a surgical procedure and when the robotic arm is repositioned. Repositioning the arm can subject the arm to different load conditions, as determined by the arrangement of sensors (e.g., strain gauges). As previously described, a new baseline may be performed in a manner similar to establishing an initial baseline. In some embodiments, operation 1130 may be considered optional. In one embodiment, a plurality of reference states may be determined, each of which corresponds to a different stage of an operation to be performed by the robotic arm and tool and / or a different condition that may exist during the operation.

[0048] At 1135, the processor 10 monitors the signals from the sensors S1 to S4, together with the images from the camera, to continuously (or incrementally) generate a real-time strain signature (or profile) for the mechanical arm. The processor 10 may evaluate the strain profile generated by the sensor based on an algorithm stored in a memory 20 on or outside the mechanical device. When generating the strain profile, the processor 10 may compare the profile with the reference state determined in operation 1130. As previously noted, in one embodiment, multiple reference states may be applied. In this case, the processor 10 may compare the profile with one of the reference states corresponding to a stage or condition of the operation that occurs subsequently. If the processor 10 determines based on the comparison that the strain profile has not changed significantly, the method proceeds to the next operation 1140. In one embodiment, the reference state may be different. In one embodiment, the reference state may include at least one reference value (or a set of values) that defines a point at which a fault or other condition may occur or is occurring significantly with a high probability. In one embodiment, the reference state may include at least one range defined by a predetermined maximum value and a predetermined margin value relative to the maximum value. When falling within this range, processor 10 may determine that a fault or other condition may have occurred or is occurring with a high probability.

[0049] In one embodiment, the processor 10 may compare one or more signals from sensors S1 to S4 to one or more previous strain values ​​or profiles. For example, one or more strain signals from sensors S1 to S4 may be compared to the immediately preceding strain value or profile from those sensors or one or more preceding values ​​or profiles generated at another previous time. If the difference between the current strain value or profile and the previous strain value or profile exceeds a predetermined difference, it may be determined that the difference is sufficiently large to warrant a conclusion that a failure or other condition may have occurred or is occurring with a high probability during operation of the robot arm and / or tool.

[0050] In one embodiment, one or more prediction qualities may be obtained by taking time into account. An applicable threshold to be used may be based on reaching a certain value, or the decision to change state may be based on the fact that the trend of the observed strain values ​​is that they will exceed the threshold within the next predefined time window (e.g., less than 10 milliseconds). Thus, if the sensor value has not yet reached the threshold, but the slope of the curve is steep enough that the next one or two data points will exceed the threshold, the system may stop before the threshold is actually reached.

[0051] At 1140, a determination is made as to whether the method should reposition the robotic device, e.g., to position the robotic device for the next step of a procedure. For example, this may be warranted when the current procedure in an amputation process has been completed and the robotic arm is to be moved to position the bone saw in another location to proceed to the next stage of the operation. In another case, the procedure may correspond to a total knee arthroplasty (TKA). In many types of robotic surgical procedures, the robotic arm may not be discretely positioned for a particular cut, but rather moves in a less structured fashion. Therefore, according to one embodiment, a repositioning may be performed.

[0052] The precondition for repositioning the mechanical arm may be to determine that no inappropriate load condition is detected at the time. If the mechanical arm is not to be repositioned, the method returns to operation 1135 where the next generated strain profile is compared to the previous strain profile. The method may be looped for a predetermined time (e.g., less than 2 milliseconds) until the comparison performed at operation 1135 determines that the strain profile has changed significantly or determines that the mechanical device will be repositioned at operation 1140.

[0053] If, based on the comparison at operation 1135, the processor 10 determines that the strain profile has changed significantly or is trending such that the change has been determined to be extremely significant, the method will perform a second check. An alarm may or may not be triggered at this time. For example, when the difference between the current strain profile and the reference state and / or one or more previous strain values ​​or profiles is greater than a predetermined amount, then the change in the strain profile may be considered significant. For example, the type of alarm that is triggered may be determined based on the type of change between the current profile and the reference state / previous value or profile.

[0054] At 1145, the change in the strain profile is compared to a predetermined threshold. If the change is equal to or below the threshold, the method returns to operation 1135 to continue monitoring the change in the strain profile. If the change is above the threshold, the processor 10 may determine that an abnormal condition may occur in the near future. This may involve, for example, the processor 10 analyzing the data trend over time and also triggering when the data is trending toward exceeding a threshold, for example, within less than 10 milliseconds.

[0055] In one embodiment, the operation may involve signature matching. In this case, the embodiments described herein related to comparison with a threshold value may be performed in the same manner, except that the comparison with the threshold value is replaced or includes a comparison with a signature. The signature matching may then involve determining a percentage or score indicating the degree of match of the profile to the signature.

[0056] At 1150, to prevent the abnormality from occurring, the processor may generate a signal to disable the robotic arm 50 and the bone saw 70 and suspend operation. If the robotic arm and the bone saw are not enabled at this time, the processor 10 may prevent the robotic arm and the saw from being placed in an enabled state. Taking these proactive measures before (or just after) the abnormality occurs can protect the patient's safety and avoid injury, which may be serious if the medical tool is a bone saw.

[0057] The critical value or condition may be based on the type of anomaly (or other event) detected that may occur or exist. For example, the critical value may be an instantaneous peak load or matching score generated by a cross-correlation algorithm that involves comparing the transient dynamic strain gauge response to a particular type of event. Anomalies may include, but are not limited to, for example, excessive vibration, accidental collisions with a robotic arm or operating table / bed, and earthquakes. An example of a cross-correlation algorithm is an example of analyzing time series data and comparing different signals for alignment. In one embodiment, the cross-correlation algorithm may be performed based on the following formula:

[0058]

[0059] The above formula involves convolving signal X with the sum of signal Y and normalizing based on the product of the amplitudes of the two signals. The result of the formula is a score for each point in the time series. The value of the score indicates how aligned the two signals are in phase and amplitude. The maximum score is achieved when the two curves are identical and aligned in time offset. For the above normalized formula, the maximum score is 1 and the minimum score is 0.

[0060] The template would be a predefined signal (signal X) that represents a previously characterized event, such as a person colliding with an operating table. A window of data from the sensor is acquired over a period of time that is longer than the template period. The acquired data is signal Y. The cross-correlation algorithm determines when the collision occurred, along with a score indicating how well it matches the template provided. In this case, the threshold would be this score. Once it is determined that the score exceeds a threshold of confidence, a table collision has occurred, and the algorithm can then calculate the time when the table collision occurred.

[0061] Thus, cross-correlation can be understood as a way of comparing two signals over time, rather than just comparing the signals to a threshold at one instant. According to one or more embodiments, a cross-correlation algorithm is applied as a way of examining a strain profile generated based on the signals from the sensors and comparing the strain profile to the signature of a particular type of vibration that may occur, for example, from bumping into a cart, dropping an instrument on a cart, etc.

[0062] At 1155, once it is determined at operation 1140 that the robotic arm will be repositioned, a new reference state is captured at the new location and the processor begins comparing strain signatures / profiles again to detect or predict abnormal conditions. As previously noted, repositioning of the robotic arm may occur, for example, when the current phase of amputation has been completed and a new phase of operation is to be performed at a different location. During the repositioning process, the processor 10 may continue to monitor the load on the arm to ensure that there are no unexpected load conditions.

[0063] At 1160, once the robotic arm and saw have been disabled, the surgeon or technician may take steps to correct one or more factors that caused the strain profile change. This may involve, for example, simply waiting until the vibration caused by the accidental collision stops. In another case, this may involve repositioning the robotic arm and bone saw. In yet another case, the corrective action may involve replacing the saw (if damaged), repairing or replacing the retaining arm (e.g., the mechanical structure that retains the mechanical device) when it is broken, or waiting a period of time until the patient has calmed down or is in a more stable state. In another case, the technician may change the mechanical load on the robotic arm, such as by repositioning the arm, increasing physical interaction with the arm or tool, and / or removing components.

[0064] Detection Example

[0065] The following examples describe practical situations where anomalies or other conditions associated with the operation of the robotic arm and / or medical tool may be expected to occur. For each situation, the system's memory 20 may store a corresponding detection algorithm (or model) that may be used as a basis for analyzing strain profiles generated by sensors arranged on the robotic arm. Each algorithm (or model) may use different maximum limits and margins and / or may perform different forms of analysis, for example, based on the specific anomaly or condition that the algorithm is intended to detect in advance.

[0066] FIG. 6A to FIG. 6CAn example is shown when an excessive load is detected on the robot arm 50. For example, when the robot arm 50 is in a locked state, the excessive load may be caused by excessive downward force applied to the bone saw. For example, the force may be applied based on the control of the mechanical device and / or based on the force applied by the user to the bone saw. Fig. 6A As shown, strain gauges S1 and S3 detect excessive loads in the form of bending rectangles that exceed a predetermined threshold or critical value. The direction of sensitivity of strain gauges S1 and S3 (as well as S2 and S4) can be substantially aligned with the longitudinal axis of the horizontal link member of the retaining arm on which the strain gauges S1 and S3 are mounted. This orientation makes the strain gauges S1 to S4 sensitive to detecting bending forces in a plane that is perpendicular to the ground and is essentially linear. Bending forces can include moments about an axis that is perpendicular to the longitudinal crossbar of the horizontal link and parallel to the ground.

[0067] Figure 6B A cross-sectional view of the mechanical arm 50 is shown at the location of the strain gauges. In this example, the bending moment applies a tensile force F1 to the strain gauge S1 and a compressive force F2 to the strain gauge S3. The strain gauges S1 and S3 generate signals indicative of these forces. These signals are then processed by the processor 10 to generate a changing strain profile (at operation 1135) that exceeds a critical value (at operation 1145).

[0068] In one embodiment, the threshold value may be less than the strain level required to produce the abnormal condition. For example, the threshold value may be just below (e.g., by a predetermined margin) the strain level that would interfere with control of the saw. In another example, the threshold value may be based on a predetermined amount of vibration (e.g., of the holding arm and / or operating table) determined, for example, by signature matching. When the processor 10 (based on signals from strain gauges S1 and S3) generates a strain profile that equals or exceeds the threshold value, proactive measures may be taken to prevent the abnormal condition from occurring, which may include, for example, disabling the robotic arm and bone saw, as previously described with respect to FIG. Figure 3 The method is described in operation 1150.

[0069] Figure 6C 1 is a diagram showing an example of a strain profile generated by the processor 10 based on signals received from the strain gauges S1 to S4. In the diagram, an upper limit 1410 corresponds to a predetermined maximum positive (tensile) strain, and a lower limit 1420 corresponds to a predetermined negative (compressive) strain. These limits correspond to unsafe limits, for example, that could result in a robotic arm failure that could pose a threat to a patient or that could otherwise be considered unacceptable or unsafe.

[0070] The strain profiles include a strain profile 1430 generated based on a signal output from the strain gauge S1, a strain profile 1440 generated based on a signal output from the strain gauge S2, a strain profile 1450 generated based on a signal output from the strain gauge S3, and a strain profile 1460 generated based on a signal output from the strain gauge S4. Because the bending moment on the robot arm 50 mainly affects the strain gauges S1 and S3, the strain profiles 1440 and 1460 corresponding to the strain gauges S2 and S4 are very low, for example, not close to either the positive maximum limit 1410 or the negative maximum limit 1420 throughout the duration of the graph. In contrast, the strain profiles 1430 and 1450 produce very prominent peak load signals (see circles) that fall within predetermined margins 1480 and 1490 of the positive maximum limit and the negative maximum limit, respectively. The predetermined margin may be, for example, 25% of the positive maximum limit or the negative maximum limit.

[0071] In one embodiment, current values ​​from one or more strain gauges may be compared to one or more corresponding previous values ​​from the strain gauges to determine a fault or condition that occurred with the robot arm or tool during operation or some other condition. Figure 6C This is illustrated by the values ​​generated by strain gauge S1 at points P1 and P2 in FIG. In this embodiment, the processor 10 may continuously compare the current value of one or more strain gauges with the previous value, and if the difference between them is equal to or greater than a predetermined difference value Δ, the processor may determine that, for example, there is a high probability that the robot arm may or is failing, or that it is unsafe to continue operating.

[0072] Figure 7 is a graph showing an example of what happens when an impulsive force (of short duration) strikes a robotic arm. Such an impulse may overwhelm or distort the imaging capabilities of the robotic device's vision system. Examples of impulsive forces that may cause these effects include collisions applied to the robotic arm or an operating table. When this occurs, high frequency changes may be produced that are short enough in duration to not immediately trigger the safety interlocks on the robotic arm. Figure 7 , the impulse forces in this example are most evident in the strain profile 1510 corresponding to strain gauge S1 and the strain profile 1520 corresponding to strain gauge S3. The range of the impulse forces is much smaller in the strain profiles 1530 and 1540 of strain gauges S2 and S4, respectively. Because the impulse forces in the strain profiles of strain gauges S1 and S3 cause the signature to deviate significantly (due to the signature matching performed), an abnormal condition can be detected that can cause the processor 10 to suspend the operation of the robotic arm and the bone saw for at least a predetermined time, such as Figure 3In one embodiment, the impulse forces in the strain profiles of strain gauges S1 and S3 exceed predetermined margins 1501 and 1502 (see circled portions) relative to positive maximum strain limits 1580 and negative maximum strain limits 1590. In this example, the predetermined margins may be set to 30% of their respective maximum limits (as indicated by dashed lines).

[0073] by Figure 7 Generating strain profiles in the manner indicated in can also allow for the detection of vibrations, which can produce robot arm deflections at rates exceeding the imaging capabilities of the robot's vision system. In one embodiment, a hardware implementation can be used to generate and detect Figure 7 Such hardware implementations can mitigate the risks associated with software failures that may result from image-based detection.

[0074] Figure 8 1 is a diagram showing an example of detecting when the horizontal link of the robot arm is in a locked or unlocked state. In this figure, a predetermined positive maximum strain limit 1680 and a negative maximum strain limit 1690 are set, and strain profiles generated by the processor 10 for all four strain gauges S1 to S4 are included. The strain profiles 1610 and 1620 of the strain gauges S1 and S3 exceed the corresponding margins 1601 and 1602, which indicates that the robot arm has entered a locked state. For example, a locked robot arm may generate a substantially triangular shape as shown in FIG. Figure 8 Identifiable and detectable strain profiles are shown. In one embodiment, the magnitude of strain profiles 1610 and 1620 can indicate the strength of the lock.

[0075] The strain profiles 1630 and 1640 for strain gauges S2 and S4 show a much smaller response to the locked condition. At a later point in time, the strain profiles for all four strain gauges show a smaller but detectable drop, which presents a signature that identifies when the robotic arm is unlocked. The algorithm used to detect these conditions may also be able to detect wear on the robotic arm and / or medical tool over time. The difference between steady-state high and low strains may eventually increase over time due to part wear. Therefore, the present algorithm can be used to predict failure or the need for preventative maintenance before a failure actually occurs.

[0076] Fig. 9 The arrangement structure of the strain gauges on the robot arm according to another embodiment is shown. In this embodiment, all four strain gauges are symmetrically arranged at predetermined positions of the robot arm 50. For example, the strain gauges can be arranged radially and symmetrically with respect to the cross section of the robot arm, for example, at positions 0°, 90°, 180° and 270°.

[0077] Fig.10An example showing how a robotic device and a robotic arm can be used to perform an operation in a real-world scenario. In this example, it is assumed that the robotic arm is holding a bone saw, but in another embodiment, the arm can hold a different type of tool.

[0078] refer to Fig.10 , the robot 1810 and its accompanying robot arm 1820 can be located in an operating room 1801 including a sterile drape 1802. In this example, the operating room includes an operating table 1803 that supports a patient (as indicated by the mark) to be amputated using a bone saw. The bone saw includes a bone saw controller 1806, which is used to control a saw blade 1807 held by the robot arm. As previously described, the bone saw will amputate near the tibial array 1811 and the femoral array 1812, including infrared or other sensors to facilitate the three-dimensional camera 1821 to capture images. The surgeon and nurse (as indicated by the mark) are also in the room to guide the operation using the robot and assist in various stages of the operation. The circulator and stylus 1809 can be used with a cart 1831 for operating and transporting the robot.

[0079] According to one or more of the foregoing embodiments, a system and method for controlling a robotic arm to perform a predetermined function is provided. In one example, a robotic arm may hold a medical tool for performing a medical procedure on a patient. The medical procedure may be performed for surgical purposes, diagnostic purposes, exploratory purposes, or another purpose, depending on, for example, the type of tool held by the robotic arm. The arm may include one or more sensors to guide a processor (internal or separate from the mechanical device) to control the arm.

[0080] In one or more embodiments, an algorithm stored in a memory can control a processor to detect the state of operation of a robotic arm based on one or more sensor signals. The state of operation is identified by an algorithm to determine the probability of a certain condition that will occur or is occurring, which may indicate a failure or other condition of a mechanical device or a robotic arm, or may otherwise threaten patient safety. For example, the probability may be limited to a predetermined margin range corresponding to an upper and / or lower limit of the condition. For example, the margin range may be within 30% of the upper or lower limit. When it is determined that there is such a probability, the processor generates a signal to quickly disable the robotic arm at least temporarily (e.g., with a response time in the millisecond range) to prevent the condition from occurring or continuing to occur or to take other preventive measures. Examples of factors that may cause the condition to occur include vibration, collision, unexpected, sudden (pulse) or excessive force on the arm, and various defects and conditions associated with the locked and unlocked states of the arm. All of these control functions can be implemented to ensure patient safety as best as possible while achieving improved accuracy and other beneficial effects in performing medical procedures using mechanical devices.

[0081] The methods, processes and / or operations described herein may be performed by code or instructions, which will be performed by a computer, processor, controller or other signal processing device. As previously described in accordance with one or more embodiments, the code or instructions may be stored in a non-transient computer readable medium. Because the algorithm for forming the basis of the method (or the operation of a computer, processor, controller or other signal processing device) is described in detail, the code or instructions for implementing the operation of the method embodiment may convert a computer, processor, controller or other signal processing device into a dedicated processor for performing the method herein.

[0082] The processor, controller, sensor, and other signal generation and signal processing functions of the embodiments disclosed herein may be implemented in logic components (e.g., which may include hardware, software, or both). When implemented at least partially in hardware, the processor, controller, sensor, and other signal generation and signal processing functions of the embodiments may be, for example, any of a variety of integrated circuits, including but not limited to an application specific integrated circuit, a field programmable gate array, a combination of logic gate circuits, a system on a chip, a microprocessor, or another type of processing circuit or control circuit.

[0083] When implemented at least in part in software, the processor, controller, sensor, and other signal generation and signal processing functions of the embodiments may include, for example, a memory or other storage device for storing code or instructions, which are executed by, for example, a computer, processor, microprocessor, controller, or other signal processing device. The computer, processor, microprocessor, controller, or other signal processing device may be those described herein or a device other than the elements described herein. Because the algorithms that form the basis for the methods (or operations of the computers, processors, microprocessors, controllers, or other signal processing devices) are described in detail, the code or instructions for implementing the operations of the method embodiments may convert the computer, processor, controller, or other signal processing device into a dedicated processor for performing the methods described herein.

[0084] Although various exemplary embodiments have been described in detail with particular reference to certain exemplary aspects, it will be appreciated that the invention is capable of other exemplary embodiments and that its details are capable of modification in various obvious respects. It will be apparent to those skilled in the art that variations and modifications may be implemented while remaining within the spirit and scope of the invention. Therefore, the foregoing disclosure, description and drawings are for illustrative purposes only and do not in any way limit the invention, which is limited only by the claims.

Claims

1. A method for managing a surgical robot having an arm configured to hold a medical tool, comprising: receiving a set of sensor signals from sensors on the arm; generating a time-based profile based on the set of sensor signals; comparing the time-based profile to at least one time-based signature; determining a state of the arm based on a result of the comparing; as well as Operation of the mechanical device is changed based on the state of the arm, wherein the state of the arm indicates a predetermined condition that has occurred or is highly likely to occur, and wherein changing the operation of the arm prevents the condition from occurring or continuing.

2. The method according to claim 1, wherein: The set of sensor signals includes one or more strain signals.

3. The method according to claim 1 or claim 2, wherein: The set of sensor signals includes one or more accelerometer signals, gyroscope signals, potentiometer signals, or linear variable differential transformer and transducer (LVDT) signals.

4. A method according to any one of the preceding claims, wherein: The at least one time-based signature comprises a vibration profile.

5. A method according to any one of the preceding claims, wherein: When the time-based profile matches the at least one time-based signature by a predetermined percentage, the operation of changing the arm is performed.

6. A method according to any one of the preceding claims, wherein: The at least one time-based signature includes at least one residual value; the state of the arm corresponds to at least one sensor signal of the set of sensor signals having a value within a predetermined range, the value being less than a sensor value corresponding to a condition; and The margin value corresponds to a lower value of the predetermined range.

7. The method according to claim 6, wherein: The at least one time-based signature comprises a value of a first strain signal at a first time; comparing the time-based profile to the at least one time-based signature comprises comparing a value of the first strain signal at a second time to the value of the first strain signal at the first time; and Determining the state of the arm includes determining that a difference between the value of the first strain signal at the first time and the value of the first strain signal at the second time is greater than a predetermined difference.

8. A method according to any one of the preceding claims, wherein: Changing the operation of the arm includes disabling operation of the arm for at least a temporary period of time.

9. A method according to any one of the preceding claims, wherein: The condition corresponds to a force on the arm that causes the arm to deflect in an unintended direction.

10. The method according to claim 9, wherein: The condition corresponds to a force on the arm caused by a collision or vibration.

11. A method according to any one of the preceding claims, wherein: The condition corresponds to an improper or unsafe locking state of the robot arm during operation.

12. A system for managing a surgical robot having an arm configured to hold a medical tool, comprising: an interface configured to receive a set of sensor signals; a memory configured to store one or more algorithms; as well as A processor, the processor being configured to: execute the one or more algorithms to control the arm including the sensor that generates the set of sensor signals, execute the one or more algorithms to: generate a time-based profile based on the set of sensor signals; compare the time-based profile with at least one time-based signature; determine a state of the arm based on a result of the comparison; and generate a signal to change the operation of the mechanical device based on the state of the mechanical arm, wherein the state of the arm indicates a predetermined condition that has occurred or is highly likely to occur, and wherein changing the operation of the arm prevents the condition from occurring or continuing.

13. The system according to claim 12, wherein: The set of sensor signals includes a set of strain signals.

14. A system according to claim 12 or claim 13, wherein: The set of sensor signals includes one or more accelerometer signals, gyroscope signals, potentiometer signals, or linear variable differential transformer and transducer (LVDT) signals.

15. A system according to any one of claims 12 to 14, wherein: The at least one time-based signature comprises a vibration profile.

16. A system according to any one of claims 12 to 15, wherein: When the time-based profile matches the at least one time-based signature by a predetermined percentage, the operation of changing the arm is performed.

17. A system according to any one of claims 12 to 16, wherein: The at least one time-based signature corresponds to at least one residual value; the state of the arm corresponds to at least one sensor signal of the set of sensor signals having a value within a predetermined range, the value being less than a sensor value corresponding to the condition; and The margin value corresponds to a lower value of the predetermined range.

18. A system according to any one of claims 12 to 17, wherein: The at least one time-based signature includes a value of a first strain signal at a first time; and The one or more algorithms will cause the processor to compare the value of the first strain signal at the second time with the value of the first strain signal at the first time, and determine that the difference between the value of the first strain signal at the first time and the value of the first strain signal at the second time is greater than a predetermined difference.

19. A system according to any one of claims 12 to 18, wherein: The one or more algorithms will cause the processor to disable operation of the robotic arm for at least a temporary period of time.

20. The system according to any one of claims 12 to 19, wherein: The condition corresponds to a force on the arm that causes the arm to deflect in an unintended direction.

Images