Robotic Imaging System with Force-Based Collision Avoidance Mode - Patent application

Abstract

The robotic imaging system includes a camera configured to obtain one or more images of a target site. A robotic arm is operatively connected to the camera, the robotic arm adapted to selectively move the camera in a movement sequence. A force-based sensor is configured to detect and transmit sensor data related to at least one of a force and / or a torque applied by a user to move the camera. The system includes a controller configured to receive the sensor data. The controller has a processor and a tangible, non-transitory memory in which instructions are recorded. The controller is adapted to selectively execute a collision avoidance mode including applying a respective corrective force to modify the movement sequence if the camera and / or the robotic arm enter a predefined buffer zone.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 315,129, filed March 1, 2022, which is incorporated by reference in its entirety.

[0002] The present disclosure relates generally to robotic imaging systems. More specifically, the present disclosure relates to collision avoidance modes in robotic imaging systems. Various imaging modalities are commonly used to image different parts of the human body. Robotic systems have been developed to improve the efficiency of medical procedures using these imaging modalities. Robotic systems may incorporate multiple parts or components that assist a user in operating the system. This can make it difficult to avoid self-collisions of multiple components, such as collisions between one robot link and another, collisions between a camera unit and a robot arm and / or a storage unit, and collisions between a camera unit and an image plane. Summary of the Invention [Means for solving the problem]

[0003] Disclosed herein is a robotic imaging system having a camera configured to obtain one or more images of a target site. A robotic arm is operatively connected to the camera, the robotic arm adapted to selectively move the camera in a movement sequence. A force-based sensor is configured to detect and transmit sensor data related to at least one of a force and / or a torque applied by a user to move the camera. The system includes a controller configured to receive the sensor data. The controller has a processor and a tangible, non-transitory memory in which instructions are recorded. The controller is adapted to selectively execute a collision avoidance mode including applying a respective corrective force to modify the movement sequence if at least one of the camera and the robotic arm enter a predefined buffer zone.

[0004] The camera may be a stereoscopic camera configured to record left and right images to generate at least one stereoscopic image of the target site. The force-based sensor may include a six-degree-of-freedom haptic device. The controller is adapted to calculate the respective corrective forces using a closed-loop control module including at least one of a proportional integral controller, a proportional derivative controller, and a proportional integral derivative controller. In some embodiments, the predetermined buffer zone is within a delta value of at least one no-go zone, and application of the respective corrective forces pushes at least one of the camera and the robot arm away from the at least one no-go zone.

[0005] The controller can be adapted to calculate respective correction forces for one or more checkpoints, each having a respective position and a respective velocity along the first direction. The robotic imaging system can include a head unit for housing the camera and a coupling plate for mechanically coupling the head unit to the robotic arm, the head unit operably connected to the cart. The checkpoints can be located on at least one of the head unit, the coupling plate and the robotic arm. The controller can use a proportional-derivative controller to:

number

number

[0006] The controller may be adapted to perform a joint limit calculation, and the respective corrective forces include force contributions from the joint limit calculation. Performing the joint limit calculation may include identifying a respective torque for at least one joint via a closed-loop module, and converting the respective torques to respective tool forces using a transpose of a Jacobian matrix. The force contributions are based on a sum of the respective tool forces for each of the one or more joints.

[0007] The robotic imaging system may include a cart operatively connected to the robot arm. The respective corrective forces may include a force contribution from a cart limit calculation. The controller is adapted to perform a cart limit calculation including calculating respective spherical distances and respective velocities of one or more checkpoints from a surface of the cart. The controller is adapted to obtain a radial force if the respective spherical distances are less than a predetermined threshold. The force contribution may be obtained as the radial force multiplied by a fraction. The surface of the cart may be modeled as a sphere, and the respective spherical distances (r) may be:

number

number

number

number

number

number

number

[0008] In some embodiments, the respective corrective forces may include a force contribution from an interface limit calculation. The controller is adapted to perform an interface limit calculation including obtaining at least one interface perpendicular to a first direction and calculating respective distances and respective velocities of one or more check points from the at least one interface along the first direction. The controller is adapted to obtain respective force components when the respective distances are less than a predetermined threshold. The force contribution is obtained as a respective sum of the respective force components. The interface may be proximate to the patient.

[0009] The controller may be adapted to calculate the respective force components based in part on the respective buffer distances to the at least one no-entry zone, the respective distances from the at least one boundary surface, and the respective velocities. i )teeth,

number

number

[0010] The above and other features and advantages of the present disclosure will become readily apparent from the following detailed description of the best modes for carrying out the disclosure when considered in conjunction with the accompanying drawings. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a partial schematic diagram of a robotic imaging system having a camera and a controller with a collision avoidance mode. [Diagram 2] 4 is a flow chart of an exemplary method for operating a collision avoidance mode. [Diagram 3] FIG. 2 is a partial plan view schematic of the robotic imaging system of FIG. 1 showing one or more checkpoints. [Figure 4] FIG. 13 is a partial schematic diagram showing buffer distances from no-entry zones for checkpoints. [Diagram 5] 3 is a flow chart of a portion of the method of FIG. 2 illustrating an exemplary cart limit calculation. [Figure 6] 6 is a schematic graph showing a model of the cart surface that can be employed in the cart limit calculation of FIG. 5. [Figure 7] 3 is a flow chart of a portion of the method of FIG. 2 illustrating an exemplary interface limit calculation. [Figure 8] 3 is a flow chart of a portion of the method of FIG. 2 illustrating an example joint limit calculation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Exemplary embodiments of the present disclosure are shown by way of non-limiting examples in the drawings and are described in more detail below. It should be understood, however, that the novel aspects of the present disclosure are not limited to the specific forms shown in the drawings listed above. Rather, the present disclosure is intended to encompass modifications, equivalents, combinations, subcombinations, permutations, groupings, and alternatives that are included within the scope of the present disclosure, such as those encompassed by the appended claims.

[0013] Referring to the drawings, in which like reference numbers refer to like components, FIG. 1 illustrates a schematic of a robotic imaging system 10 having a camera 12 with a collision avoidance mode 14. The robotic imaging system 10 is configured to image a target site 16. The camera 12 illustrated in FIG. 1 is a stereoscopic camera 12, although it is understood that other types of cameras (e.g., cameras that capture a single image) may be used. With reference to FIG. 1, the camera 12 is located at least partially within a head unit 18 of a housing assembly 20, which is configured to be at least partially directed toward the target site 16. The camera 12 is configured to record first and second images of the target site 16, which may be used to generate a live two-dimensional stereoscopic view of the target site 16. The target site 16 may be an anatomical location of a patient, a biological specimen in a laboratory, a calibration slide / template, or the like.

[0014] 1, a robotic imaging system 10 includes a controller C having at least one processor P and at least one memory M (or non-transitory tangible computer-readable storage medium) having instructions recorded therein for executing a method 100 (described below with respect to FIG. 2) for operating a collision avoidance mode 14. The memory M can store a controller-executable instruction set, and the processor P can execute the controller-executable instruction set stored in the memory M.

[0015] 1, at least one input device 22 (hereafter omitted "at least one") is operatively connected to the camera 12 (e.g., at the head unit 18) to allow a user to manually position the camera 12. The input device 22 may include respective controls for enabling or selecting certain features of the camera 12, such as adjusting focus, magnification, amount / type of light projected onto the target site 16, and other features. It is understood that the number and form of the input devices 22 may vary, for example, the input devices 22 may include a joystick, a wheel, a mouse, or a touch screen device. In some embodiments, the input devices 22 may be controlled by a remote control 66 (see FIG. 1).

[0016] With reference to FIG. 1, the robotic imaging system 10 includes a robotic arm 24 operably connected to a head unit 18 and configured to selectively move the head unit 18. The head unit 18 may be mechanically coupled to the robotic arm 24 via a coupling plate 26. A user may position and orient the camera 12 with assistance from the robotic arm 24. As shown in FIG. 1, a force-based sensor 28 is operably connected to the robotic arm 24 and / or the coupling plate 26. The force-based sensor 28 is configured to detect forces and / or torques applied by a user to move the camera 12. The force-based sensor 28 is adapted to detect a movement or force of the user relative to the camera 12 and convert the detected force / movement into rotational and / or translational data.

[0017] The robot arm 24 includes one or more joints, e.g., a first joint 30 and a second joint 32, configured to provide an additional degree of positioning and / or orientation of the head unit 18. Data from the force-based sensor 28 may be used to identify which joint of the robot arm 24 should rotate and how quickly the joint should rotate to provide assisted movement of the camera 12. Referring to FIG. 1 , a corresponding joint motor (e.g., joint motor 31) and a respective joint sensor (e.g., joint sensor 33) may be coupled to each joint. The joint motor 31 is configured to rotate the first joint 30 about an axis, while the joint sensor 33 is configured to transmit the position of the first joint 30 (in 3D space).

[0018] The collision avoidance mode 14 is force based and acts on acceleration dynamics to avoid self-collisions of various components, such as collisions between the camera 12 and the robotic arm 24, between the camera 12 and a storage unit (e.g., cart 34), and between the first joint 30 and the second joint 32. The collision avoidance mode 14 described herein provides a smooth transition and is part of the control system dynamics.

[0019] The robotic arm 24 includes an assisted drive function 50 that incorporates a user-guided control system. In one embodiment, a user may hold the input device 22 and actuate or press a release button. Upon actuating the release button, the camera 12 sends a message to the controller C indicating that the user wishes to initiate the assisted drive function 50. The controller C configures the robotic arm 24 and / or coupling plate 26 to allow the user to gently steer the camera 12 in a desired direction. During this movement, the controller C causes the robotic arm 24 and / or coupling plate 26 to move the camera 12 in a "power steering" manner, safely supporting its weight, and automatically identifying which joints should be actuated and which joints should be braked in a coordinated manner to achieve the user's desired movement.

[0020] In some embodiments, the robotic arm 24 may allow for unassisted or at least initially unassisted movement by the user. In these examples, the force-based sensor 28 detects a motion imparted by the user, which is used by the robotic imaging system 10 to subsequently rotate one or more joints, thereby providing assisted movement. The time from the initial detection of the motion or force causing the motion to the robotic imaging system 10 briefly rotating the joints may be, for example, less than 200 milliseconds or as little as 10 milliseconds, such that the user is unaware of the initial unassisted movement time of the robotic arm 24.

[0021] The force-based sensor 28 may include a six degree of freedom haptic module. In these embodiments, the force-based sensor 28 may detect translational forces or motions in the X, Y, and Z axes, and rotational forces or motions about the yaw, pitch, and roll axes separately. Decoupling of translational and rotational forces may enable the robotic imaging system 10 to calculate forward and / or inverse kinematics for control of the robotic arm 24. The force-based sensor 28 may include optical sensors (e.g., force / torque sensors) that allow the robotic arm 24 to electromechanically respond to a user's gentle pressure on the camera 12. The optical sensor may include electro-optical devices configured to convert applied forces and / or torques into electrical signals, thereby allowing the desired force / torque input by the user to be sensed and translated into motion requests provided in the sensed linear and / or rotational directions. It is understood that other types of sensor technologies may be used. For example, the force-based sensor 28 may include strain gauges or piezoelectric devices configured to sense haptic signals from the user.

[0022] The location of the force-based sensor 28 may be varied based on the application at hand, for example, at the interface between the coupling plate 26 and the camera 12. Additional types of sensors 29 may be deployed.

[0023] With reference to FIG. 1 , the robot arm 24 (and / or the coupling plate 26) may be controlled via an embedded processor, such as controller C and / or robot arm controller 42. The robot arm 24 may be selectively operable to extend the field of view of the camera 12 along the X-axis, the Y-axis, and the Z-axis. The robot arm controller 42 may include a processor, server, microcontroller, workstation, etc., configured to convert one or more messages or commands from the controller C into messages and / or signals that rotate any one of the joints. The robot arm controller 42 is also configured to receive sensor information, such as joint position and / or velocity, from the robot arm 24 and / or the coupling plate 26 and convert it into one or more messages to the controller C. A stereoscopic visualization camera is described in U.S. Patent Application Publication No. 16 / 398,014 (filed April 29, 2019), the contents of which are incorporated herein by reference in their entirety, along with an integrated robotic platform.

[0024] The head unit 18 may be connected to a cart 34 having at least one display medium (which may be a monitor, a terminal, or other form of two-dimensional visualization), such as the first and second displays 36 and 38 shown in FIG. 1. Referring to FIG. 1, the controller C may be configured to process signals for broadcast on the first and second displays 36 and 38. The housing assembly 20 may be completely self-contained and movable between various locations. The image streams from the cameras 12 may be sent to the controller C and / or a camera processor (not shown), which may be configured to prepare the image streams for viewing. For example, the controller C may combine or interleave the first and second video signals from the cameras 12 to create a stereoscopic signal. The controller C may be configured to store the video and / or stereoscopic video signals in a video file and stored in the memory M. The first and second displays 36 and 38 may incorporate a stereoscopic display system, where the two-dimensional display has separate images for the left and right eyes. To view a stereoscopic display, a user may wear special glasses that operate in conjunction with the first and second displays 36, 38 to present a left field of view to the user's left eye and a right field of view to the user's right eye.

[0025] 1, the first display 36 may be connected to the cart 34 via a flexible mechanical arm 40 having one or more joints to allow for flexible positioning. The flexible mechanical arm 40 may be configured to have a length sufficient to extend over the patient to provide a relatively close view to the surgeon during surgery. The first and second displays 36 and 38 may include any type of display, such as a high definition television, an ultra high definition television, smart glasses, a projector, one or more computer screens, a laptop computer, a tablet computer, and / or a smartphone, and may include a touch screen.

[0026] The camera 12 is configured to obtain images of the target region 16 that may be presented in a variety of forms, including, but not limited to, captured still images, real-time images, and / or digital video signals. As used herein, "real-time" generally refers to updating information at the same rate as the data is received. More specifically, "real-time" means that image data is acquired, processed, and transmitted at a sufficiently high data rate and sufficiently low latency that the object moves smoothly without jerks or latency that are perceived by the user when the data is displayed. Typically, real-time occurs when new images are acquired, processed, and transmitted at a rate of at least about 30 frames per second (fps) and displayed at about 60 fps, and when the combined processing of the video signals has a latency of about 1 / 30th of a second or less.

[0027] The controller C is adapted to calculate respective correction forces for one or more check points 200 located on the head unit 18, the robot arm 24, and elsewhere, as shown in FIG. 3. The check points 200 each have a respective position and a respective velocity, both along a first direction (e.g., the x-direction in FIG. 3). The respective positions and velocities of the check points 200 can be calculated relative to a fixed coordinate frame (e.g., either the camera frame or the robot base frame). The fixed coordinate frame can be selected or changed based on the application at hand. With reference to FIG. 3, the check points 200 can include multiple points on the input device 22, the head unit 18, the coupling plate 26, and the robot arm 24. With reference to FIG. 3, the input device 22 can include a first handle 202 and a second handle 204. In some embodiments, the input device 22 includes multiple check points, such as a first handle apex 210, a first handle bottom point 214, a second handle apex 212, and a second handle bottom point 216. 3 , the checkpoints 200 may include respective points on the head unit 18, such as a camera proximal point 218, a camera distal point 220, a first side camera midpoint 222, and a second side camera midpoint 224. The set of checkpoints 200 may further include a tool center point 226 and a joint extrusion point 228 at a joint 230 of the robot arm 24.

[0028] The collision avoidance mode 14 of FIG. 1 is activated when the camera 12 and / or the robot arm 24 enter a predefined area, such as the predefined buffer zone 250 shown in FIG. 4. The predefined buffer zone 250 is shown relative to a reference axis 252 extending from an origin O. The predefined buffer zone 250 is within a delta value 254 of at least one "no entry" area, such as the no entry zone 256. Application of a corrective force pushes the camera 12 and / or the robot arm 24 away from the no entry zone 256 that would otherwise cause a collision. As an example, the first reference point 258 of FIG. 4 is not within the predefined buffer zone 250 and therefore no corrective force is applied. However, the second reference point 260 is within the predefined buffer zone 250 (within a buffer distance 262) and a corrective force is applied.

[0029] The corrective forces may be calculated using feedback control by a closed loop control module, which may be a proportional integral controller, a proportional derivative controller, or a proportional integral derivative controller. For example, using a proportional derivative controller, the respective corrective forces for check point 200 may be calculated as follows:

number

number

[0030] Referring now to Figure 2, a flow chart of an exemplary method 100 for operating the collision avoidance mode 14 of Figure 1 is shown. The method 100 may be embodied as computer readable code or instructions stored in and partially executable by the controller C of Figure 1. The method 100 need not be applied in the particular order described herein, but may be performed dynamically. Furthermore, it should be understood that some steps may be omitted. The method 100 may be performed periodically or at predetermined time intervals.

[0031] 2, where the controller C is programmed to receive input data, such as joint position data of the robot arm 24 and sensor data from the force-based sensors 28 related to forces and / or torques applied by a user to the camera 12. The sensor data includes an input force vector representing the forces applied by the user and / or an input torque vector representing the torque applied by the user. In accordance with block 102, the method 100 may also include filtering the sensor data, for example with a first low pass filter, a second low pass filter and / or a notch filter targeted at cart vibrations. In some embodiments, no filtering is applied.

[0032] 2, the controller C is programmed to calculate gravity and apply gravity compensation by removing the effects of the Earth's gravity from the sensor data. Gravity calculation software available to those skilled in the art may be employed.

[0033] Proceeding to block 106 in Figure 2, the controller C is programmed to perform a number of spatial limit or boundary calculations. The spatial limits may be defined in joint space, Cartesian space or another suitable coordinate frame. Exemplary implementations of the cart limit calculation 300, the interface surface limit calculation 400 and the joint limit calculation 500 are described below with respect to Figures 5-8. It will be appreciated that other limit calculations may be employed.

[0034] Cart Limit Calculator 5, a flow chart of an exemplary implementation of the cart limit calculation 300 is shown. Pursuant to block 302 of FIG. 5, the controller C is programmed to obtain the respective positions (and velocities) of the checkpoints 200, for example, relative to the origin O of FIG. 4. Proceeding to block 304 of FIG. 5, the controller C is programmed to calculate the respective spherical distances and respective velocities for the checkpoints 200 from the surface of the cart 34. The surface of the cart 34 may be modeled as a sphere. FIG. 6 is a schematic graph illustrating the modeling of the cart surface, where a circumscribing sphere 350 and a circumscribing rectangle 352 are used to bound the dimensions of the cart. The respective spherical distances (r) are given by:

number

number

number

number

number

[0035] Proceeding to block 306 of FIG. 5, the controller C controls r <r TIt is programmed to obtain the radial force when the radial force (F R ) are the respective spherical distances (r) and respective velocities

number

number

number

[0036] Proceeding to block 308 of FIG. 5, controller C is programmed to calculate the force contribution to the corrective force. The force contribution is calculated based on the radial force (F R ) multiplied by a fraction (f). In some embodiments, the fraction is

number

[0037] Boundary limit calculation 7, a flow chart of an exemplary implementation of the interface limit calculation 400 is shown. In accordance with block 402 of FIG. 7, the controller C is programmed to obtain at least one interface 52 (see FIG. 1) perpendicular to a first direction (e.g., Z-axis in FIG. 1). The interface 52 may be in close proximity to the patient, for example within a clearance distance. Proceeding to block 404 of FIG. 7, the controller C calculates the respective distances (z) and respective velocities along the first direction (e.g., z-direction in FIG. 1) from the interface 52 to the checkpoint 200.

number

[0038] Proceeding to block 406 of FIG. 7, the controller C is programmed to calculate the respective force components when the respective distances (z) are below a predetermined threshold (b). i )teeth,

number

number

[0039] Joint Limit Calculation Referring to Figure 8, a flow chart of an exemplary implementation of a joint limit calculation 500 is shown. In accordance with block 502 of Figure 8, the controller C is programmed to identify a respective torque for each of the joints in the robot arm 24 (such as the first joint 30 and the second joint 32 shown in Figure 1). The respective torques (τi )teeth,

number

number

[0040] Proceeding to block 504 of FIG. 8, the controller C calculates the τ=J(q) using the transpose of the Jacobian matrix (e.g., τ=J(q) T The controller C is programmed to convert the respective torques (τ) for each joint into respective tool forces (F') using the Jacobian transpose. After the tool forces are calculated using the transpose of the Jacobian to obtain the joint torques, tuning constants are obtained for each joint. The tuning constants can be set as either 0 or 1 and are used to selectively enable or disable individual axes. Proceeding to block 506 of FIG. 8, the controller C can obtain the force contribution as the sum of the respective tool forces (ΣF') for each of the joints (modulated by the tuning constants).

[0041] 2, the method 100 proceeds to block 108, where the controller C is programmed to obtain a correction force as a sum of the respective force contributions from the multiple limit calculations (obtained in block 106). The correction force is applied to the movement sequence to modify the movement sequence if the camera 12 and / or the robot arm 24 enter a predefined buffer zone 250 (see FIG. 4). The movement sequence is obtained using kinematics, e.g., inverse kinematics and / or Jacobian kinematics (e.g., the inverse of the Jacobian matrix). The controller C specifies a movement sequence that specifies how certain joints of the robot arm and / or the coupling plate 26 should be moved in a coordinated manner, e.g., specifying joint rotation speed, joint rotation direction and / or joint rotation duration. The movement sequence may also specify the sequence in which the joints of the robot arm 24 and / or the coupling plate 26 are rotated. Any of the joints of the robot arm 24 and / or the coupling plate 26 may rotate individually or move overlappingly according to the movement sequence. The Jacobian kinematics equations define how specific joints of the robot arm 24 and / or coupling plate 26 should move based on scaled translation / rotation vectors. Jacobian kinematics provides velocity control, while inverse kinematics provides position control. Other robot arm control routines may be used.

[0042] Proceeding to block 110 in FIG. 2, the method 100 includes transforming coordinates from the sensor (force / torque) frame to the robot base frame, which is the coordinate axis on the cart 34 to which the base of the robot arm 24 is mounted. The controller C is programmed to use the joint position data along with the compensated and filtered force / torque output data to perform this coordinate transformation. The transformation may include one or more predefined equations or relationships based on the position and orientation of the force-based sensor 28. The transformation may be based on parameters obtained through calibration. In an embodiment where the assisted drive function 50 operates in the (input) camera frame, the joints, boundaries and cart limits may be analytically calculated in the robot base frame and transformed to the camera frame before the signals are sent back to the assisted drive function 50.

[0043] Proceeding to block 112 of FIG. 2, the controller C is programmed to enable commands to ensure that the commands (or signals indicative of commands) are within the operating parameters of the joint motors (e.g., duration, rotational speed, etc.). The controller C and / or the robot arm controller 42 may also enable commands by comparing the commands to current thresholds to ensure that the robot arm 24 does not consume excessive current at any stage of the movement sequence. In accordance with block 112 of FIG. 2, the controller C is also programmed to send commands by one or more signals to the appropriate joint motors of the robot arm 24 and / or coupling plate 26 in accordance with the movement sequence. The sent commands cause the motors at the respective joints to move the robot arm 24 and / or coupling plate 26, thereby moving the camera 12 while avoiding collisions.

[0044] The controller C of Figure 1 may include or otherwise access information downloaded from remote sources and / or executable programs. With reference to Figure 1, the controller C may be configured to communicate with a remote server 60 and / or a cloud unit 62 via a network 64. The remote server 60 may be a private or public information source maintained by an organization such as, for example, a research institute, a company, a university, and / or a hospital. The cloud unit 62 may include one or more servers hosted on the Internet to store, manage, and process data.

[0045] Network 64 may be a serial communication bus in the form of a local area network. Local area networks may include, but are not limited to, Controller Area Network (CAN), Controller Area Network with Flexible Data Rate (CAN-FD), Ethernet, Bluetooth, WIFI, and other data formats. Network 64 may be a wireless local area network (LAN) that links multiple devices in a wirelessly distributed manner, a wireless metropolitan area network (MAN) that connects multiple wireless LANs, or a wireless wide area network (WAN) that covers a large area such as a nearby city or town. Other types of connections may be employed.

[0046] The controller C of FIG. 1 may be an integral part of the robotic imaging system 10 or may be a separate module operably connected to the robotic imaging system 10. The controller C includes computer-readable media (also referred to as processor-readable media), including non-transitory (e.g., tangible) media involved in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media may take many forms, including, but not limited to, non-volatile media and volatile media. Non-volatile media may include, for example, optical or magnetic disks and other persistent memories. Volatile media may include, for example, dynamic random access memory (DRAM), which may constitute a main memory. Such instructions may be transmitted by one or more transmission media, including coaxial cables, copper wires, and optical fibers, including the wiring that comprises a system bus coupled to the processor of the computer. Some forms of computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, other magnetic media, CD-ROM, DVD, other optical media, punch cards, paper tape, other physical media with patterns of holes, RAM, PROM, EPROM, Flash EEPROM, other memory chips or cartridges, or other computer readable media.

[0047] The lookup tables, databases, data repositories, or other data stores described herein may include various types of mechanisms for storing, accessing, and retrieving various types of data, including a hierarchical database, a set of files in a file system, a proprietary application database, a relational database management system (RDBMS), and the like. Each such data store may be contained within a computing device employing a computer operating system as described above, and may be accessed over a network in one or more of a variety of ways. The file system may be accessible from the computer operating system and may include files stored in various formats. The RDBMS may employ a structured query language (SQL) in addition to a language for creating, saving, editing, and executing stored procedures, such as the PL / SQL language described above.

[0048] The flowcharts presented herein illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of code, and includes one or more executable instructions for implementing the specified logical functions. It should also be noted that each block of the block diagrams and / or example flowcharts, and combinations of blocks in the block diagrams and / or example flowcharts, may be implemented by a special purpose hardware-based device or a combination of special purpose hardware and computer instructions that perform the specified functions or acts. These computer program instructions may also be stored on a computer-readable medium. The computer program instructions, including instructions for implementing the functions / acts specified in the blocks of the flowcharts and / or block diagrams, may instruct a controller or other programmable data processing apparatus to function in a specific manner to produce an article of manufacture.

[0049] Numerical values ​​of parameters (e.g., amounts or conditions) in this specification, including the appended claims, are to be understood as being modified by "about" in each and every instance, regardless of whether the term "about" is actually placed before the numerical value. "About" indicates that the stated numerical value allows for some slight imprecision (in some approaches, it is the exact value; roughly or reasonably close; approximately). When the imprecision provided by "about" is not understood in this ordinary sense in the art, "about" as used herein at least indicates the variation that may result from the ordinary method of measuring and using such parameters. In addition, the disclosure of a range includes the disclosure of each value and each sub-range within the entire range. Each value within a range and the endpoints of the range are disclosed herein as separate embodiments.

[0050] The detailed description and drawings or figures support and explain the disclosure, but the scope of the disclosure is defined only by the claims. Although the best mode and some of the other embodiments for carrying out the claimed disclosure have been described in detail, there are various alternative designs and embodiments for carrying out the disclosure as defined in the appended claims. Furthermore, the features of the embodiments shown in the drawings or the various embodiments mentioned in this specification should not necessarily be understood as independent embodiments from each other. Rather, each of the characteristics described in one of the example embodiments can be combined with one or more other desirable characteristics from other embodiments, resulting in other embodiments not described in words or by reference to the drawings. Therefore, such other embodiments are included within the framework of the appended claims.

Claims

1. 1. A robotic imaging system, comprising: a camera configured to record one or more images of the target site; a robotic arm operatively connected to the camera, the robotic arm adapted to selectively move the camera in a movement sequence, the robotic arm including one or more joints; a force-based sensor configured to detect and transmit sensor data related to at least one of a force and / or a torque applied by a user to move the camera; a controller configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory in which instructions are recorded; wherein the controller is adapted to selectively execute a collision avoidance mode including applying respective corrective forces to modify the movement sequence if at least one of the camera and the robot arm enter a predetermined buffer zone.

2. The robotic imaging system of claim 1 , wherein the camera is a stereoscopic camera configured to record left and right images to generate at least one stereoscopic image of the target area.

3. The robotic imaging system of claim 1 , wherein the force-based sensor comprises a six-degree-of-freedom haptic device.

4. The robotic imaging system of claim 1 , wherein the controller is adapted to calculate the respective corrective forces using a closed loop control module including at least one of a proportional integral controller, a proportional derivative controller, and a proportional integral derivative controller.

5. the predetermined buffer zone being within a delta value of at least one no-go zone, and the application of the respective corrective forces pushing at least one of the camera and the robot arm away from the at least one no-go zone; The robotic imaging system of claim 1 , wherein the controller is adapted to calculate the respective corrective forces for one or more checkpoints, each checkpoint having a respective position and a respective velocity along a first direction.

6. a head unit for housing the camera; a coupling plate that mechanically couples the head unit to the robot arm, the head unit being operably connected to a cart; 6. The robotic imaging system of claim 5, further comprising: wherein the one or more checkpoints are located on at least one of the head unit, the binding plate, and the robot arm.

7. The controller uses a proportional-derivative controller to [0010] where F represents the respective correction force, b represents a respective buffer distance to the at least one no-entry zone, and x and [0025] represents the respective positions and respective velocities of the one or more checkpoints, and K ｐ and K. ｄ 6. The robotic imaging system of claim 5, wherein: y represents an adjustable gain coefficient for each of the proportional-derivative controllers.

8. The robotic imaging system of claim 1 , wherein the controller is adapted to perform a joint limit calculation, and the respective corrective forces include force contributions from the joint limit calculation.

9. performing the joint limit calculations determining, via a closed-loop module, a respective torque for at least one joint; converting the respective torques into respective tool forces using a transpose of a Jacobian matrix, the force contributions being based on a sum of the respective tool forces for each of the one or more joints; The robotic imaging system of claim 8 .

10. a cart operatively connected to the robotic arm; each said corrective force including a force contribution from a cart limit calculation; The controller is adapted to perform the cart limit calculation, the cart limit calculation comprising: calculating a respective spherical distance and respective velocity of one or more checkpoints from a surface of the cart; obtaining a radial force when said respective spherical distance is less than a predetermined threshold value; 2. The robot imaging system of claim 1, wherein the force contribution is obtained as the radial force multiplied by a fraction.

11. The surfaces of the cart are modeled as spheres, and the respective spherical distances (r) are: [0030] where (w ０ , w １ , w ２ ) represents the coordinates of each of the one or more checkpoints, and (s ０ , s １ , s ２ 11. The robotic imaging system of claim 10, wherein x, y, z, y, z represent respective coordinates of the centers of the spheres.

12. Each of the above speeds [0045] 12. The robotic imaging system of claim 11, wherein x is derived based in part on the respective coordinates of the centers of the spheres and time derivatives of the respective coordinates of the one or more checkpoints.

13. Each of the above speeds [0050] teeth, [006] where: [0070] The robot imaging system of claim 12 , wherein: x represents the time derivative of the respective coordinates of the one or more checkpoints.

14. 12. The robotic imaging system of claim 11, wherein the controller is adapted to calculate the radial force based in part on a respective buffer distance to at least one no-entry zone, the respective spherical distance, and the respective velocity.

15. The radial force is [0080] where r is the respective spherical distance, [0090] is the respective speed, b is the respective buffer distance to the at least one no-entry zone, and K ｐ , K ｄ The robotic imaging system of claim 14 , wherein: n is a plurality of adjustable constants of a proportional-derivative controller.

16. The respective corrective forces include a force contribution from an interface limit calculation, the controller being adapted to perform the interface limit calculation, the interface limit calculation comprising: obtaining at least one interface perpendicular to a first direction; calculating a respective distance and a respective velocity of one or more checkpoints from the at least one boundary surface along the first direction; obtaining a respective force component if the respective distance is less than a predetermined threshold; obtaining said force contribution as a sum of each of said respective force components; The robotic imaging system of claim 1 .

17. The robotic imaging system of claim 16 , wherein the at least one interface is proximate to a patient.

18. 17. The robotic imaging system of claim 16, wherein the controller is adapted to calculate the respective force components based in part on a respective buffer distance to at least one no-entry zone, the respective distance from the at least one boundary surface, and the respective velocity.

19. Each of the force components (F ｉ )teeth, [0089] where z is the respective distance from the at least one interface; ##EQU00011## is the respective speed, b is the respective buffer distance to the at least one no-entry zone, and K ｐ , K ｄ 20. The robotic imaging system of claim 18, wherein x,y,z,y,z are a plurality of adjustable constants of a proportional-derivative controller.

20. 1. A robotic imaging system, comprising: a camera configured to record one or more images of the target site; a robotic arm operatively connected to the camera, the robotic arm adapted to selectively move the camera in a movement sequence, the robotic arm including one or more joints; a force-based sensor configured to detect and transmit sensor data related to at least one of a force and / or a torque applied by a user to move the camera; a controller configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory in which instructions are recorded; Including, the controller is adapted to selectively execute a collision avoidance mode including applying respective corrective forces to modify the movement sequence when at least one of the camera and the robot arm enter a predetermined buffer zone; The respective corrective forces include force contributions from an interface limit calculation, a cart limit calculation, and a joint limit calculation.

Images