Robotic Imaging System with Velocity-Based Collision Avoidance Mode

Abstract

The robotic imaging system includes a camera configured to capture one or more images of the target site. A robotic arm is operatively connected to the camera, the robotic arm adapted to selectively move the camera in a movement sequence. The robotic imaging system includes a sensor configured to detect and transmit sensor data associated with a respective position and / or a respective velocity of the camera. A controller is configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory in which instructions are recorded. The controller is adapted to selectively execute a collision avoidance mode including determining a trajectory scale factor for the camera. The trajectory scale factor is applied to modulate the respective velocities of the camera and / or the robotic arm when they are in a predefined buffer zone.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 315,130, filed March 1, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] The present disclosure generally relates to a robotic imaging system. More specifically, the present disclosure relates to a collision avoidance mode in a robotic imaging system. Various imaging modalities are commonly used to image different parts of the human body. Robotic systems have been developed to improve the efficiency of medical procedures using these imaging modalities. The robotic system may incorporate multiple parts or components to assist a user in operating the system. Therefore, it may be difficult to avoid self-collision of multiple components, for example, collision of one robotic link with another robotic link, collision of a camera unit with a robotic arm and / or a storage unit, and collision of a camera unit with an image plane. Summary of the Invention [Means for solving the problem]

[0003] Disclosed herein is a robotic imaging system having a camera configured to capture one or more images of a target site. A robotic arm is operatively connected to the camera, the robotic arm adapted to selectively move the camera in a movement sequence. The robotic imaging system includes a sensor configured to detect and transmit sensor data associated with a respective position and / or a respective velocity of the camera. A controller is configured to receive the sensor data, the controller having a processor and a tangible non-transitory memory in which instructions are recorded. The controller is adapted to selectively execute a collision avoidance mode including determining a trajectory scale factor for the camera. The trajectory scale factor is applied to modulate the respective velocities of the camera and the robotic arm when at least one of them is in a predefined buffer zone.

[0004] The camera may be a stereoscopic camera configured to record left and right images to generate at least one stereoscopic image of the target site. In some embodiments, the predefined buffer zone is within a delta value of the at least one keep-out zone and the application of the trajectory scale factor moves the camera away from the at least one keep-out zone. The robotic imaging system may include a head unit for housing the camera and a coupling plate for mechanically coupling the head unit to the robot arm, the head unit operably connected to the cart. The controller is adapted to calculate a trajectory scale factor for a set of checkpoints disposed on the head unit, the robot arm, and / or the coupling plate. The set of checkpoints each have a respective position along a first direction and a respective velocity along the first direction.

[0005] The controller may be adapted to initialize a trajectory scale factor to a normalized value when the robot arm and / or camera are outside a predefined buffer zone, the trajectory scale factor being based on a multiple limit calculation. The controller is adapted to obtain a minimum of a plurality of local scales respectively obtained from the multiple limit calculation, the trajectory scale factor being selected as the lower value of the normalized value and the minimum of the plurality of local scales. The multiple limit calculations each employ a respective linear function raised to a predetermined scale exponent of about 1 to about 2, inclusive.

[0006] The trajectory scale factor may be based on a multi-limit calculation, including a joint avoidance calculation, where the multiple joints define respective joint angles therebetween, and the controller is adapted to perform the joint avoidance calculation by checking the distance and velocity of each joint angle of the robot arm against respective fixed joint angle limits.

[0007] The robotic imaging system may include a cart operatively connected to the camera. The trajectory scale factor may be based on multiple limit calculations, including a cart avoidance calculation. The controller is adapted to perform a cart avoidance calculation for a set of checkpoints disposed on the camera, including checking a speed and distance of each of the set of checkpoints against a surface of the cart. The surface of the cart may be modeled as a sphere. The trajectory scale factor may be based on multiple limit calculations, including a boundary plane avoidance calculation. The controller is adapted to perform a boundary plane avoidance calculation for a set of checkpoints disposed on the camera, including checking a speed and distance of each of the set of checkpoints against at least one predefined boundary plane.

[0008] The robotic imaging system may include an annular scan mode executable by the controller to enable the robot arm to sweep an annular trajectory in at least a partially circumferential direction around the target site. Executing the collision avoidance mode includes generating an adjusted annular trajectory based in part on the trajectory scale factor, the adjusted annular trajectory being defined in a spherical coordinate axis defining a first spherical angle and a second spherical angle. The target site includes a serrated fossa of the eye. The robotic imaging system may include a low pass filter selectively executable by the controller to smooth changes in the second spherical angle in each cycle. The robotic imaging system may include a saturation function selectively executable by the controller, the saturation function limiting a magnitude of the second spherical angle to within 0 degrees and 90 degrees, inclusive, in each cycle.

[0009] The controller may be adapted to change the field of view angle of the circular orbit by repeating the second spherical angle while holding the first spherical angle until a desired field of view angle is reached. The controller is adapted to selectively command the circular orbit by repeating the first spherical angle between a predefined start angle and a predefined end angle while the second spherical angle is at the desired field of view angle. The controller is adapted to generate an adjusted circular orbit from the circular orbit via a limit feedback term and a reset feedback term. The limit feedback term reduces a cycle radius of the adjusted circular orbit near a joint limit and the reset feedback term resets the cycle radius of the adjusted circular orbit back when the joint limit is cleared.

[0010] The limiting feedback term is based on the trajectory scale factor, the time derivative of the trajectory scale factor, the first proportional gain constant, and the first differential gain constant. limit ) can be obtained as follows:

number

number

number

number

[0011] The above-referenced features and advantages, as well as other features and advantages of the present disclosure, are readily apparent from the following detailed description of the best modes for carrying out the disclosure when taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a partial schematic diagram of a robotic imaging system having a camera and a controller with a collision avoidance mode. [Diagram 2] 4 is a flow chart of an exemplary method for operating a collision avoidance mode. [Diagram 3] FIG. 2 is a schematic partial top view of the robotic imaging system of FIG. 1 showing one or more checkpoints. [Figure 4] FIG. 13 is a schematic partial view showing the buffer distance from the keep-out zone of the checkpoint. [Diagram 5] 1 is a schematic graph of an exemplary scaling curve showing radius on the horizontal axis and scale on the vertical axis. [Figure 6] FIG. 2 is a schematic diagram of a circular orbit modified by the collision avoidance mode of FIG. 1; [Figure 7] 2 is a schematic partial cross-sectional view of an eye illustrating an exemplary circular trajectory performed by the robotic imaging system of FIG. 1. [Figure 8] FIG. 7 is a schematic diagram of a virtual sphere that can be employed by a robotic imaging system when executing the trajectory of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Exemplary embodiments of the present disclosure are shown in the drawings as non-limiting examples and are described in more detail below. However, it should be understood that the novel aspects of the present disclosure are not limited to the specific forms shown in the drawings listed above. Rather, the present disclosure is intended to encompass modifications, equivalents, combinations, subcombinations, permutations, groupings, and alternatives that are included within the scope of the present disclosure, such as those encompassed by the appended claims.

[0014] Referring to the drawings, in which like reference numbers refer to like components, FIG. 1 illustrates generally a robotic imaging system 10 having a camera 12 with a collision avoidance mode 14. The robotic imaging system 10 is configured to image a target site 16. The camera 12 illustrated in FIG. 1 is a stereoscopic camera 12, although it is understood that other types of cameras (e.g., cameras that capture a single image) may also be employed. With reference to FIG. 1, the camera 12 is disposed at least partially within a head unit 18 of a housing assembly 20, which is configured to be oriented at least partially toward the target site 16. The camera 12 may be configured to record first and second images of the target site 16 and may be used to generate a live two-dimensional stereoscopic view of the target site 16. The target site 16 may be an anatomical location of a patient, a biological specimen in a laboratory, a calibration slide / template, or the like.

[0015] 1, a robotic imaging system 10 includes a controller C having at least one processor P and at least one memory M (or non-transitory tangible computer-readable storage medium) having instructions recorded thereon for executing a method 100 (described below with respect to FIG. 2) for operating a collision avoidance mode 14. The memory M can store a controller-executable instruction set, and the processor P can execute the controller-executable instruction set stored in the memory M.

[0016] 1, at least one input device 22 (hereafter omitted "at least one") is operatively connected to the camera 12 (e.g., at the head unit 18) to allow a user to manually position the camera 12. The input device 22 may include respective controls for activating or selecting certain features of the camera 12, such as adjusting focus, magnification, amount / type of light projected onto the target site 16, and other features. It is understood that the number and form of the input devices 22 may vary, for example, the input device 22 may include a joystick, a wheel, a mouse, or a touch screen device. In some embodiments, the input device 22 may be controlled via a remote control 66 (see FIG. 1).

[0017] With reference to FIG. 1, the robotic imaging system 10 includes a robotic arm 24 operatively connected to a head unit 18 and configured to selectively move the head unit 18. The head unit 18 may be mechanically coupled to the robotic arm 24 via a coupling plate 26. A user may position and orient the camera 12 with assistance from the robotic arm 24. As shown in FIG. 1, a sensor 28 is operatively connected to the robotic arm 24 and / or the coupling plate 26. The sensor 28 is configured to detect and transmit sensor data related to the position and / or velocity of the camera 12. The position of the sensor 28 may be varied based on the application at hand, for example, at the interface between the coupling plate 26 and the camera 12. It is understood that different types of sensor technologies available to one skilled in the art may be utilized for the position / velocity based sensor 28.

[0018] The robot arm 24 may include one or more joints, such as a first joint 30 and a second joint 32, configured to provide further degrees of positioning and / or orientation of the head unit 18. With reference to Figure 1, a corresponding joint motor (e.g., joint motor 31) and a respective joint sensor (e.g., joint sensor 33) may be coupled to each joint. The joint motor 31 is configured to rotate the first joint 30 about an axis, while the joint sensor 33 is configured to transmit a position (in 3D space) of the first joint 30.

[0019] With reference to FIG. 1 , the robot arm 24 (and / or the coupling plate 26) may be controlled via an embedded processor, such as a controller C and / or a robot arm controller 42. The robot arm 24 may be selectively operable to extend the field of view of the camera 12 along the X-axis, the Y-axis, and the Z-axis. The robot arm controller 42 may include a processor, server, microcontroller, workstation, etc., configured to convert one or more messages or commands from the controller C into messages and / or signals that rotate any one of the joints. The robot arm controller 42 is also configured to receive and convert sensor information, such as joint position and / or velocity, from the robot arm 24 and / or the coupling plate 26 into one or more messages to the controller C. A stereoscopic visualization camera is described in U.S. Patent Application No. 16 / 398,014 (filed April 29, 2019), the entire contents of which are incorporated herein by reference, along with an integrated robotic platform.

[0020] The head unit 18 may be connected to a cart 34 having at least one display medium (which may be a monitor, a terminal, or other form of two-dimensional visualization), such as the first and second displays 36 and 38 shown in FIG. 1. Referring to FIG. 1, the controller C may be configured to process signals for broadcast on the first and second displays 36 and 38. The housing assembly 20 may be completely self-contained and movable between various locations. The image streams from the cameras 12 may be sent to the controller C and / or a camera processor (not shown), which may be configured to prepare the image streams for viewing. For example, the controller C may combine or interleave the first and second video signals from the cameras 12 to create a stereoscopic signal. The controller C may be configured to store the video and / or stereoscopic video signals in a video file, stored in the memory M. The first and second displays 36 and 38 may incorporate a stereoscopic display system, where the two-dimensional display has separate images for the left and right eyes. To view a stereoscopic display, a user may wear special glasses that operate in conjunction with the first and second displays 36, 38 to present a left field of view to the user's left eye and a right field of view to the user's right eye.

[0021] 1, the first display 36 may be connected to the cart 34 via a flexible mechanical arm 40 having one or more joints to allow for flexible positioning. The flexible mechanical arm 40 may be configured to have a length sufficient to extend over the patient to provide a relatively close view to the surgeon during surgery. The first and second displays 36 and 38 may include any type of display, such as a high definition television, an ultra high definition television, smart glasses, a projector, one or more computer screens, a laptop computer, a tablet computer, and / or a smartphone, and may include a touch screen.

[0022] The camera 12 is configured to acquire images of the target site 16, which may be presented in a variety of forms, including, but not limited to, captured still images, real-time images, and / or digital video signals. As used herein, "real-time" generally refers to updating information at the same rate as the data is received. More specifically, "real-time" means that image data is acquired, processed, and transmitted at a sufficiently high data rate and sufficiently low latency so that when the data is displayed, the subject moves smoothly without jerks or wait times that are noticeable to the user. Typically, real-time occurs when new images are acquired, processed, and transmitted at a rate of at least about 30 frames per second (fps) and displayed at about 60 fps, and when the combined processing of the video signals has a latency of about 1 / 30th of a second or less.

[0023] As described below, the collision avoidance mode 14 is velocity-based. The controller C is adapted to calculate a single multiplier (herein referred to as the trajectory scale factor) and apply it to the velocity command (for the robot arm 24) to slow the robot velocity down to a stop as a certain limit is approached. The trajectory scale factor is utilized to avoid self-collisions of various components, e.g., collision of the camera 12 with the robot arm 24, collision of the camera 12 with a storage unit (e.g., the cart 34), and collision of the first joint 30 with the second joint 32.

[0024] At each control update of the robot arm 24, the desired velocity is checked and a scale down occurs if it is determined that the robot arm 24 is approaching a limit. The limit may be defined in joint space, Cartesian space, or other reference frame. If the robot arm 24 is not approaching a limit, the trajectory scale factor is set to 1.0 to allow for quick reversals from the limit. The collision avoidance mode 14 provides relatively smooth motion when entering an area to be avoided. There is no need to "clear" the limit to move out of the restricted area.

[0025] The controller C is adapted to calculate trajectory scale factors for one or more checkpoints 200 located on the head unit 18, the robot arm 24, and elsewhere, as shown in FIG. 3. The checkpoints 200 each have a respective position and a respective velocity, both along a first direction (e.g., the x-direction in FIG. 3). The position and velocity of each of the checkpoints 200 may be calculated with respect to a fixed coordinate frame (e.g., either the camera frame or the robot base frame). The fixed coordinate frame may be selected or changed based on the application at hand. With reference to FIG. 3, the checkpoints 200 may include multiple points on the input device 22, the head unit 18, the coupling plate 26, and the robot arm 24. With reference to FIG. 3, the input device 22 may include a first handle 202 and a second handle 204. In some embodiments, the input device 22 includes multiple checkpoints, such as a first handle upper point 210, a first handle lower point 214, a second handle upper point 212, and a second handle lower point 216. 3 , the check points 200 may include respective points on the head unit 18, such as a camera proximal point 218, a camera distal point 220, a first side camera intermediate point 222, and a second side camera intermediate point 224. The set of check points 200 may further include a tool center point 226 and a joint extrusion point 228 at a joint 230 of the robot arm 24.

[0026] The collision avoidance mode 14 of FIG. 1 is activated when the camera 12 and / or the robot arm 24 enter a predefined region, such as the predefined buffer zone 250 shown in FIG. 4. The trajectory scale factor is initialized to a normalized value (e.g., 1) when the robot arm 24 and / or the camera 12 are outside the predefined buffer zone 250. With reference to FIG. 4, the predefined buffer zone 250 is shown with respect to a reference axis 252 extending from an origin O. The predefined buffer zone 250 is within a delta value 254 of at least one "keep out" region, such as a keep out zone 256. The trajectory scaling prevents the camera 12 and / or the robot arm 24 from moving into the keep out zone 256 to prevent a collision. As an example, the first reference point 258 of FIG. 4 is not within the predefined buffer zone 250, so trajectory scaling is not applied. However, the second reference point 260 is within the predefined buffer zone 250 (within the buffer distance 262) and trajectory scaling will be applied.

[0027] Referring now to FIG. 2, a flow chart of an exemplary method 100 for operating the collision avoidance mode 14 of FIG. 1 is shown. This embodiment is velocity-based and functions by adjusting the velocity of the robot arm 24. The method 100 may be embodied as computer readable code or instructions stored in and partially executable by the controller C of FIG. 1. The method 100 does not have to be applied in the particular order described herein and may be executed dynamically. Furthermore, it should be understood that some steps may be omitted. The method 100 may be executed periodically at predefined time intervals.

[0028] The method 100 begins at block 102 of FIG. 2 where the controller C is programmed to receive input data, such as joint position data of the robot arm 24 and sensor data from the sensors 28 related to the positions and / or velocities of various checkpoints 200. Proceeding to block 104 of FIG. 2, the controller C may be programmed to infer the user's intent in terms of how the user wishes to steer the camera 12. This may be done via a force-based sensor 29 that detects the forces and / or torques applied by the user to move the camera 12. The robot arm 24 may also include an assisted drive function incorporating a user-guided control system, whereby the controller C causes the robot arm 24 and / or the coupling plate 26 to move the camera 12 in a "power steering" manner to achieve the user's desired movement.

[0029] 2, the method 100 includes a coordinate transformation from the sensor frame to the robot base frame (the coordinate axes on the cart 34 to which the base of the robot arm 24 is mounted). The transformation may include one or more predefined equations or relationships based on the position and orientation of the sensor 28. The transformation may be based on parameters obtained via calibration.

[0030] Proceeding to block 108 of Figure 5, the controller C is programmed to get the previous joint command (from the previous cycle). The collision avoidance mode 14 takes as input the last joint position and the current joint position, in other words, tracks the joint coordinate of the previous cycle. When used with the velocity command, the current joint position can be used to approximate the next joint coordinate as follows: the next joint coordinate is approximated as the sum of the current joint coordinate and the product of the sample period and the velocity command.

[0031] Proceeding to block 110 of Figure 5, the method 100 includes obtaining multiple local scales from multiple limit calculations. The local scales are obtained from multiple independent limit calculations each employing a respective linear function raised to a predetermined scale exponent (V), the scale exponent being from about 1 to about 2 inclusive. The limit calculation uses the same algorithm whether using the distance from one joint to the joint limit or the radial distance in Cartesian space.

[0032] In the embodiment described herein, the method 100 uses three different types of limit calculations: joint avoidance calculations, cart avoidance calculations, and boundary plane avoidance calculations. However, it is understood that other types of limit calculations may also be employed. For joint avoidance calculations, the controller C performs it by checking the distance and velocity of each joint angle of the robot arm 24 against the respective fixed joint angle limits. For cart avoidance calculations, for a set of checkpoints 200, the controller C checks the respective distances (e.g., radial distance r) and respective velocities of the checkpoints 200 relative to the surface of the cart 34.

number

[0033] Each of the limit calculations has a set of adjustable parameters that govern how quickly the robot arm 24 decelerates to zero as the respective limit is approached. The set of adjustable parameters may include which axis to limit and the origin of the limit. Because the articulated links can move position and change orientation, the checkpoints 200 may collide with each other from many different angles. To prevent this, the X, Y, and Z axes must be selectively constrained. In other words, the adjustable parameters have active axis members that set respective multipliers for whether the X, Y, and Z axes are active. For link boundaries, the origin is presumed to be where the frame of reference for the link boundaries resides. The set of adjustable parameters may include the trigger distance (D T ) and stopping distance (D S ) Trigger distance 306 (D T ) is the distance that the checkpoint 200 should be from the boundary origin (e.g., origin O in FIG. 4) before the movement of the robot arm 24 is scaled to a speed slower than the maximum allowable speed for the current user settings. S ) is the trigger distance 306 (D T ) is the percentage.

[0034] A graph showing how the local scale is selected for each of the limit calculations is shown in Figure 5. The graph 300 of Figure 5 shows radial distance D on the horizontal axis and scale factor S on the vertical axis. The controller C determines the radial distance D, the predefined minimum scale 304 (S min ), predefined maximum scale 308(S max ), stopping distance 302(D S ), and trigger distance 306 (D T ) The scale factor S for each of the limit calculations is obtained as follows:

number

[0035] Local Scale (S V ) is obtained as the scale factor S raised to an exponent (herein referred to as the scale exponent V). The scale exponent is one of the adjustable factors that govern the process. The higher the value of the scale exponent V, the more aggressively the collision avoidance mode 14 will scale the velocity of the robot arm 24. The higher the selected value of the scale exponent V, the more aggressively the collision avoidance mode 14 will scale the velocity of the robot arm 24.

[0036] Further, according to block 110 of FIG. 2, the controller C is programmed to obtain a trajectory scale factor. The trajectory scale factor is selected as the lower of the normalized value (typically 1.0) and the minimum value of the local scale. For example, trajectory scale factor = min[1.0, minS of all limit calculations V In another approach, the trajectory scale factor can be initialized to 1.0 and passed through multiple bounded calculations, each using a linear function raised to a scale exponent V to produce a multiplier between 0 and 1. At each step, the trajectory scale factor is updated to be the minimum of the current value and the intermediate calculations.

[0037] Proceeding to block 112 of FIG. 2, the controller C is programmed to validate the command to scale the trajectory or velocity of the robot arm 24 to ensure that the command (or a signal indicative of the command) is within the operating parameters of the joint motors (e.g., duration, rotational speed, etc.). The controller C and / or the robot arm controller 42 may also validate the command by comparing it to a current threshold to ensure that the robot arm 24 does not consume excessive current during any stage of the movement sequence. Finally, the controller C is programmed to send commands via one or more signals to the appropriate joint motors of the robot arm 24 and / or the coupling plate 26 according to the movement sequence. The sent commands cause the motors at the respective joints to move the robot arm 24 and / or the coupling plate 26, thereby moving the camera 12 while avoiding collisions. The collision avoidance mode 14 avoids contacts that may damage equipment and is very fast to compute.

[0038] Circular orbit correction using collision avoidance mode 6-8, an exemplary implementation of the collision avoidance mode 14 is presented for annular scanning. In the embodiment shown in FIG. 7, the target site 16 (see FIG. 1) is an eye 400 having a lens 402. The target site 16 is a serrated junction between the retina 406 and the iris corona region 408, including a portion of the serrated fossa 404 that defines the transition point between the non-sensory and multi-sensory regions of the eye 400. With reference to FIG. 7, an operator / surgeon can center the robotic imaging system 10 on the eye 400 and move the robotic arm 24 in an annular orbit 410 to perform a circumferential scan around the eye 400. The annular orbit 410 can cover an angle 412 of approximately 180 degrees to 360 degrees.

[0039] Due to workspace and other limitations, the robotic imaging system 10 may not be able to complete a full rotation of the annular scan. Such limitations are typically due to joint limitations. A common joint limit is full elbow extension or compression. Current trajectory algorithms may get stuck on a joint limit and be unable to complete a specified rotation trajectory, requiring manual intervention by the user to exit the limit and reposition the camera 12. FIG. 6 is a schematic diagram of an adjusted annular trajectory 420 modified by the collision avoidance mode 14 and superimposed on an unconstrained annular trajectory 410. As will be described below, this modification is based in part on the trajectory scale factor obtained by the method 100 shown in FIG. 2.

[0040] The circular trajectory 410 may be defined in terms of a spherical coordinate system having a first spherical angle (U) and a second spherical angle (V) as shown in FIG. 8 for an exemplary location T in the XYZ space and a projection Q onto the XY plane. The circular trajectory 410 may be performed by iterating through at least one of the first spherical angle (U) and the second spherical angle (V). First, the viewing angle of the circular trajectory 410 is changed by keeping the first spherical angle (U) fixed (e.g., at 0 degrees, 10 degrees, or other angle) and iterating the second spherical angle (V) from (V+ΔV) to (V-ΔV) until a desired viewing angle is reached. The desired viewing angle represents the angle required to view a desired portion of the eye anatomy and may be pre-programmed into the controller C based on an eye model and / or anatomical data.

[0041] The circular trajectory 410 is then achieved by repeatedly moving along the first spherical angle (U) of the virtual sphere 500 while holding the second spherical angle (V) constant at the desired viewing angle, an example of which is shown in Figure 8. The controller C and / or the robot arm controller 42 of Figure 1 allow the operator to move the camera 12 on the outer surface 502 (see Figure 8) of the virtual sphere 500 to an end position 506 while keeping the camera 12 pointed toward the center 504 (as shown by the view vectors 508, 510). In other embodiments, the virtual sphere 500 may be represented with a different shape and the outer surface 502 may be represented by a flat surface 512.

[0042] The controller C has a limit feedback term and a reset feedback term [ΔR = R limit +R reset ], a tuned circular orbit 420 can be generated by modifying the circular orbit 410 by the limiting feedback term (R limit ) can be obtained as follows:

number

number

number

number

[0043] As the trajectory scale factor approaches a joint limit and drops, the limit feedback term reduces the cycle radius (as shown in region 422 of FIG. 6 ), pulling the robot arm 24 away from the joint limit and preventing the trajectory scale factor from dropping to zero. As the trajectory scale factor converges to one, the reset feedback term increases the cycle radius (as shown in region 424 of FIG. 6 ) toward the nominal radius that represents the desired field of view angle. This results in the cycle radius decreasing while the joint limit is near (see region 422), and then resetting back toward the nominal radius (see region 424) that represents the desired field of view angle once the limit is cleared. The technical advantage here is that the system can complete many circular rotations that would otherwise fail or get stuck (without adjustments).

[0044] In summary, when performing a circular orbit in a scenario where the robot's joint limits prevent a complete rotation of the first spherical angle (U) at the desired viewing angle, the collision avoidance mode 14 can dynamically adjust the cycle radius (as shown by region 422 in FIG. 6) to avoid the joint limits. Referring to FIG. 1, the robotic imaging system 10 can include a low pass filter 52 selectively executable by the controller C to smooth the change in the second spherical angle (V) at each cycle. The robotic imaging system 10 can include a saturation function 54 selectively executable by the controller C to limit the magnitude of the second spherical angle (V) to within 0 degrees to 90 degrees (inclusive) at each cycle.

[0045] The controller C of Figure 1 may include or otherwise access information downloaded from remote sources and / or executable programs. With reference to Figure 1, the controller C may be configured to communicate with a remote server 60 and / or a cloud unit 62 via a network 64. The remote server 60 may be a private or public information source maintained by an organization such as, for example, a research institute, a company, a university, and / or a hospital. The cloud unit 62 may include one or more servers hosted on the Internet for data storage, management, and processing.

[0046] The network 64 may be a serial communication bus in the form of a local area network. The local area network may include, but is not limited to, a Controller Area Network (CAN), Controller Area Network with Flexible Data Rate (CAN-FD), Ethernet, blue tooth, WIFI, and other data formats. The network 64 may be a wireless local area network (LAN) that links multiple devices in a wirelessly distributed manner, a wireless metropolitan area network (MAN) that connects multiple wireless LANs, or a wireless wide area network (WAN) that covers a large area such as a nearby city or town. Other types of connections may also be used.

[0047] The controller C of FIG. 1 may be an integral part of the robotic imaging system 10 or may be a separate module operably connected to the robotic imaging system 10. The controller C includes computer-readable media (also referred to as processor-readable media), including non-transitory (e.g., tangible) media involved in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media may take many forms, including, but not limited to, non-volatile media and volatile media. Non-volatile media may include, for example, optical or magnetic disks and other persistent memories. Volatile media may include, for example, dynamic random access memory (DRAM), which may constitute a main memory. Such instructions may be transmitted by one or more transmission media, including coaxial cables, copper wires, and optical fibers, including the wires that comprise a system bus coupled to the processor of the computer. Some forms of computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, magnetic tape, other magnetic media, CD-ROM, DVD, other optical media, punch cards, paper tape, other physical media with patterns of holes, RAM, PROM, EPROM, Flash EEPROM, other memory chips or cartridges, or other media from which a computer can read.

[0048] The lookup tables, databases, data repositories, or other data stores described herein may include various types of mechanisms for storing, accessing, and retrieving various types of data, including a hierarchical database, a set of files in a file system, a proprietary application database, a relational database management system (RDBMS), and the like. Each such data store may be contained within a computing device employing a computer operating system, such as one of the computer operating systems described above, and may be accessed over a network in one or more of a variety of ways. The file system may be accessible from the computer operating system and may include files stored in various formats. The RDBMS may employ a structured query language (SQL) in addition to a language for creating, saving, editing, and executing stored procedures, such as the PL / SQL language described above.

[0049] The flowcharts presented herein illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or part of code, and includes one or more executable instructions for implementing the specified logical functions. It should also be noted that each block of the block diagrams and / or example flowcharts, and combinations of blocks in the block diagrams and / or example flowcharts, may be implemented by a special purpose hardware-based device that performs the specified functions or acts, or a combination of special purpose hardware and computer instructions. These computer program instructions may also be stored on a computer-readable medium. The computer program instructions may be stored on a computer-readable medium, including instructions for implementing the functions / acts specified in the blocks of the flowcharts and / or block diagrams, to instruct a controller or other programmable data processing apparatus to function in a specific manner to produce an article of manufacture.

[0050] Numerical values ​​of parameters (e.g., amounts or conditions) in this specification, including the appended claims, should be understood to be modified by "about" in each and every instance, regardless of whether the term "about" is actually placed before the numerical value. "About" indicates that the stated numerical value allows for some slight imprecision (in some approaches, being the exact value, being roughly or reasonably close to the value, approximately). If the imprecision provided by "about" is not understood in this ordinary sense in the art, "about" as used herein at least indicates the variation that may result from ordinary methods of measuring and using such parameters. In addition, the disclosure of a range includes the disclosure of each value and further divided ranges within the entire range. Each value within a range and the endpoints of the range are disclosed herein as separate embodiments.

[0051] The detailed description and drawings or figures support and explain the disclosure, but the scope of the disclosure is defined only by the claims. Although some of the best modes and other embodiments for carrying out the claimed disclosure have been described in detail, there are various alternative designs and embodiments for carrying out the disclosure defined in the appended claims. Furthermore, the features of the embodiments shown in the drawings or the various embodiments mentioned herein should not necessarily be understood as independent embodiments from each other. Rather, each of the characteristics described in one of the example embodiments can be combined with one or more other desirable characteristics from the other embodiments, resulting in other embodiments not described in words or by reference to the drawings. Therefore, such other embodiments are included within the framework of the appended claims.

Claims

1. 1. A robotic imaging system, comprising: a camera configured to record one or more images of the target site; a robotic arm operatively connected to the camera, the robotic arm adapted to selectively move the camera through a sequence of movements, the robotic arm including one or more joints; a sensor configured to detect and transmit sensor data relating to a respective position and / or a respective velocity of said cameras; a controller configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory in which instructions are recorded; Equipped with The controller, based on the sensor data, determining a trajectory scale factor for the camera; applying the trajectory scale factor to modulate the respective velocities of the camera and the robot arm when at least one of the camera and the robot arm is in a predefined buffer zone; 16. A robotic imaging system adapted to selectively execute a collision avoidance mode, comprising:

2. The robotic imaging system of claim 1 , wherein the camera is a stereoscopic camera configured to record left and right images to generate at least one stereoscopic image of the target area.

3. 2. The robotic imaging system of claim 1, wherein the predefined buffer zone is within a delta value of at least one keep-out zone, and application of the trajectory scale factor moves the camera away from the at least one keep-out zone.

4. a head unit for housing the camera; a coupling plate that mechanically couples the head unit to the robot arm, the head unit being operably connected to a cart; and Further comprising: the controller is adapted to calculate the trajectory scale factor for a set of checkpoints located on the head unit, the robot arm, and / or the binding plate, the set of checkpoints each having a respective position along a first direction and a respective velocity along the first direction; The robotic imaging system of claim 1 .

5. The controller is adapted to initialize the trajectory scale factor to a normalized value when the robot arm and / or the camera are outside the predefined buffer zone, the trajectory scale factor being based on a multi-threshold calculation; the controller is adapted to obtain a minimum value of a plurality of local scales obtained respectively from the multiple limit calculations, and the trajectory scale factor is selected as the lower value of the normalized value and the minimum value of the plurality of local scales. The robotic imaging system of claim 1 .

6. 6. The robotic imaging system of claim 5, wherein the multiple limit calculations each employ a respective linear function raised to a predetermined scaling exponent of about 1 to about 2, inclusive.

7. the robotic arm includes a plurality of joints defining respective joint angles therebetween; the trajectory scale factor is based on multiple limit calculations, including a joint avoidance calculation; the controller is adapted to perform the joint avoidance calculations by checking distances and velocities of each joint angle of the robot arm against respective fixed joint angle limits. The robotic imaging system of claim 1 .

8. a cart operably connected to the camera; the trajectory scale factor is based on multiple limit calculations, including a cart avoidance calculation; the controller is adapted to perform the cart avoidance calculations relative to a set of checkpoints located on the camera, the checkpoints including checking the respective speeds and distances of the set of checkpoints relative to a surface of the cart; The robotic imaging system of claim 1 .

9. The robotic imaging system of claim 8 , wherein the surface of the cart is modeled as a sphere.

10. the trajectory scale factor is based on multiple limit calculations, including a boundary plane avoidance calculation; the controller is adapted to perform the boundary plane avoidance calculations against a set of check points located on the camera, the check points including checking the respective speeds and distances of the set of check points against at least one predefined boundary plane; The robotic imaging system of claim 1 .

11. and an annular scan mode executable by the controller to enable the robotic arm to sweep a circular orbit in an at least partially circumferential direction around the target region; performing the collision avoidance mode includes generating an adjusted circular orbit based in part on the orbit scale factor, the adjusted circular orbit being defined on spherical coordinate axes defining a first spherical angle and a second spherical angle; The robotic imaging system of claim 1 .

12. The robotic imaging system of claim 11 , wherein the target site comprises a serrated fossa of the eye.

13. a low pass filter selectively executable by the controller to smooth the change in the second spherical angle in each cycle; a saturation function selectively executable by the controller to limit the magnitude of the second spherical angle to within 0 degrees and 90 degrees inclusive in each cycle; The robotic imaging system of claim 11 further comprising:

14. the controller is adapted to vary the field of view angle of the annular orbit by iterating the second spherical angle while holding the first spherical angle until a desired field of view angle is reached; the controller is adapted to selectively command the circular orbit by cycling the first spherical angle between a predefined start angle and a predefined end angle while the second spherical angle is at the desired field of view angle. The robotic imaging system of claim 11.

15. the controller is adapted to generate the adjusted circular orbit from the circular orbit via a limit feedback term and a reset feedback term; the limit feedback term reduces a cycle radius of the adjusted circular orbit near a joint limit, and the reset feedback term resets the cycle radius of the adjusted circular orbit back when the joint limit is cleared. The robotic imaging system of claim 11.

16. The robotic imaging system of claim 15 , wherein the limiting feedback term is based on the trajectory scale factor, a time derivative of the trajectory scale factor, a first proportional gain constant, and a first derivative gain constant.

17. The limiting feedback term (R ｌｉｍｉｔ )but, [0010] where SF denotes the orbit scale factor, [0025] denotes the time derivative of the trajectory scale factor, and K ｐ１ denotes the first proportional gain constant, and K ｄ１ denotes the first differential gain constant; The robotic imaging system of claim 16.

18. 16. The robotic imaging system of claim 15, wherein the reset feedback term is based on a cycle radius corresponding to the second spherical angle, a time derivative of the cycle radius, a nominal radius corresponding to a desired field of view angle, a second proportional gain constant, and a second derivative gain constant.

19. The reset feedback term (R ｒｅｓｅｔ )but, [0030] where R denotes the cycle radius and R ０ denotes the nominal radius, [0045] denotes the time derivative of the cycle radius, and K ｐ２ denotes the second proportional gain constant, and K ｄ２ denotes the second differential gain constant; 20. The robotic imaging system of claim 18.

20. 1. A robotic imaging system, comprising: a camera configured to record one or more images of the target site; a robotic arm operatively connected to the camera, the robotic arm adapted to selectively move the camera through a sequence of movements, the robotic arm including one or more joints; a sensor configured to detect and transmit sensor data relating to a respective position and / or a respective velocity of said cameras; a controller configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory in which instructions are recorded; Equipped with the controller is adapted to selectively execute a collision avoidance mode, comprising: determining a trajectory scale factor for the camera based on the sensor data; and applying the trajectory scale factor to modulate a velocity of each of the camera and the robot arm when at least one of the camera and the robot arm is in a predefined buffer zone; The trajectory scale factor is used for joint avoidance calculations, cart avoidance calculations, and boundary plane avoidance calculations. Based on multiple marginal calculations, including Robotic imaging system.

Images