Contact management mode for robotic imaging system using multiple sensors

Abstract

A robotic imaging system includes a body having a head unit, a robotic arm, and a coupling plate. A camera is positioned in the head unit, the camera configured to record one or more images of a target site. The camera is operably connected to one or more handles. The plurality of sensors are configured to transmit sensor data. The sensors include at least one primary sensor configured to detect a respective force and / or a respective torque applied at the body and at least one secondary sensor configured to detect a respective force and / or a respective torque applied at the one or more handles. The controller is configured to receive the sensor data and execute a contact management mode, including determining a contact location of the body with the external object based in part on the sensor data and determining a reverse trajectory for moving the body away from the contact location.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Application No. 63 / 373,890, filed on August 30, 2022, which is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to a robotic imaging system. More specifically, the present disclosure relates to a contact management mode in a robotic imaging system. Different imaging modalities are often used to image different parts of the human body. Robotic systems have been developed to improve the efficiency of medical procedures that employ these imaging modalities. The robotic system may include multiple parts or components to assist the user in operating the system. Therefore, it may be challenging to avoid physical contact between various components of the robotic system and external objects (including people) in the surrounding environment. Summary of the invention

[0004] A robotic imaging system is disclosed herein, the robotic imaging system having a body having a head unit, a robotic arm, and a coupling plate coupling the head unit to the robotic arm. The robotic arm is adapted to selectively move the head unit. A camera is positioned in the head unit, the camera being configured to record one or more images of a target site. The camera is operably connected to one or more handles. The system includes a plurality of sensors configured to transmit sensor data. The sensors include at least one main sensor and at least one auxiliary sensor, the at least one main sensor being configured to detect a corresponding force and / or a corresponding torque applied to the body, and the at least one auxiliary sensor being configured to detect a corresponding force and / or a corresponding torque applied to one or more handles. A controller is configured to receive the sensor data. The controller has a processor and a tangible non-transitory memory having instructions recorded thereon. The controller is adapted to execute a contact management mode, including determining a contact position of the body with an external object based in part on the sensor data. The contact management mode includes determining a reverse trajectory for moving the body away from the contact position.

[0005] The camera may be a stereo camera configured to record a left image and a right image for generating at least one stereo image of the target part. The target part may be an eye. The plurality of sensors may respectively include a six-degree-of-freedom tactile force sensing device.

[0006] The handle includes a first handle and a second handle, which are rigidly attached to the handle bar and connected by the handle bar. In some embodiments, the auxiliary sensor can be a single sensor positioned on the handle bar. In other embodiments, the auxiliary sensor includes a first auxiliary sensor attached to the first handle and a second auxiliary sensor attached to the second handle. The main sensor can be located at a center point between the robot arm and the camera. Multiple sensors define a corresponding reference coordinate system.

[0007] In some embodiments, the controller is adapted to transform the corresponding reference coordinate system to the common coordinate system. The controller can be adapted to apply a correction factor to the sensor data, including applying a low pass filter and / or performing gravity correction on the multiple sensors based on the corresponding positions of the multiple sensors. The controller can be adapted to obtain a boundary plane extending through the contact position and perpendicular to the first direction. The reverse trajectory begins in a direction orthogonal to the boundary plane.

[0008] In some embodiments, the reverse trajectory is constrained to the surface of a virtual sphere. The sensor data provides a measured torque and a measured force at a measurement location, the measurement location and the contact location being connected by a displacement vector. The controller may be adapted to determine the torque applied at the contact location as the difference between the measured torque and the cross product of the applied force and the displacement vector.

[0009] A robotic imaging system is disclosed herein, the robotic imaging system having a body, the body including a head unit, a robotic arm, and a coupling plate coupling the head unit to the robotic arm. The robotic arm is adapted to selectively move the head unit. A camera is positioned in the head unit, the camera being configured to record one or more images of a target site. The camera is operably connected to one or more handles. The system includes a plurality of sensors configured to transmit sensor data. The plurality of sensors include at least one main sensor and at least one auxiliary sensor, the at least one main sensor being configured to detect a corresponding force and / or a corresponding torque applied to the body, the at least one auxiliary sensor being configured to detect a corresponding force and / or a corresponding torque applied to the one or more handles.

[0010] A controller is configured to receive the sensor data, the controller having a processor and a tangible non-transitory memory having instructions recorded thereon. The controller is adapted to execute a contact management mode, including obtaining a contact position of the body with the external object based in part on the sensor data, obtaining a boundary plane extending through the contact position, and determining a reverse trajectory for moving the body away from the contact position. The boundary plane is perpendicular to the reference axis.

[0011] The above features and advantages and other features and advantages of the present disclosure are readily apparent from the following detailed description of the best modes for carrying out the present disclosure when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a schematic partial diagram of a robotic imaging system having a camera, a sensor, and a controller with a contact management mode;

[0013] Figure 2 yes Figure 1 A schematic partial top view of a robotic imaging system;

[0014] Figure 3 is a flow chart of an example method for operating a contact management mode;

[0015] Figure 4 According to an alternative embodiment Figure 1 A schematic partial perspective view of the positioning of sensors in a system;

[0016] Figure 5 It is shown Figure 1 A schematic partial diagram of a handle and auxiliary sensors that may be used in the system;

[0017] Figure 6 It is shown Figure 1 A schematic partial diagram of the reverse trajectory of the system; and

[0018] Figure 7 is a diagram of a virtual sphere, showing Figure 1 Example constraint trajectories for a system of

[0019] Representative embodiments of the present disclosure are shown in the accompanying drawings by way of non-limiting examples and are described in more detail below. However, it should be understood that the novel aspects of the present disclosure are not limited to the specific forms shown in the above enumerated drawings. Instead, the present disclosure will cover modifications, equivalents, combinations, sub-combinations, permutations, groupings, and alternatives that fall within the scope of the present disclosure, such as those covered by the appended claims. DETAILED DESCRIPTION

[0020] Referring to the drawings, wherein like reference numerals refer to like parts, Figure 1 to Figure 2 A robotic imaging system 10 is schematically shown with a camera 12. Figure 1 The camera 12 is shown as a stereo camera 12, but it should be understood that other types of cameras (e.g., cameras that capture a single image) may also be used. The camera 12 is configured to record a first image and a second image of a target site 16, which may be used to generate a real-time two-dimensional stereoscopic view of the target site 16. The target site 16 may be an anatomical location of a patient, a laboratory biological sample, a calibration slide / template, etc. In one example, the target site 16 is an eye.

[0021] exist Figure 2 A top view of the robotic imaging system 10 is shown in FIG. Figure 1 to Figure 2 , the camera 12 is at least partially located in the head unit 18 of the body 20. The head unit 18 is configured to be at least partially pointed at the target site 16. One or more handles 22 (hereinafter omitted "one or more") are operably connected to the camera 12 (for example, at the head unit 18) to allow a user to manually position the camera 12. The handle 22 may include corresponding controls for activating or selecting specific features of the camera 12, such as focusing, zooming in, adjusting the amount / type of light projected onto the target site 16, and other features. Reference Figure 2 , the handle 22 may include a first handle 22A and a second handle 22B. Although the example shows two handles, it should be understood that the number and shape / form of the handle 22 may vary.

[0022] refer to Figure 1 The body 20 includes a head unit 18 that houses the camera 12, a robotic arm 24, and a coupling plate 26. The robotic arm 24 is configured to selectively move the camera 12 via the head unit 18. A user can position and orient the camera 12 with the assistance of the robotic arm 24. The coupling plate 26 mechanically couples the head unit 18 to the robotic arm 24.

[0023] refer to Figure 1 The robotic imaging system 10 includes a controller C having at least one processor P and at least one memory M (or a non-transitory tangible computer-readable storage medium) having recorded thereon a method 100 for executing the operating contact management mode 30 (hereinafter referred to as Figure 3 As described below, the contact management mode 30 utilizes redundant gravity / torque sensors to accurately detect potential contact of the body 20 on a reference axis (e.g., the Z axis) and reverse the body 20 out of potential contact. The memory M may store a controller executable instruction set, and the processor P may execute the controller executable instruction set stored in the memory M.

[0024] like Figure 1 to Figure 2 As shown, the system 10 includes a plurality of sensors 32, the plurality of sensors including at least one main sensor 34 and at least one auxiliary sensor 36, the at least one main sensor being configured to detect a corresponding force and / or a corresponding torque applied to the body 20, and the at least one auxiliary sensor being configured to detect a corresponding force and / or a corresponding torque applied to the handle 22. When a portion of the body 20 comes into contact with an external object (e.g., Figure 6When an object 400 in the image is contacted, the sensor readings from the primary sensor 34 and the auxiliary sensor 36 will change, thereby enabling the detection of an example contact position 402. This can be done using a weighted difference of the two sensor readings.

[0025] As described below, once contact is detected, the controller C defines a boundary plane B (see Figure 1 and Figure 6 ), and command the reverse trajectory (for example, Figure 6 In some embodiments, the robot arm 24 can move relatively quickly without user intervention. In other embodiments, the user may be required to keep the button engaged for movement to occur. The contact management mode 30 acts on the acceleration dynamics to mitigate the contact of various components of the body 20 with external objects in the surrounding environment. The method 100 described herein provides smooth transitions and is integrated into the dynamics of the control system.

[0026] The sensor 32 may include a six-degree-of-freedom tactile force sensing module. Here, the sensor 32 may detect translational forces or motions on the X-axis, Y-axis, and Z-axis, and separately detect torques or motions around the yaw axis, pitch axis, and roll axis. In addition to force sensors and torque sensors, linear accelerometers and rotational gyroscopes may be employed. The detected forces / movements are converted into rotational and / or translational data. The separation of translational and rotational forces enables the robotic imaging system 10 to calculate forward and / or inverse kinematics for controlling the robotic arm 24.

[0027] refer to Figure 2 , the main sensor 34 may be located at a center point 54 between the robot arm 24 and the camera 12. Figure 2 In the illustrated embodiment, there are two auxiliary sensors: a first auxiliary sensor 36A attached to the first handle 22A, and a second auxiliary sensor 36B attached to the second handle 22B. Figure 4 An alternative embodiment is shown in FIG. 2 having a single auxiliary sensor for the handle 222. Figure 4 The handle bar 223 connects the first handle 222A and the second handle 222B. Figure 4 , the auxiliary sensor is a single auxiliary sensor 236 rigidly attached to the handle bar 223. The number of sensors 32 may vary based on the application at hand.

[0028] The sensor 32 may include an optical sensor having an electro-optical device configured to convert the applied force and / or torque into an electrical signal. It should be understood that other types of sensor technologies may be employed. For example, the sensor 32 may include a strain gauge or a piezoelectric device configured to sense a tactile signal.

[0029] refer to Figure 1 , the robotic arm 24 includes one or more joints, such as a first joint 38 and a second joint 40, which are configured to provide a further degree of positioning and / or orientation of the head unit 18. Data from the sensor 32 can be used to determine which joints of the robotic arm 24 are to be rotated and how fast the joints should be rotated in order to provide auxiliary movement of the camera 12. Figure 1 , a corresponding joint motor (such as joint motor 41) and a corresponding joint sensor (such as joint sensor 42) can be connected to each joint. The joint motor 41 is configured to rotate the first joint 38 around the axis, and the joint sensor 42 is configured to transmit the position of the first joint 38 (in 3D space). Figure 2 , showing the joint compression point 43 in the joint 38.

[0030] The robotic arm 24 may include an assisted drive function in conjunction with a user-guided control system, thereby enabling the user to gently manipulate the camera 12 in a desired direction. In one embodiment, the user may grasp the handle 22 and actuate or press a release button. Actuation of the release button causes the camera 12 to transmit a message to the controller C indicating that the user desires to begin the assisted drive function. During this movement, the controller C causes the robotic arm 24 and / or the coupling plate 26 to move the camera 12 in a "power steering" manner, safely supporting its weight and automatically determining which joints should be activated and which joints should be braked in a coordinated manner to achieve the user's desired movement. In some embodiments, the robotic arm 24 may allow movement of the user without assistance, or with at least initial assistance.

[0031] refer to Figure 1 , the head unit 18 may be connected to a cart 44 having at least one display medium (which may be a monitor, terminal, or other form of two-dimensional visualization), such as Figure 1 The first display 46 and the second display 48 are shown. Figure 1, the controller C can be configured to process signals for broadcasting on the first display 46 and the second display 48. The body 20 can be independent and movable between different positions. The image stream from the camera 12 can be sent to the controller C and / or the camera processor (not shown), which can be configured to prepare the image stream for viewing. For example, the controller C can combine or interweave the first video signal and the second video signal from the camera 12 to create a stereo signal. The controller C can be configured to store the video signal and / or the stereo video signal in a video file stored in the memory M. The first display 46 and the second display 48 can include a stereoscopic display system, in which the two-dimensional display has separate images for the left eye and the right eye, respectively. In order to view the stereoscopic display, the user can wear special glasses, which are used in combination with the first display 46 and the second display 48 to display the left view to the user's left eye and the right view to the user's right eye.

[0032] refer to Figure 1 , the first display 46 can be connected to the cart 44 via a flexible robotic arm 50 having one or more joints to enable flexible positioning. The flexible robotic arm 50 can be configured to be long enough to extend above the patient during surgery to provide a relatively close view for the surgeon. The first display 46 and the second display 48 can include any type of display, such as a high-definition television, an ultra-high-definition television, smart glasses, a projector, one or more computer screens, a laptop computer, a tablet computer, and / or a smart phone, and can include a touch screen.

[0033] The camera 12 is configured to collect images of the target area 16, which can be presented in different forms, including but not limited to captured static images, real-time images and / or digital video signals. "Real-time" as used herein generally refers to updating information at the same rate as the received data. More specifically, "real-time" refers to collecting, processing and transmitting image data at a sufficiently high data rate and a sufficiently low latency so that when the data is displayed, the object moves smoothly without jitter or delay noticed by the user. Typically, this occurs when new images are collected, processed and transmitted at a rate of at least about 30 frames per second (fps), displayed at about 60fps, and when the delay of the combined processing of the video signal does not exceed about one-thirtieth of a second.

[0034] refer to Figure 1, the robot arm 24 (and / or the coupling plate 26) can be controlled via a controller C and / or an integrated processor (such as a robot arm controller 52). The robot arm 24 can be selectively operated to extend the viewing range of the camera 12 along the X-axis, Y-axis, and Z-axis. The robot arm controller 52 may include a processor, a server, a microcontroller, a workstation, etc., which is configured to convert one or more messages or instructions from the controller C into messages and / or signals that cause any one joint to rotate. The robot arm controller 52 is also configured to receive sensor information (such as joint position and / or velocity from the robot arm 24 and / or the coupling plate 26) and convert it into one or more messages for the controller C. U.S. Application No. 16 / 398,014 (filed on April 29, 2019) describes a stereoscopic visualization camera with an integrated robotic platform, the contents of which are hereby incorporated by reference in their entirety.

[0035] Reference now Figure 3 , shows the operation for Figure 1 Flow chart of an example method 100 for contact management mode 30. The method 100 may be implemented as a stored Figure 1 The method 100 is a computer readable code or instruction on and executable by the controller C of the present invention. The method 100 need not be applied in the specific order described herein, but can be performed dynamically. In addition, it should be understood that some steps can be eliminated. The method 100 can be performed periodically or at predetermined time intervals.

[0036] Method 100 from Figure 3 The controller C is programmed to receive sensor data from the sensor 32, including an input force vector and / or an input torque vector representing a detected force and a detected torque, respectively. In other words, the sensor 32 measures six degrees of freedom (F x 、F y 、F z , τ x , τ y and τ z ). In addition, according to box 102, the controller C receives the joint position data of the robot arm 24.

[0037] Go to Figure 3In box 104, the method 100 includes applying a correction factor to the sensor data. The sensor load varies with the orientation of its position on the body 20 relative to gravity and the acceleration or deceleration during the movement of the robot arm 24. The correction factor is applied to compensate for these effects. For example, the controller C is programmed to calculate gravity and apply gravity compensation by removing the effect of the earth's gravity from the sensor data. The gravity effect of each sensor can be removed in a world coordinate system (e.g., a robot base coordinate system) as (-mg), where m is the mass of the sensor and g is the gravity constant. Gravity calculation software available to those skilled in the art can be used. In addition, according to box 106, the method 100 may include filtering the sensor data, such as using a first low-pass filter, a second low-pass filter and / or a notch filter for cart vibrations. It should be understood that the order of the boxes can be changed.

[0038] Proceed to Figure 3 At block 106, the controller C is programmed to transform the respective sensor coordinate systems (of the sensors 32) to a common coordinate system (e.g., a camera coordinate system or a robot base coordinate system or another reference coordinate system). The sensors 32 are placed in the common coordinate system so that they can be compared. As described above, at least one auxiliary sensor 36 only loads the handle 22, while at least one main sensor 34 loads the body 20 including the coupling plate 26, the handle 22, and the camera 12. Figure 5 An example torque transformation is shown from an applied force / torque 300 (at a first location 302) to a measurement location 304 on a handle 322. A sensor 332 is attached to the handle 322. The sensor 332 provides a measured torque and a measured force at the measurement location 304, which coincides with the origin or center of the sensor 332.

[0039] refer to Figure 5 , the force applied at the first position 302 is the same as the force measured at the measurement position 304. The measurement position 304 and the first position 302 are connected by a displacement vector 306. The transformation can be characterized as:

[0040]

[0041] Here, I is a 3×3 identity matrix and 0 is a 3×3 zero matrix. Torque is the force (including static force, dynamic force, and working force) multiplied by the distance from the sensor origin (e.g., the measurement position 304) to the point where the force is applied (e.g., the first position 302). In addition, [d] xis a matrix representation of the cross product, which represents the product of a skew-symmetric matrix and a vector (see: https: / / en.wikipedia.org / wiki / Cross_product, section entitled “Conversion to matrix multiplication”). The controller C may be adapted to convert the torque (τ) applied at the first position 302 to 施加 ) is determined as the measured torque (τ 测量 ) and the applied force (F 施加 ) and the cross product of displacement vector 306(d), as follows:

[0042] [τ 施加 =τ 测量 -(F 施加 ×d)].

[0043] Go to Figure 3 In frame 108, when a portion of the body 420 contacts the external object 400 (e.g. Figure 6 ), the controller C is programmed to determine the contact position 402 based on the sensor data. Figure 6 Also shown in FIG. 4 is a moving path 404 of the body 420. The contact position 402 may be calculated with reference to a fixed coordinate system (eg, a camera coordinate system or a robot base coordinate system), which may be selected or varied based on the application at hand.

[0044] Proceed to Figure 3 In block 110, the method 100 includes defining a boundary plane B (see Figure 6 ) and calculates the reverse trajectory 406 of the body 420. The boundary plane B extends through the contact location 402 and is relative to the preselected first direction (e.g., Figure 1 The reverse trajectory 406 starts in a direction orthogonal to the boundary plane B. The controller C is adapted to calculate and provide a repulsive force that gently pushes the body 420 away from the contact position 402 in the reverse direction.

[0045] In some embodiments, the motion of the body 20 needs to move along a constrained trajectory. In these cases, it may not always be possible to directionally reverse the body 20 out of position. For example, the motion of the body 20 may be constrained to the surface of a virtual sphere. Figure 7500 is a schematic diagram of an example virtual sphere 500 representing the surface of the constraint trajectory. Here, a given point on the outer surface 502 of the virtual sphere 500 can be represented by the rotation sphere angles V and U. The view vector 508 of the camera 12 points to the center 504 of the virtual sphere 500, which has the same coordinates in XYZ space as the selected point in the target part 16. The controller C and / or the robot arm controller 52 can be adapted to convert 'up', 'down', 'left' and 'right' from camera coordinates to robot base coordinate system coordinates, which are provided as X vectors and Y vectors. The X vector and Y vector are used by the controller C and / or the robot arm controller 52 to directly determine how the camera 12 moves on the virtual sphere 500, thereby determining the end position 506 (with the view vector 510). Alternatively, the virtual sphere 500 can be represented by a different shape, and the outer surface 502 can be represented by a planar surface 512.

[0046] refer to Figure 6 , the reverse trajectory 406 can be constrained to the surface of the sphere 410. In order to maintain the constrained trajectory, the robot arm 24 is moved away in the reverse direction along the trajectory using the input force calculated by the closed-loop control module through feedback control. The closed-loop control module can be a proportional-integral controller, a proportional-differential controller, or a proportional-integral-differential controller. For example, using a proportional-differential controller, the input force (F_ 边界平面 ) can be obtained as Wherein, z represents the position of the body 420, represents the velocity of the reverse trajectory 406, and Z0 represents the position of the boundary plane B. It is assumed that the boundary plane B is perpendicular to the Z axis (first direction) at Z0. In addition, Kp and Kd are the proportional constant and the differential constant from the proportional-derivative controller, respectively, and the process variable is the difference between the boundary plane B and the current position (z). The constants Kp and Kd can be tuned via a calibration process where the changes in position and velocity are known. The controller C can inject a small amount of damping into the control system dynamics to improve smoothness and eliminate oscillations.

[0047] The sequence of obtaining the reverse trajectory 406 may include using inverse kinematics and / or Jacobian kinematics (e.g., inversion of the Jacobian matrix). The controller C determines a movement sequence that specifies how certain joints of the robot arm and / or the connecting plate 26 should move in a coordinated manner and specifies, for example, joint speeds, joint rotation directions, and / or joint rotation durations. The joints of the robot arm 24 and / or the connecting plate 26 may rotate individually or may have overlapping movements depending on the movement sequence. The Jacobian kinematics equations define how certain joints of the robot arm 24 and / or the connecting plate 26 will move based on proportionally adjusted translation / rotation vectors. Jacobian kinematics provides velocity control, while inverse kinematics provides position control. Other robot control routines may be employed.

[0048] Go to Figure 3 At block 112 in the method 100, the coordinates are transformed to a robot base coordinate system, which is a coordinate system on the cart 44 on which the base of the robot arm 24 is mounted. 边界平面 ) can be converted into a proportionally adjusted force (F 按比例调节 ),as follows:

[0049] [F 按比例调节 =α*Cam_To_Base*F 边界平面 ].

[0050] Here, α is an adjustable scale factor and Cam_To_Base is the 4×4 matrix coordinate transformation from the camera coordinate system to the robot base coordinate system. rev ) is the proportionally adjusted force (F 按比例调节 ) is projected onto the outer surface 502 (or the plane surface 512) of the virtual sphere 500. When the boundary plane B is cleared by the body 420, the repulsive force tends to zero, and the incremental step size of the reverse trajectory 406 tends to zero.

[0051] In systems with velocity or position based joint limits, the output force can be converted to a displacement vector with a scaling constant, projected onto the input direction vector, and added to the input direction vector at each update cycle, causing movement away from the limit. The controller C is programmed to perform this coordinate transformation using the joint position data in combination with the compensated filtered force / torque output data. The transformation can include one or more predefined equations or relationships based on the position and orientation of the sensor 32. The transformation can be based on parameters obtained via calibration.

[0052] Proceed to Figure 3 In block 114, controller C is programmed to verify the command (implementing reverse trajectory 406) to ensure that it is within the operating parameters of the joint motor (e.g., duration, speed, maximum current, etc.). Figure 3 At block 112, the controller C is programmed to transmit commands (via signals) to the appropriate joint motors of the robot arm 24 and / or the link plate 26 according to the reverse trajectory 406. The transmitted commands cause the motors at the corresponding joints to move the robot arm 24 and / or the link plate 26, thereby moving the camera 12 while avoiding collisions.

[0053] Figure 1 The controller C may include or otherwise access information downloaded from a remote source and / or executable program. Figure 1 , the controller C can be configured to communicate with a remote server 60 and / or a cloud unit 62 via a network 64. The remote server 60 can be a private or public information source maintained by an organization such as a research institute, a company, a university, and / or a hospital. The cloud unit 62 can include one or more servers hosted on the Internet to store, manage, and process data.

[0054] The network 64 may be a serial communication bus in the form of a local area network. The local area network may include, but is not limited to, a controller area network (CAN), a controller area network with flexible data rate (CAN-FD), Ethernet, Bluetooth, WIFI, and other forms of data. The network 64 may be a wireless local area network (LAN) that links multiple devices using a wireless distribution method, a wireless metropolitan area network (MAN) that connects several wireless LANs, or a wireless wide area network (WAN) that covers a large area such as adjacent towns and cities. Other types of connections may be used.

[0055] Figure 1 The controller C may be a component of the robotic imaging system 10, or a separate module operably connected to the robotic imaging system. The controller C includes a computer-readable medium (also referred to as a processor-readable medium), which includes a non-transient (e.g., tangible) medium that participates in providing data (e.g., instructions) that can be read by a computer (e.g., by a processor of a computer). Such media can take a variety of forms, including but not limited to non-volatile media and volatile media. Non-volatile media can include, for example, optical or magnetic disks and other persistent memories. Volatile media can include, for example, dynamic random access memory (DRAM), which can constitute a main memory. Such instructions can be transmitted through one or more transmission media, including coaxial cables, copper wires, and optical fibers, including wires that include a system bus connected to a processor of a computer. Some forms of computer-readable media include, for example, floppy disks, floppy disks, hard disks, magnetic tapes, other magnetic media, CD-ROMs, DVDs, other optical media, punch cards, paper tapes, other physical media with hole patterns, RAMs, PROMs, EPROMs, FLASH-EEPROMs, other memory chips or boxes, or other media that can be read by a computer.

[0056] The lookup tables, databases, data repositories, or other data stores described herein may include various mechanisms for storing, accessing, and retrieving various data, including hierarchical databases, a set of files in a file system, an application database in a proprietary format, a relational database management system (RDBMS), and the like. Each such data storage device may be contained in a computing device that employs a computer operating system (such as one of the above-mentioned operating systems) and may be accessed via a network in one or more of various ways. The file system may be accessed from the computer operating system and may include files stored in various formats. The RDBMS may employ structured query language (SQL), as well as languages ​​for creating, storing, editing, and executing stored programs, such as the PL / SQL language mentioned above.

[0057] The flowchart presented herein shows the architecture, function and operation of the possible implementation of the system, method and computer program product according to multiple different embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, a fragment or a code, which includes one or more executable instructions for implementing (multiple) specific logic functions. It should also be noted that each frame in the block diagram and / or the flow chart diagram and the combination of frames in the block diagram and / or the flow chart diagram can be implemented by a combination of a dedicated hardware-based device or dedicated hardware and computer instructions that perform a specified function or action. These computer program instructions can also be stored in a computer-readable medium, which can guide a controller or other programmable data processing device to work in a particular manner, so that the instructions stored in the computer-readable medium produce an article including instructions for implementing the function / action specified in the flow chart and / or the block diagram frame.

[0058] The numerical values ​​of parameters (e.g., amounts or conditions) in this specification (including the appended claims) are understood to be modified by the term "about" in each corresponding instance, regardless of whether "about" actually appears before the numerical value. "About" indicates that the numerical value is slightly imprecise (the value is somewhat close to precision; about or quite close to the value; almost close). If the imprecision provided by "about" is not otherwise understood as this ordinary meaning in the art, "about" as used herein at least represents the changes that may be caused by the common methods of measuring and using such parameters. In addition, the disclosure of the range includes the disclosure of each value within the entire range or the further divided range. Each value within the range and the endpoints of the range are thus disclosed as separate embodiments.

[0059] The specific embodiments and the drawings or figures are supportive and descriptive for the present disclosure, but the scope of the present disclosure is limited only by the claims. Although some of the best modes and other embodiments for implementing the claimed disclosure have been described in detail, there are various alternative designs and embodiments to practice the disclosure defined in the appended claims. In addition, the features of the embodiments shown in the drawings or the multiple different embodiments mentioned in this specification are not necessarily to be understood as independent embodiments of each other. Instead, each feature described in one of these examples of the embodiment can be combined with one or more other desired features from other embodiments to produce other embodiments that are not described in words or with reference to the drawings. Therefore, such other embodiments fall within the scope framework of the appended claims.

Claims

1. A robotic imaging system, comprising: a body including a head unit, a robotic arm, and a coupling plate coupling the head unit to the robotic arm, the robotic arm being adapted to selectively move the head unit; a camera positioned in the head unit and configured to record one or more images of a target site, the camera operably connected to one or more handles; A plurality of sensors configured to transmit sensor data, the plurality of sensors comprising: at least one primary sensor configured to detect a respective force and / or a respective torque applied at the body; and at least one auxiliary sensor configured to detect the respective force and / or the respective torque applied at the one or more handles; a controller configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory having instructions recorded thereon; and Wherein the controller is adapted to execute a contact management mode comprising determining a contact location of the body with an external object based in part on the sensor data and determining a reverse trajectory for moving the body away from the contact location.

2. The robotic imaging system of claim 1, wherein: The camera is a stereo camera configured to record a left image and a right image for generating at least one stereo image of the target site.

3. The robotic imaging system of claim 1, wherein: The plurality of sensors respectively include six-degree-of-freedom tactile force sensing devices.

4. The robotic imaging system of claim 1, wherein: The target site is the eye.

5. The robotic imaging system of claim 1, wherein: The one or more handles include a first handle and a second handle rigidly attached to and connected by a handle bar, and the at least one auxiliary sensor is a single sensor positioned on the handle bar.

6. The robotic imaging system of claim 1, wherein: The one or more handles include a first handle and a second handle, and the at least one auxiliary sensor includes a first auxiliary sensor attached to the first handle and a second auxiliary sensor attached to the second handle.

7. The robotic imaging system of claim 1, wherein: The at least one main sensor is located at a center point between the robot arm and the camera.

8. The robotic imaging system of claim 1, wherein: The plurality of sensors define respective reference coordinate systems, and the controller is adapted to transform the respective reference coordinate systems into a common coordinate system.

9. The robotic imaging system of claim 1, wherein: The controller is adapted to apply a correction factor to the sensor data, including applying a low pass filter.

10. The robotic imaging system of claim 1, wherein: The controller is adapted to apply a correction factor to the sensor data, including gravity correction for the plurality of sensors based on respective positions of the plurality of sensors.

11. The robotic imaging system of claim 1, wherein: The controller is adapted to obtain a boundary plane extending through the contact location and perpendicular with respect to the first direction.

12. The robotic imaging system of claim 11, wherein: The reverse trajectory starts in a direction orthogonal to the boundary plane.

13. The robotic imaging system of claim 12, wherein: The reverse trajectory is constrained to the surface of a virtual sphere.

14. The robotic imaging system of claim 1, wherein: The sensor data provides a measured torque and a measured force at a measurement location, the measurement location and the contact location being connected by a displacement vector; and The controller is adapted to determine the applied torque at the contact location as a difference between the measured torque and a cross product of the applied force and the displacement vector.

15. A robotic imaging system, comprising: a body including a head unit, a robotic arm, and a coupling plate coupling the head unit to the robotic arm, the robotic arm being adapted to selectively move the head unit; a camera positioned in the head unit and configured to record one or more images of a target site, the camera operably connected to one or more handles; A plurality of sensors configured to transmit sensor data, the plurality of sensors comprising: at least one primary sensor configured to detect a respective force and / or a respective torque applied at the body; and at least one auxiliary sensor configured to detect the respective force and / or the respective torque applied at the one or more handles; a controller configured to receive the sensor data, the controller having a processor and a tangible, non-transitory memory having instructions recorded thereon; and Wherein, the controller is adapted to execute a contact management mode, comprising: obtaining a contact position of the body with an external object based in part on the sensor data; obtaining a boundary plane extending through the contact location, the boundary plane being perpendicular to a reference axis; and A reverse trajectory is determined for moving the body away from the contact location.

16. The robotic imaging system of claim 15, wherein: The one or more handles include a first handle and a second handle rigidly attached to and connected by a handle bar, and the at least one auxiliary sensor is a single sensor positioned on the handle bar.

17. The robotic imaging system of claim 15, wherein: The one or more handles include a first handle and a second handle, and the at least one auxiliary sensor includes a first auxiliary sensor attached to the first handle and a second auxiliary sensor attached to the second handle.

18. The robotic imaging system of claim 15, wherein: The reverse trajectory is constrained to the surface of a virtual sphere.

Images