Sensing in continuous robot

Abstract

To provide means that is improved and accurate means that senses and determines a position of a robot arm, the method requiring no calibration that requires much time.SOLUTION: Provided is a sensing system for a continuous arm robot. The sensing system includes at least one camera ring system mounted to the continuous robot. The camera ring system includes at least two cameras facing forward and directed to a distal end along an axis of the robot.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a means for visually sensing the position of a continuum robot, in particular to a ring camera system used to sense and determine the position of a continuum robot system. [Background technology]

[0002] Continuum arm or snake arm robots are of growing interest in many applications because they can be maneuvered into spaces that are not easily accessible to other robotic systems or human operators. This is due to the ability to manipulate the body with many degrees of freedom so that end tools can be positioned precisely and easily. This positioning is controlled by actuators that manipulate tendons within the robot so that each joint of the arm can be individually controlled with high positional accuracy.

[0003] Most continuum or snake arm robots have six or fewer degrees of freedom. However, when a task requires a higher degree of dexterity, the number of degrees of freedom required is increased. In such cases, the number of degrees of freedom needs to be increased to seven to nine. This increase in degrees of freedom means that the arm can operate in a limited area, for example in the repair of complex structures or for use in minimally invasive surgery. Continuum arm robots are designed along two main lines: first, there are snake-type robots that consist of multiple rigid link sections connected by either rigid R / U / S (revolute / swivel / ball) joints or compliant joints. Each section is composed of one or more fragments and is controlled independently of the others by on-board or remote actuation. Second, there are continuum robots that consist of a compliant backbone whose local and global deformations are controlled by one or more actuators.

[0004] However, one of the problems with using continuum robots is that tip position is difficult to detect. This limits the usefulness of the robot to know the location of the robot parts. This is because tip control is key to be able to use robots in confined spaces or in delicate or fragile areas, as contact between the robot arm and the side can potentially result in damage. In the prior art, string gauges are placed at predefined locations on the robotic arm, where readings from the gauges are used to measure changes in distance between points. From these measurements calculations can be performed and geometry can be used to determine the position of the end effector. In another example, the tip of a bionic arm is connected to the tip of a Kuka robot, and by using the conjugate robot movements, the time domain Cartesian position of the tip can be provided by Kuka Control Software (KSS). However, these are complex systems that need to be calibrated to determine position. A less complex method is to use an electromagnetic (EM) position tracking system. In this case, a tracking coil is attached to the distal tip of the robot arm and can detect the position and orientation of the end effector. Alternatively, vision-based measurement systems can be used, but these require calibration or have limited accuracy. Therefore, there is a need to provide an improved and accurate means of sensing and determining the position of a robotic arm that does not require extensive time consuming calibration. Summary of the Invention [Means for solving the problem]

[0005] According to a first aspect of the present disclosure, there is provided a sensing system for a continuum-arm robot, the sensing system comprising at least one camera ring system mounted to the continuum robot, the camera ring system having at least two forward-facing cameras pointing along an axis of the robot toward a tip end.

[0006] An additional camera system may be mounted at the end of the robotic arm. At least two camera ring systems may be mounted around the periphery of the continuum-arm robot, the camera rings having optical markers on the rear-facing side of the camera ring system that are within the field of view of one or more of the forward-facing cameras.

[0007] At least one of the camera ring systems may be provided with at least two laterally facing cameras oriented substantially perpendicular to an axis extending along the body of the continuum arm robot.

[0008] Each camera ring system may be provided with a side-facing camera. There may be two front-end cameras.

[0009] A camera ring system may be incorporated into the robot. The camera ring system may be provided with a clamping mechanism so that the camera ring system can be added retroactively to the continuum robot.

[0010] The camera ring system may have a thickness of 2 to 10 mm. There may be 2-6 forward-facing cameras per ring system and 2-6 optical markers on the ring system.

[0011] The tip may further be provided with an LED lighting system. At least one of the camera ring systems may be provided with an LED lighting system.

[0012] The sensing system may be coupled to a computer that is used to process images from the camera system to determine the shape and position of the continuum robot. Images from the camera system can be displayed on a monitor output for a computer.

[0013] According to a second aspect of the present disclosure, there is provided a continuum-arm robot having the above sensing system.

[0014] According to a third aspect of the present disclosure, there is provided a method of operating a continuum-arm robot having a sensing system according to the above discussion, the method comprising: Inserting a continuum arm robot into an operating area; recording images of the continuum arm robot operating within the working envelope using cameras of a camera ring system; processing the images to determine a shape of the continuous-arm robot; Includes.

[0015] As will be appreciated by those skilled in the art, unless mutually exclusive, any feature described in relation to any one of the above aspects may also be applied mutatis mutandis to any other aspect. Further, unless mutually exclusive, any feature described herein may be applied to any aspect and / or may be combined with any other feature described herein. Embodiments will now be described, by way of example only, with reference to the figures. [Brief description of the drawings]

[0016] [Figure 1] Figure 1a shows a prior art example of a cutaway view of a continuum arm robot; Figure 1b shows an example of a joint of a continuum arm robot. [Diagram 2] FIG. 1 illustrates an example of a robotic system according to the present disclosure. [Diagram 3] FIG. 1 illustrates an example of an advanced camera system according to the present disclosure. [Figure 4] FIG. 2 illustrates an example of a forward-facing camera in a camera ring system according to the present disclosure. [Diagram 5] FIG. 2 illustrates an example of a side-facing camera in a camera ring system according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Aspects and embodiments of the present disclosure will now be described with reference to the accompanying drawings, and further aspects and embodiments will be apparent to those skilled in the art.

[0018] FIG. 1a shows a prior art example of a cutaway view of a continuum arm robot. The prior art continuum arm robot comprises a continuum arm robot portion 101 that is permanently integrated and extends from an actuator pack 102. The actuator pack 102 contains a number of independent actuators 103. These actuators are used to adjust tension in tendons that extend through the continuum arm 101. The tendons are associated with joints in the arm, each of which is designed to move in response to tensioning or relaxing the tendons associated with the joint. This tensioning or relaxing of the tendons thus causes the joint to contract or extend, which allows the continuum arm to bend. The actuator pack is shown positioned on a rail or support 104 that is positioned near the component to be inspected. The actuators are also provided with a number of power and signal cables 105 that are used to power and address the actuators. Individual signals throughout the range of the actuators provide control of the joints so that the continuum arm 101 can be commanded. Not shown in Figure 1 is the need for an operator to control the movement of the continuum arm and perform the desired task using a computing device linked to the actuator. The computing device connected to the prior art actuator may be any suitable computing system, such as a laptop computer, featuring the requisite operating software for the robot and a control input, such as a joystick, that allows the continuum arm to be controlled.

[0019] FIG. 1b shows an example of a continuum arm robot joint. The arm has multiple joints requiring at least two cables per joint. For example, a system with three joints, each with four tendons per joint, would require 12 actuators to actuate. To increase the number of joints, the number of actuators would need to be increased or the number of tendons per joint would need to be reduced. The highlighted joints 106, 107, 108 can be manipulated to move in three dimensions. The joints are configured such that joints 106 and 108 can bend in the same plane relative to the center of the arm, while the plane in which joint 107 can move is shifted by 90° relative to joints 106 and 108. It is this repeating configuration of alternating joint angles, each of which results in movement in a different orthogonal plane, throughout that allows the arm to be manipulated in three dimensions. Each joint in the arm has a limit to the amount they can bend, which is dictated by the design of the arm and the materials used. The bending limits at each joint set requirements for arm properties such as minimum bending radius and torque required to cause a resulting change in the joint. It is the presence of space at the joints that allows the joints to move and the ease of movement of the joints resulting in low stiffness of the arm in comparison to other robotic arms of the same length. This is because the structural behavior of the snake robotic manipulator can be likened to a cantilever beam under load, since the system is fixed to a base with an actuation puck at one end and the remainder of the arm is used to navigate through the environment without other points of contact. In this situation, any load applied to the body and / or tip of the snake robot, including the weight of the snake robot itself, imposes significant deflections from the ideal position. At the end of the arm is located an instrument or probe that is designed to perform one or more functions once the continuum arm is in place. Control cables, power connectors for the instruments can run through the center of the joint in the continuum arm.This has the benefit of protecting the cable from any potential damage.

[0020] FIG. 2 shows a continuum arm robot with a vision-based sensing system according to one embodiment of the present disclosure. The use of a camera-based system is a way to overcome problems associated with the prior art. This is because it provides a means to overcome the problem of accurately monitoring the position of the robotic tip in real time. An additional advantage of this is that there is no requirement for pre-calibration of the robot, which may present accuracy issues with cable-based systems. The continuum arm 201 is connected to an actuator mechanism 202. The actuator mechanism is used to control the actuator cables in the continuum robot. The imaging system for the continuum arm robot may comprise a forward-facing camera system 203 mounted on the continuum arm robot. The forward-facing camera system is positioned at the tip of the continuum arm robot and is responsible for observing the view along the axis of the body of the continuum robot. Similar to a snake, the direction of the tip or head of the robot is considered to be facing forward. The imaging system for the continuum robot comprises at least one camera ring system 204 mounted on the continuum robot. FIG. 2 shows a system with multiple such camera ring systems. The camera ring system is responsible for the view along the body of the continuum robot. Additionally, the ring camera system may be equipped with optical markers so that the ring camera system can record the position of the camera ring system in front of it as it travels along the continuum robot from the actuator to the tip. This information can be used to accurately determine the shape of the robot at any time. Additionally, information from the cameras can be used to help detect and locate the continuum robot within the working environment. The camera ring system may also be equipped with side-facing cameras.

[0021] FIG. 3 shows an expanded example of a tip monitoring system according to the present disclosure. A robotic arm 301 extends from a deployment mechanism 302. Tip section 303 features a camera system 304. The camera system may comprise one or more cameras. The camera may be any suitable image sensor system. For example, it may be a charge coupled device chip or an active pixel sensor, either of which may have associated circuitry and a lens system. Alternatively, it may be part of a fiber optic based camera system. Alternatively, the camera may be a LiDAR or infrared sensor. The camera may be positioned to point along the axis of the arm. Alternatively, the camera may be positioned to point at an angle to the axis of the continuum arm. If multiple cameras are used as part of the camera system, the cameras may point in the same direction. Alternatively, the camera system may be configured such that one of the cameras points along the axis of the robotic arm and one of the cameras is angled to the axis. If the camera is angled to the axis, this may be at any suitable angle. For example, this may be 60-120 degrees to the axis. The tip camera system may work in conjunction with markers on the area of ​​the workpiece to be inspected / worked upon. Additionally or alternatively, markers can be used along with known reference points in the workspace. The presence of these markers or known points is used to allow the position of the tip to be precisely determined by an operator and / or a computer program. If multiple cameras are used in the camera system, techniques such as stereo imaging can be used to determine the position and distance of the tip relative to known points in the workspace.

[0022] FIG. 4 shows an example of a forward facing vision system for an arm section of a robot according to the present disclosure. A camera ring system 401 is formed as part of a ring system, which can be incorporated into the device when the robot is built. Alternatively, the camera ring system can be added retroactively to the robot arm. Retroactive addition can be achieved by fastening a camera ring to the outer surface of the continuum arm robot at an appropriate point along its length. Retroactive addition can be achieved using an interlocking feature between the robot and the ring. Alternatively, retroactive addition can be performed by friction or compression by the ring body against the robot body. The compression or friction fit can be achieved using a clamping mechanism. The clamping mechanism can be mechanical using a cam clamp or a screw clamp. Alternatively, the clamping mechanism can be magnetic. The ring system can be located in any suitable location. For example, it can be at the start of the active section or midway through the active section. Alternatively, it can be at the start of the active section with a second camera ring midway through the active section and a third camera ring near the tip of the active section. The position of the ring is determined by the number of camera rings used, the length of the active section, and the required accuracy that needs to be achieved by the system for accurate operation. Increasing the number of camera rings increases the accuracy of the system's position information since the bending / bending of the robotic arm can be determined more accurately. The forward facing camera ring may be fitted with two or more cameras 402. More preferably, there are 2-6 cameras in each camera ring. The camera may be any suitable image sensor system. For example, this may be a charge coupled device chip or an active pixel sensor, either of which have their associated circuitry and lens system. Alternatively, the camera may be part of a fiber optic based camera system. Alternatively, the camera may be a LiDAR or infrared sensor. The camera is mounted within a protrusion from the ring. On the opposite side of the ring to the camera is an optical marker 403.The ring closest to the tip of the arm / actuator end may not have a camera, but may have an optical marker on it that is visible to the camera ring closer to the actuator.

[0023] FIG. 5 shows an example of a side-facing camera system according to an example of the present disclosure. A side-facing camera 504 is included in the same ring as a forward-facing camera 502. More than one side-facing camera may be mounted on the ring. The side-facing cameras are used to image the surroundings of the robot arm. The side-facing cameras may be symmetrically arranged around the ring body. The presence of the cameras allows the robot operator to clearly see the area in which the robot is operating, which may prevent the robot from touching the sides of the workspace, or may help the operator navigate the robot around potential barriers or accurately position the robot to perform a desired task. Side-facing cameras may be provided on all camera ring systems. Alternatively, side-facing cameras may only be present on alternative camera ring systems. The camera may be any suitable image sensor system. For example, it may be a charge-coupled device chip or an active pixel sensor, either of which have their associated circuitry and lens systems. Alternatively, the camera may be a LiDAR or infrared sensor. When both forward and side-facing cameras are present, the front-looking probe camera on the body ring provides feedback on the tip position of each section relative to the base plane of that section. The side-looking camera on the body ring is used to locate the robot within the working environment while locating the location of each tip. These two measurements are mutually compensated to give continuous measurements of tip position for each section.

[0024] The camera system may be linked to a computer system capable of processing images from the camera. The computer system may be the same as the system connected to the actuator pack to control the continuum arm robot. Images from the camera may be transmitted via a cable that may run along the outside or inside of the robot arm. Alternatively, signals may be transmitted to the computer using wireless signals. The computer may be provided with hardware to receive signals from the camera mounted on the snake arm. The computer may be equipped with a computer program capable of processing signals transmitted to the computer from the camera. The software may be capable of processing the distance between the optical markers and the camera. Using these determined distances and displacements between the camera and the optical markers, the program may calculate the curvature and flexion of the robot arm. The curvature and flexion of the robot arm may be used to either control input or perform a task within a desired position. Thus, the arm may follow a known programmed sequence of movements, which allows it to enter a workspace and position itself to perform its task. Positional flexion may also be used to trigger an alarm if the flexion is greater than required for a given task. The computer program also allows the operator to see the camera signal in real time. Review can be done on a camera-by-camera basis, or the pilot can select one or more camera image feeds to review.

[0025] The camera ring sensing system design provides a means to monitor the shape of the robot arm while also providing information about the tip position. Knowledge of the position can be used to compensate for errors arising from theoretical models and random uncertainties used in the control algorithm. The body camera ring and tip camera can also be mutually compensated for more accurate measurements. The compact on-body structure with the camera ring that can protrude less than 10 mm from the outer diameter of the Snake robot allows the Snake to access confined spaces in continuum robotics and provide real-time in-process shape / position information along the arm. Furthermore, compared to other high-resolution self-sensing solutions such as fiber Bragg grating system sensors, the probe camera is a relatively economical solution.

[0026] In the above example, the system is a continuum robot, but the same principles can be applied to a borescope or other maneuverable compliant arm robotic system.

[0027] It is understood that the present invention is not limited to the above-described embodiments, and various modifications and improvements can be made without departing from the concepts described herein. Any feature can be used individually or in combination with any other feature, except where mutually exclusive, and the present disclosure extends to and includes all combinations and subcombinations of one or more features described herein. [Explanation of symbols]

[0028] 101 Continuum arm robot part, continuum arm 102 Actuator Pack 103 Actuator 104 Rails or supports 105 Power and signal cables 106,107,108 Joints 201 Continuum Arm 202 Actuator Mechanism 203 Forward Facing Camera System 204 Camera Ring System 301 Robot Arm 302 Deployment mechanism 303 Tip Section 304 Camera System 401 Camera Ring System 402 Camera 403 Optical Marker 502,504 Cameras

Claims

1. A sensing system for a continuum arm robot, comprising at least two camera ring systems mounted on the continuum robot, wherein each camera ring system has at least two forward-facing cameras facing toward the tip along the axis of the robot, and each of the at least two camera ring systems has an optical marker on the rear-facing side of the camera ring system, wherein the optical marker is within the field of view of one or more of the forward-facing cameras.

2. The sensing system according to claim 1, wherein an additional camera system is attached to the tip of the robotic arm.

3. The sensing system according to claim 1, wherein at least one of the camera ring systems is provided with at least two side-facing cameras oriented substantially perpendicular to an axis extending along the body of the continuous arm robot.

4. The sensing system according to claim 3, wherein each camera ring system is provided with a camera facing to the side.

5. The sensing system according to claim 1, wherein the system has two cameras at the front.

6. The sensing system according to claim 1, wherein the camera ring system is incorporated into the robot.

7. The sensing system according to claim 1, wherein the camera ring system is provided with a clamping mechanism so that the camera ring system can be retrospectively added to a continuum robot.

8. The sensing system according to claim 1, wherein the camera ring system has a thickness of 2 to 10 mm.

9. The sensing system according to claim 1, wherein each ring system has two to six forward-facing cameras, and the ring system has two to six optical markers.

10. The sensing system according to claim 1, wherein an LED lighting system is further provided at the tip.

11. The sensing system according to claim 1, wherein at least one of the camera ring systems is provided with an LED lighting system.

12. The sensing system according to claim 1, wherein the sensing system is coupled to a computer used to process images from the camera system and determine the shape and position of the continuum robot.

13. A continuous arm robot having the sensing system according to any one of claims 1 to 12.

14. A method for operating a continuous arm robot having a sensing system according to any one of claims 1 to 12, The steps include inserting the aforementioned continuous arm robot into the operating area, The steps include recording images of the continuous arm robot operating within the operating area using the camera of the camera ring system, The steps include: processing the aforementioned image to determine the shape of the continuous arm robot; A method that includes this.

Images