Fiber-optic visual-tactile fusion rotating mechanical claw system

Through the rotating mechanical claw system that integrates fiber-optic vision and tactile, using lidar and fiber-optic haptic sensor combined with rotary connectors, the problems of complex grasping process and slow signal transmission speed in the existing technology are solved, and efficient and stable Optical signal transmission and object capture enhance the system's application range and anti-interference ability.

CN115503007BActive Publication Date: 2024-11-22TIANJIN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211316701.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-26
Publication Date
2024-11-22
Estimated Expiration
2042-10-26

AI Technical Summary

Technical Problem

The existing visual tactile fusion system has complex grasping process, slow response speed, and tactile sensors are susceptible to electromagnetic interference, and the transmission problem of optical signals during rotation has not been effectively solved, which cannot meet the needs of high-speed and large-capacity data transmission.

Method used

The rotating mechanical claw system is adopted that integrates fiber-optic vision and tactile vision, and the optical tactile sensor replaces the electrical sensor, and the stable transmission of optical signals is achieved through the rotating connector to ensure low loss of information during rotation. High-speed transmission.

Benefits of technology

It improves the application range and transmission efficiency of the system, achieves faster object grabbing and higher information capacity, while reducing the system size and enhancing the resistance to electromagnetic interference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115503007B_ABST
    Figure CN115503007B_ABST
Patent Text Reader

Abstract

The present invention discloses a rotating mechanical claw system with fiber-optic visual-tactile fusion, comprising a mechanical claw, a support plate, a rotating connector and a host computer, wherein a hollow turntable is installed on the support plate, and the rotating connector comprises a rotor end and a stator end, wherein the rotor end and the stator end are corresponding to a plurality of optical fiber transmission channel groups; the mechanical claw comprises a mechanical claw motor and a plurality of mechanical claw fingers, and the mechanical claw is connected to the hollow turntable through a bottom plate, and the upper end of the hollow turntable is connected to the rotor end of the rotating connector; optical fiber tactile sensors respectively connected to the rotor ends are installed on the inner sides of the mechanical claw fingers, and the channels corresponding to the stator ends are all connected to a grating demodulator or a spectrometer; a laser radar is fixed to the lower end of the bottom plate of the mechanical claw through a U-shaped plate, and the laser radar is connected to the rotor end through an optical fiber, and the output optical fiber corresponding to the stator end is connected to an acquisition card through a photoelectric converter. The main function of the system is to realize accurate grasping of objects and ensure stable transmission of optical signals during the rotation of the mechanical claw.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a visual-tactile fusion system based on the optical fiber perception principle, and in particular to a rotating mechanical claw system of optical fiber visual-tactile fusion. Background Art

[0002] With the research of sensor technology and artificial intelligence theory and technology, people pay more attention to the interaction ability of robots with the outside world. Touch and vision are important perception parts of human-computer interaction. A single way of obtaining information is no longer sufficient to respond to the effects of the external environment. In order to make up for the shortcomings of single vision or touch, touch and vision are integrated to better improve the robot's cognition and discrimination ability, which is suitable for more different application scenarios.

[0003] The patent document with the patent publication number CN 114851227A and the publication date of August 5, 2022 discloses a device based on the fusion perception of machine vision and touch. It mainly includes a visual processing module, a tactile processing module, an object recognition module and a robot action module. The image data is acquired by an RGBD camera, and the image information is converted into an electrical signal for transmission. The visual data is processed by the image preprocessing module and the deep learning model. The contact of the object is achieved by the robotic arm through coordinate transformation, and the tactile sensor is combined for tactile exploration to extract more object features. Finally, deep learning or machine learning methods are used to fuse multi-source heterogeneous information for object identification. However, the device has a complex grasping process and a slow response speed, and the specific use principle of the tactile sensor is not proposed.

[0004] The patent document with the publication number CN111055279A and the publication date of April 24, 2020 discloses a multimodal object grasping method and system based on the combination of touch and vision. The method uses a camera for image acquisition and a six-axis force sensor based on electromagnetic theory for tactile information acquisition. The movement and rotation angle of the end of the robotic arm are solved by the inverse kinematics of the rotating structure, and the robotic arm and the mechanical claw are used to complete the object grasping and rotation action. The main purpose of the entire system is to simulate the process of the robot grasping the target object in a real scene. The system focuses on the analysis of the object grasping process and improves the accuracy of grasping the object through the fusion of visual and tactile information. However, in the overall process, the robot arm motion data requires a series of conversions, the execution process is cumbersome, the grasping efficiency is slow, and the tactile sensor used is easily interfered by external electromagnetic signals.

[0005] The patent document with the patent publication number CN 114700947A and the publication date of July 5, 2022 discloses a robot based on visual-tactile fusion and its grasping system and method. Visual information is obtained by using a visual camera, and tactile information is obtained by using tactile sensors such as GelSight based on optical characteristics. The robot dexterous hand platform is used to control the movement and rotation of the end effector to achieve the movement and rotation of the grasped object. The main purpose of the system is to use the new features obtained after the fusion of visual and tactile information to predict the probability of slippage during the grasping process of the object, and to ensure stable grasping of the object with appropriate force by continuously adjusting the force of grasping the object. The optical tactile sensors such as GelSight mentioned in the patent are tactile sensors designed based on the principle of optical imaging. They transmit the surface morphology of the object to the terminal in the form of an image by contacting with the object. The information processing calculation is large, and the imaging system is large, resulting in a large volume of the tactile perception system. In addition, the entire system does not consider the problem of rotational transmission of optical signals during the rotation of the end effector.

[0006] The patent document with the patent publication number of US20220143835A1 and the publication date of May 12, 2022 discloses a small robot with a visual tactile perception device, including a visual perception module, a tactile perception module, a control module and a signal transmission module. In this invention, the visual module uses a camera for data acquisition, and the tactile perception module uses an electrical sensor and a digital-to-analog conversion circuit to obtain tactile information. During the operation of the entire device, both modal information is transmitted as electrical signals, and the rotation of the robotic arm is not involved.

[0007] In summary, for visual-tactile fusion perception technology, most patents use cameras to collect images to obtain visual information, and convert image information into electrical signals for transmission in the line; most commonly used tactile sensors are based on electrical principles. However, with the development of information technology, the transmission capacity and rate of signal data are gradually increasing, and electrical signal transmission can no longer meet the requirements of current information perception and information transmission. In comparison, the signal transmission capacity and speed of optical signals are higher than those of electrical signals, which facilitates high-speed data transmission and large-capacity data transmission. Since optical fibers are small in size and easier to integrate than other transmission media, tactile sensors based on the principle of optical fiber perception have great potential application value. At present, the movement requirements for mechanical claws generally involve multiple degrees of freedom, not only grasping and translation, but also rotation. Based on the above problems and requirements, the present invention proposes a new optical vision and tactile fusion method, and both types of sensors adopt optical fiber structures. In order to ensure that the perceived optical signal can be stably transmitted during the rotation process, full-range optical signal transmission is required, so an optical fiber rotary connector is added to the system to realize optical signal rotation transmission. Summary of the invention

[0008] In view of the above-mentioned prior art, the present invention provides a fiber-optic visual-tactile fusion rotating mechanical claw system, the main function of which is to achieve accurate grasping of objects while ensuring stable transmission of optical signals during the rotation of the mechanical claw.

[0009] In order to solve the above technical problems, the present invention proposes a fiber-optic visual-tactile fusion rotating mechanical claw system, including a mechanical claw, a displacement lifting platform, a support plate and a host computer, wherein a hollow turntable driven by a turntable motor is installed on the support plate, and the support plate is provided with a through hole which penetrates the center hole of the hollow turntable, the mechanical claw includes a mechanical claw bottom plate, the lower end of the mechanical claw bottom plate is connected to a plurality of synchronously moving mechanical claw fingers which are driven by the mechanical claw motor and driven by a connecting rod mechanism, the upper end of the mechanical claw bottom plate is connected to the hollow turntable through a plurality of first studs, and the upper end of the hollow turntable is connected to the rotor end of a rotary connector through a plurality of second studs; the rotary connector includes multiple channels, and fiber-optic tactile sensors are installed on the inner side of the mechanical claw fingers, and all fiber-optic tactile sensors Each of them is connected to a channel at the rotor end of the rotary connector through an optical fiber, and the output optical fiber at the stator end of the rotary connector corresponding to the channel occupied by each fiber optic tactile sensor is connected to a grating demodulator or a spectrometer; a laser radar is fixed to the lower end of the mechanical claw base plate through a screw, and the laser radar is installed at a position where the laser direction coincides with the central axis of the mechanical claw, and the laser radar is connected to a channel at the rotor end of the rotary connector through an optical fiber, and the output optical fiber on the stator end of the rotary connector corresponding to the channel occupied by the laser radar is connected to the acquisition card through a photoelectric converter; the driving motor, turntable motor, mechanical claw motor, acquisition card, grating demodulator or spectrometer of the displacement lifting platform are all connected to the host computer.

[0010] Furthermore, the optical fiber type visual-tactile fusion rotating mechanical claw system of the present invention, wherein:

[0011] The installation structure of the laser radar is: the bottom plate of the mechanical claw is connected to the upper side plate of a U-shaped adapter plate with a horizontal opening through screws, and the laser radar is installed on the lower side plate of the U-shaped adapter plate at a position where the laser direction coincides with the central axis of the mechanical claw.

[0012] The optical fiber tactile sensor is arranged at the fingertip of the mechanical claw finger.

[0013] The optical fiber tactile sensor is an optical fiber Bragg grating sensor, an optical fiber Mach-Zehnder sensor, or an optical fiber Fabry-Perot sensor. The grating optical fiber sensor is an optical fiber Bragg grating or a long period grating.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] Compared with traditional image vision, laser radar ranging has a faster response speed. At the same time, compared with electrical sensors, fiber optic tactile sensors are resistant to electromagnetic interference, small in size, easy to integrate, have a high transmission rate, and a large information capacity. This system not only integrates vision and touch, but also because the information transmitted by the entire system is an optical signal and the system requires rotation, a rotary connector is added to the system for the first time, which not only increases the axial rotation of the entire system, but also ensures the stable transmission of optical signals during rotation, thereby increasing the application range of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a schematic diagram of the structure of the optical fiber visual-tactile fusion rotating mechanical claw system of the present invention;

[0017] Figure 2 A schematic diagram of the installation structure of the laser radar in the present invention;

[0018] Figure 3 A schematic diagram of the types of optical fiber tactile sensors described in the invention;

[0019] Figure 4 This is a working flow chart of the fiber-optic visual-tactile fusion rotating mechanical claw system of the present invention.

[0020] In the figure:

[0021] 1- stator end of rotary connector 2- rotor end of rotary connector 3- turntable motor

[0022] 4-Support plate 5-Mechanical claw motor 6-Mechanical claw finger

[0023] 7- Fiber optic tactile sensor 8- First stud 9- Hollow turntable

[0024] 10-Second stud 11-Mechanical claw bottom plate 12-LiDAR

[0025] 13- Object to be grasped 14- Lifting platform 15- U-shaped adapter plate

[0026] 16-Screw DETAILED DESCRIPTION

[0027] The design idea of ​​a fiber-optic visual-tactile fusion rotating mechanical claw system proposed in the present invention is: using two types of fiber-optic sensors to realize visual and tactile fusion perception, wherein the laser radar replaces the traditional image vision for visual perception, and the fiber-optic tactile sensor replaces the electrical tactile sensor to obtain tactile information. The laser radar is mainly used to sense the distance of the object to be grasped, and the main purpose of the fiber-optic tactile sensor is to obtain the tactile information of the object to be grasped, and then control the force of grasping the object. The fiber-optic tactile sensor is fixed on the inner side of the fingertip of the mechanical claw finger to obtain the information of the object, realize tactile perception, and accurately control the grasping of the object. At the same time, the rotation action in the current mechanical claw system is mainly based on a multi-degree-of-freedom mechanical arm, and the transmission and rotation of the signal are independent of each other. In the rotating mechanical claw system of the present invention, due to the addition of a rotary connector, the optical signal is ensured to be stably transmitted while rotating. The system of the present invention realizes visual-tactile fusion based on an all-fiber structure, and has a higher transmission rate and a larger transmission capacity, while its volume is smaller; at the same time, it ensures the low-loss rotation transmission of the optical signal, and has a wider range of applications.

[0028] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention in any way.

[0029] like Figure 1As shown, a fiber-optic visual-tactile fusion rotary mechanical claw system proposed by the present invention includes a mechanical claw, a displacement lifting platform 14, a support plate 4 and a host computer. A hollow turntable 9 driven by a turntable motor 3 is installed on the support plate 4. The support plate 4 is provided with a through hole that penetrates the center hole of the hollow turntable 9. The mechanical claw includes a mechanical claw bottom plate 11. The lower end of the mechanical claw bottom plate 11 is connected to a plurality of mechanical claw fingers 6 driven by a mechanical claw motor 5 and driven by a connecting rod mechanism. The upper end of the mechanical claw bottom plate 11 is connected to the lower end of the hollow turntable 9 through a plurality of first studs 10, and the upper end of the hollow turntable 9 is connected to the rotor end 2 of the rotary connector through a plurality of second studs 8. The rotary connector includes multiple channels. The rotary connector in the present invention can have different channel numbers such as four channels, eight channels, sixteen channels, twenty-four channels and fifty-six channels, and can transmit more sensor information at the same time. The rotary connector can be a coaxial type or an off-axis type rotary connector. The rotary connector in this embodiment has four channels, which are respectively recorded as channel 1, channel 2, channel 3 and channel 4. In this embodiment, the mechanical claw has three mechanical claw fingers 6, and a fiber optic tactile sensor 7 is installed on the inner side of each mechanical claw finger 6 near the fingertip. The three fiber optic tactile sensors 7 are respectively connected to channel 1, channel 2 and channel 3 of the rotor end 2 of the rotary connector through an optical fiber. The output optical fibers corresponding to channel 1, channel 2 and channel 3 occupied by each fiber optic tactile sensor 7 at the stator end 1 of the rotary connector are connected to a grating demodulator or a spectrometer. A laser radar 12 is fixed to the lower end of the mechanical claw bottom plate 11 by a screw. The laser radar 12 is installed at a position where the laser direction coincides with the central axis of the mechanical claw. The laser radar 12 is connected to channel 4 of the rotor end 2 of the rotary connector through an optical fiber. The output optical fiber corresponding to channel 4 occupied by the laser radar 12 on the stator end 1 of the rotary connector is connected to the acquisition card through a photoelectric converter. The driving motor of the displacement lifting platform 14, the turntable motor 3, the mechanical claw motor 5, the acquisition card, the grating demodulator or the spectrometer are all connected to the host computer.

[0030] like Figure 2 As shown, the installation structure of the laser radar 12 is: the mechanical claw bottom plate 11 is connected to the upper side plate of a U-shaped adapter plate 15 with a horizontal opening through screws 16, and the laser radar 12 is installed on the lower side plate of the U-shaped adapter plate 15 at a position where the laser direction coincides with the central axis of the mechanical claw.

[0031] like Figure 3As shown, the fiber optic tactile sensor in the present invention can be a single (a) fiber grating sensor, (b) fiber Mach-Zehnder sensor, (c) fiber Fabry-Perot sensor and other different structures or a sensor array of multiple different fiber optic structures. (a) The fiber grating sensor can be a fiber Bragg grating or a long period grating, etc. This type of sensor is simple to make; (b) A, B, and C in the fiber Mach-Zehnder sensor can be the same fiber or different fibers, and fiber B can be coaxial or non-coaxial with fiber A and fiber C. The fiber type, length, and misalignment amount are selected according to experimental needs; (c) D in the fiber Fabry-Perot (fiber FP) sensor can be a single-mode or multi-mode fiber, E is a capillary fiber, and the inside can be filled with materials according to the measurement parameters, and F can be a membrane made of silica or various organic polymer materials. In this embodiment, it is a fiber Bragg grating.

[0032] The tactile information referred to in the present invention refers to object information including pressure, temperature, humidity, curvature, stiffness and surface roughness.

[0033] like Figure 4 As shown, taking three fiber Bragg gratings FBG1, FBG2, and FBG3 as pressure, curvature, and temperature tactile sensors to grasp a spherical object as an example,

[0034] The movement of the fiber-optic visual-tactile fusion rotary mechanical claw system of the present invention is controlled by a host computer. First, a lifting platform 14 is placed below the mechanical claw and directly below the laser emission direction of the laser radar 12, and the object 13 to be grasped is placed on the lifting platform 14. According to the shape of the object 13 to be grasped, the host computer first controls the turntable motor 3 to drive the rotor end 2 of the rotary connector fixed to the hollow turntable 9 to rotate to a suitable position. The host computer controls the motor of the lifting platform 14 to move the grasped object 13 up and down. In the process of the object 13 to be grasped gradually approaching the mechanical claw, the interference frequency measured by the laser radar 12 gradually decreases. When the cut-off frequency is reached, the lifting platform 14 stops moving. At the same time, the host computer controls the mechanical claw motor 5 to start working, and the mechanical claw motor 5 drives the connecting rod mechanism to close the mechanical claw fingers 6 to grasp the object. When the mechanical claw finger 6 contacts the object 13 to be grasped, the optical fiber tactile sensor 7 located on the inner side of each mechanical claw finger 6 controls the movement of the mechanical claw motor as tactile feedback. When the set wavelength drift threshold is reached, the mechanical claw motor 5 stops working to complete the stable grasping of the object.

[0035] FBG1, FBG2 and FBG3 are fixed on the inner side of the fingertips of the three fingers of the mechanical claw respectively. FBG1 is used as a pressure tactile sensor, FBG2 is used as a curvature sensor, and FBG3 is used as a temperature sensor. When grasping an object, firstly, the cutoff frequency of the laser radar 12 and the wavelength drift of FBG1 are set as thresholds in the host computer according to the type of grasped object. The object 13 to be grasped is placed on the lifting and displacement platform 14. When the object approaches, the response frequency of the laser radar 12 will change. When the set frequency threshold is reached, the mechanical claw motor 5 starts to work and grasps the object. When the wavelength drift of FBG1 at the fingertips reaches the threshold, the mechanical claw motor 5 stops working to achieve stable and accurate grasping of the object. After the object is grasped stably, the curvature of the contact surface of the object is obtained through FBG2, and FBG3 is used as a temperature sensor to display the temperature of the grasped object in real time. During the grasping process, when the FBG1 wavelength drift of the fingertip is less than the threshold, the mechanical claw motor 5 rotates forward, the mechanical claw performs the grasping action, and gradually increases the force applied to the object. When the FBG1 wavelength drift exceeds the threshold, the mechanical claw motor 5 is controlled to reverse and reduce the force applied to the object. During the rotation process, the turntable motor 3 can be controlled to rotate a set number of steps or a fixed time. During the whole process, the sensing optical signals of the laser radar 12 and the three fiber grating sensors are transmitted through the rotating connector.

[0036] Although the present invention has been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many modifications without departing from the purpose of the present invention, all of which are within the protection of the present invention.

Claims

1. A fiber-optic visual-tactile fusion rotating mechanical claw system, comprising a mechanical claw, a displacement lifting platform (14), a support plate (4) and a host computer, wherein a hollow turntable (9) driven by a turntable motor (3) is installed on the support plate (4), and the support plate (4) is provided with a through hole that penetrates the center hole of the hollow turntable (9), and the mechanical claw comprises a mechanical claw bottom plate (11), and the lower end of the mechanical claw bottom plate (11) is connected to a plurality of mechanical claw fingers (6) driven by a mechanical claw motor (5) and driven by a connecting rod mechanism, and the upper end of the mechanical claw bottom plate (11) is connected to the hollow turntable (9) through a plurality of first studs (10), characterized in that: The upper end of the hollow turntable (9) is connected to the rotor end (2) of the rotary connector via a plurality of second studs (8); The rotary connector comprises multiple channels, an optical fiber tactile sensor (7) is installed on the inner side of the mechanical claw finger (6), all the optical fiber tactile sensors (7) are connected to a channel of the rotor end (2) of the rotary connector via an optical fiber, and the output optical fibers of the stator end (1) of the rotary connector corresponding to the channel occupied by each optical fiber tactile sensor (7) are connected to a grating demodulator or a spectrometer; A laser radar (12) is fixed to the lower end of the mechanical claw bottom plate (11) by a screw rod. The laser radar (12) is installed at a position where the laser direction coincides with the central axis of the mechanical claw. The laser radar (12) is connected to a channel of the rotor end (2) of the rotary connector by an optical fiber. The output optical fiber on the stator end (1) of the rotary connector corresponding to the channel occupied by the laser radar (12) is connected to the acquisition card by a photoelectric converter. The driving motor of the displacement lifting platform (14), the turntable motor (3), the mechanical claw motor (5), the acquisition card, and the grating demodulator or the spectrometer are all connected to the host computer.

2. The optical fiber visual-tactile fusion rotating mechanical claw system according to claim 1, characterized in that: The installation structure of the laser radar (12) is as follows: the mechanical claw bottom plate (11) is connected to the upper side plate of a U-shaped adapter plate (15) with a horizontal opening through screws (16), and the laser radar (12) is installed on the lower side plate of the U-shaped adapter plate (15) at a position where the laser direction coincides with the central axis of the mechanical claw.

3. The optical fiber visual-tactile fusion rotating mechanical claw system according to claim 1, characterized in that: The optical fiber tactile sensor (7) is arranged at the fingertip of the mechanical claw finger (6).

4. The optical fiber visual-tactile fusion rotating mechanical claw system according to claim 1, characterized in that: The optical fiber tactile sensor (7) is an optical fiber grating sensor, and the optical fiber grating sensor is an optical fiber Bragg grating or a long period grating.

5. The optical fiber visual-tactile fusion rotating mechanical claw system according to claim 1, characterized in that: The optical fiber tactile sensor (7) is an optical fiber Mach-Zehnder sensor.

6. The optical fiber visual-tactile fusion rotating mechanical claw system according to claim 1, characterized in that: The optical fiber tactile sensor (7) is an optical fiber Fabry-Perot sensor.