Self-adaptive grabbing system and method based on photoelectric sensing and material recognition

Abstract

The invention provides a self-adaptive grabbing system and method based on photoelectric sensing and material recognition. The system comprises a manipulator, a mechanical arm, a front-end processor and an upper computer controller. The manipulator is driven by a torque-adjustable motor, and an optical sensor module based on a photoelectric sensing principle is mounted on a fingertip of the manipulator; the optical sensor module is used for acquiring a texture grayscale image and sliding micro-displacement information of the surface of a grabbed object; the front-end processor is used for collecting data of the optical sensor module and transmitting the data to the upper computer controller; the upper computer controller recognizes the material of the object and matches the friction coefficient, the optimal clamping force increment is calculated according to the sliding micro-displacement information and the friction coefficient, and the motor is controlled to achieve self-adaptive grabbing. According to the system, the manipulator can be controlled to increase the clamping force in an optimal clamping force increment mode according to the sliding amplitude and the friction characteristic of the object, and self-adaptive stable grabbing of the object is achieved.

Description

Technical Field

[0001] This invention relates to the field of robotic gripping technology, and more specifically, to an adaptive gripping system and method based on photoelectric sensing and material recognition. Background Technology

[0002] With the continuous advancement of robotics and the rapid development of science and technology, robotic arms have been widely used in industries such as industry and agriculture. For most robotic arms, achieving stable gripping is one of their core functional goals. Traditional gripping methods based on preset fixed clamping forces can achieve stable gripping of single objects, such as handling or grasping a single type of object. However, they lack the intelligence and flexibility to automatically identify and autonomously optimize the clamping force for different types of objects. This method is only suitable for repetitive, mechanical, and stable gripping of the same object in structured environments. When robots leave industrial settings and enter unstructured environments such as our daily lives, the physical information (weight, shape, coefficient of friction) of different objects varies significantly. If a preset fixed clamping force is still used, insufficient clamping force may cause the object to slip when gripping heavy objects, resulting in damage or even rendering it unusable. Excessive clamping force may crush fragile objects such as eggs. Achieving a balance between clamping force and object deformation remains a challenging problem in the field of robotics. Therefore, achieving adaptive and stable grasping of various objects in unstructured environments while minimizing the deformation of the grasped objects is of great practical significance for the advancement of grasping technology in propulsion robots.

[0003] One of the keys to adaptive and stable gripping lies in the accurate detection of slippage. If the robotic arm can detect slippage in a timely manner and increase the corresponding gripping force, the object can still be prevented from slipping. Currently, the mainstream methods for slippage detection utilize various types of sensors, such as piezoresistive and piezoelectric sensors, to detect slippage during the gripping process. However, piezoresistive sensor-based slippage detection suffers from low accuracy and high power consumption, and its structure is usually designed as an array, resulting in large dimensions and difficulty in system integration. Piezoelectric sensor-based slippage detection suffers from poor anti-interference capabilities and susceptibility to electromagnetic interference, placing high demands on the gripping environment. Furthermore, most existing strategies for achieving stable gripping based on slippage events are still based on qualitative slippage detection, meaning they can only determine whether slippage has occurred. When slippage occurs, a pre-set gripping force increment is added based on experience, rather than automatically optimizing the optimal gripping force increment required for adaptive and stable gripping based on the magnitude of the object's slippage and its frictional characteristics. Therefore, while traditional adaptive gripping strategies based on qualitative detection sliding can achieve stable one-click gripping of most objects with good strength, they cannot guarantee that the deformation of the gripped object will be minimized while maintaining stable gripping. When the object being gripped is fragile or sensitive to deformation, using traditional adaptive gripping strategies based on qualitative detection sliding may result in damage or permanent irreversible deformation. Meanwhile, with the rapid advancement of modernization, various industries are increasingly demanding advanced gripping performance from robotic arms, urgently requiring a technical method that can achieve adaptive and stable gripping of various objects while minimizing object deformation. Summary of the Invention

[0004] To overcome the shortcomings and deficiencies of the existing technology, the present invention aims to provide an adaptive gripping system and method based on photoelectric sensing and material recognition. The system can control the robotic arm to increase the gripping force in an optimal increment manner according to the size of the sliding amplitude and the friction characteristics of the object itself, so as to achieve adaptive and stable gripping of the object. It has the advantages of low latency, strong anti-interference ability, low cost and high flexibility.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an adaptive grasping system based on photoelectric sensing and material recognition, comprising: The robotic arm is driven by a torque-adjustable motor, and the tips of the robotic fingers are equipped with optical sensor modules based on the photoelectric sensing principle. The optical sensor modules are used to collect grayscale images of the texture of the surface of the grasped object and information on the micro-displacement caused by the relative sliding between the grasped object and the tips of the robotic fingers. A robotic arm is used to adjust the position and posture of a robotic hand. The front-end processor is used to collect data from the optical sensor module and transmit it to the host computer controller via USB or serial port. The host computer controller is used to process the texture grayscale image and sliding micro-displacement information, identify the object material and match the friction coefficient, calculate the optimal clamping force increment ΔF based on the sliding micro-displacement information and friction coefficient, and control the motor to achieve adaptive gripping.

[0006] Preferably, the mechanical fingertip is provided with a receiving cavity, and the cavity wall facing the inner side of the mechanical hand has a sensing hole, which is closed by an optical lens; the optical sensor module is disposed in the receiving cavity, and collects the texture grayscale image of the surface of the grasped object and the sliding micro-displacement information caused by the relative sliding between the grasped object and the mechanical fingertip through the optical lens of the sensing hole; the optical sensor module is connected to the front-end processor.

[0007] Preferably, the optical sensor module refers to an optical mouse chip; an optical lens is provided on the side of the optical sensor module closer to the object being grasped; the distance between the optical sensor module and the contact surface of the object being grasped is 2.2mm to 2.6mm.

[0008] Preferably, the optical sensor module is connected to the front-end processor via SPI communication, and the front-end processor communicates with the host computer controller via USB-to-serial communication.

[0009] Preferably, the host computer controller has a built-in lightweight convolutional neural network model for material classification of the texture grayscale image.

[0010] An adaptive grasping method based on photoelectric sensing and material recognition, used in the aforementioned adaptive grasping system based on photoelectric sensing and material recognition, includes the following steps: The robotic arm is controlled to move to the vicinity of the object to be grasped, and the robotic arm is adjusted to contact the surface of the object to be grasped. The optical sensor module collects grayscale images of the surface texture of the object to be grasped. The data is transmitted from the front-end processor to the host computer controller. The host computer controller inputs the received surface texture grayscale image into the trained lightweight convolutional neural network model to complete the material classification and obtain the material classification result. Based on the material classification result, the host computer controller retrieves the friction coefficient that matches the material of the grasped object from the established friction coefficient database. The robotic arm is controlled to lift the robotic hand upwards; the optical sensor module detects the micro-displacement information caused by the relative sliding between the grasped object and the robotic fingertips, and transmits it to the host computer controller for processing via the front-end processor; the host computer controller calculates the optimal clamping force increment ΔF based on the micro-displacement information and the friction coefficient that matches the material of the grasped object, and controls the motor to increase the clamping force by the optimal clamping force increment ΔF until the sliding stops, thereby achieving adaptive and stable grasping of the object while minimizing the deformation of the grasped object.

[0011] Preferably, the method for calculating the optimal clamping force increment ΔF is as follows: ; in, K These are the fitting coefficients. For safety factor; f To match the coefficient of friction to the material of the object being grasped, x g This refers to the sliding micro-displacement obtained within one sliding detection cycle.

[0012] Preferably, the sliding detection cycle is 5ms.

[0013] Preferably, the method further includes: during the grasping process, the optical sensor module continuously detects the sliding micro-displacement information, calculates the optimal clamping force increment ΔF, and dynamically adjusts the clamping force to cope with external interference during the grasping process.

[0014] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. This invention proposes an adaptive grasping system and method based on photoelectric sensing and material recognition, which overcomes the shortcomings of traditional grasping strategies with pre-set fixed clamping force in grasping operations in unstructured scenarios, which lack flexibility and intelligence. It enables the robotic arm to perform adaptive grasping and material recognition on various objects with different physical information in unstructured environments, thus promoting the advancement of robot grasping technology towards human grasping level. 2. This invention proposes a technical method for determining the optimal clamping force increment based on the friction coefficient and sliding amplitude of the grasped object. This enables the robotic arm to adaptively grasp the object with minimal deformation during grasping operations, greatly improving the reliability of the robotic arm in grasping fragile or deformation-sensitive objects. 3. This invention uses a host computer controller as the core hub, and each part establishes communication with the host computer controller to achieve rapid information transmission and sharing. The time consumed by the entire process from the optical sensor module detecting the sliding to the drive motor applying the optimal clamping force increment to the grasped object is in the millisecond range. At the same time, an optical mouse chip is used as the component for developing the optical sensor module, which has extremely high optical resolution (capable of detecting nanometer and micrometer level displacement). Compared with existing technologies, this invention has the advantages of low cost, high stability, and low latency. Attached Figure Description

[0015] Figure 1 This is an overall structural diagram of the adaptive grasping system based on photoelectric sensing and material recognition of the present invention; Figure 2 This is a schematic diagram of the robotic arm in the adaptive grasping system based on photoelectric sensing and material recognition of the present invention. Figure 3 This is a schematic diagram of the installation of the robotic hand and robotic arm of the present invention; Figure 4 This is a structural diagram of the lightweight convolutional neural network model of the present invention; Figure 5 This is a flowchart of the adaptive grasping method based on photoelectric sensing and material recognition of the present invention. Detailed Implementation

[0016] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0017] Example This embodiment describes an adaptive grasping system based on photoelectric sensing and material recognition, such as... Figure 1 As shown, it includes a robotic arm, a front-end processor, and a host computer controller.

[0018] robotic arms, such as Figure 2 As shown, driven by a torque-adjustable motor 2, the mechanical fingertip is equipped with an optical sensor module 1 based on the photoelectric sensing principle; the optical sensor module 1 is used to collect the texture grayscale image of the surface of the grasped object and the sliding micro-displacement information caused by the relative sliding between the grasped object and the mechanical fingertip.

[0019] The mechanical fingertip is provided with a receiving cavity, and the cavity wall facing the inner side of the mechanical hand has a sensing hole, which is sealed by an optical lens; the optical sensor module is set in the receiving cavity, and collects the texture grayscale image of the surface of the grasped object and the sliding micro-displacement information caused by the relative sliding between the grasped object and the mechanical fingertip through the optical lens of the sensing hole.

[0020] The robotic arm is used to adjust the position and posture of the robotic hand; the installation of the robotic arm and robotic hand is as follows: Figure 3 As shown.

[0021] The front-end processor is used to acquire data from the optical sensor module and transmit it to the host computer controller via USB or serial port. The front-end processor can be an existing processor, such as an STM32 development board.

[0022] The host computer controller is used to process the texture grayscale image and sliding micro-displacement information, identify the object material and match the friction coefficient, calculate the optimal clamping force increment ΔF based on the sliding micro-displacement information and friction coefficient, and control the motor to achieve adaptive gripping.

[0023] In this embodiment, the robotic arm is a parallel two-finger gripper, which integrates a torque-adjustable motor with a working voltage of 24V to control the magnitude of the force. Simultaneously, the motor driver is controlled via CAN bus communication to achieve the clamping and releasing of the robotic arm.

[0024] The optical sensor module preferably uses an optical mouse chip, although different models can be substituted. The operating voltage of the optical mouse chip is 1.8V to 2.1V. The externally input 3.3V voltage is converted to 1.9V by a voltage conversion component on the PCB and then input to the optical mouse chip to enable its normal operation. The optical mouse chip can achieve high-sensitivity detection of micron-level sliding. The optical sensor module is fixed in the housing cavity with M1.2×5 hexagonal screws, and the distance between the optical sensor module and the contact surface of the grasped object is 2.2mm to 2.6mm. In this embodiment, the optical sensor module based on photoelectric sensing principle refers to an optical sensor module composed of an optical mouse chip with extremely high resolution and anti-electromagnetic interference capability. This module can rapidly acquire grayscale images of the texture of sliding and contacting object surfaces at the micrometer level, and transmit the acquired data to the host computer controller via full-duplex communication, greatly improving the system's data transmission efficiency. By installing the optical sensor module based on photoelectric sensing principle on the robotic arm, and utilizing the grayscale texture image and sliding displacement information acquired by the optical sensor module when the central opening of the robotic finger contacts an object, the robotic arm can achieve integrated adaptive grasping and intelligent recognition. The optical sensor module is connected to the front-end processor via SPI communication, and the front-end processor communicates with the host computer controller via USB-to-serial communication.

[0025] The host computer controller incorporates a lightweight convolutional neural network model for material classification of the texture grayscale image. The texture grayscale image acquired by the optical sensor module refers to a 36×36 grayscale image. Image preprocessing refers to data format conversion and noise reduction of the image to improve the accuracy of model recognition.

[0026] like Figure 4As shown, the lightweight convolutional neural network model consists of an input layer, an output layer, two convolutional layers, two pooling layers, and two fully connected layers. The trained model maps and outputs the corresponding category based on the unique features of the grayscale texture of the object's surface through the Softmax activation function.

[0027] The system data acquisition process is as follows: the micron-level sliding micro-displacement information and image data acquired by the optical sensor module are transmitted to the front-end processor via SPI communication, and then the front-end processor transmits the data to the host controller via USB-to-serial conversion. The entire data acquisition process takes only milliseconds, exhibiting extremely low latency.

[0028] The adaptive grasping system based on photoelectric sensing and material recognition is executed as follows: Figure 5 As shown, it includes the following steps: The robotic arm is controlled to move to the vicinity of the object to be grasped, and the robotic arm is adjusted to contact the surface of the object to be grasped. The optical sensor module collects grayscale images of the surface texture of the object to be grasped. The image is transmitted from the front-end processor to the host computer controller; the host computer controller inputs the received surface texture grayscale image into the trained lightweight convolutional neural network model; the lightweight convolutional neural network model performs material classification based on the different surface textures of different materials to obtain the material classification result; the host computer controller retrieves the friction coefficient that matches the material of the grasped object from the established friction coefficient database based on the material classification result. The robotic arm is controlled to lift the robotic hand upwards; the optical sensor module detects the micro-displacement information caused by the relative sliding between the grasped object and the robotic fingertips, and transmits it to the host computer controller for processing via the front-end processor; the host computer controller calculates the optimal clamping force increment ΔF based on the micro-displacement information and the friction coefficient matching the material of the grasped object. ; in, K These are the fitting coefficients. The safety factor is derived from experimental fitting. f To match the coefficient of friction to the material of the object being grasped, x g The sliding micro-displacement is obtained within one sliding detection cycle; one sliding detection cycle is preferably 5ms. The control motor increases the clamping force by the optimal clamping force increment ΔF until the sliding stops, thereby achieving adaptive and stable object grasping while minimizing the deformation of the grasped object.

[0029] This method can design different gripping strategies based on different sliding amplitudes and the friction coefficients matched by the material of the object. Since different gripping object materials have different friction coefficients, the increase in gripping force cannot be simply based on the size of the sliding amplitude. For example, under the same sliding amplitude, the optimal gripping force increment applied to an object with a smaller friction coefficient should be greater than the optimal gripping force increment applied to an object with a larger friction coefficient. By applying the optimal clamping force increment based on the magnitude of the sliding amplitude and the friction coefficient determined by the object's own properties, this method can achieve adaptive grasping of various objects and also cope with interference during grasping and transportation. For example, after successfully and stably grasping an object, if it is subjected to external interference during transportation, such as a slight impact or displacement due to a shift in the center of gravity, the system can detect the sliding information in real time and calculate the corresponding sliding amplitude. This is then fused with the previously matched and stored friction coefficients. Finally, based on the relationship between the sliding amplitude and the friction coefficient information matched according to the object's material and the optimal clamping force increment, the system calculates the optimal clamping force increment required for the current sliding process. This allows for a stable grasping of the object again before a complete sliding event occurs, minimizing the deformation of the object.

[0030] This invention can be deployed in two-finger / three-finger grippers and electric / pneumatic drive architectures.

[0031] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An adaptive grasping system based on photoelectric sensing and material recognition, characterized in that: include: The robotic arm is driven by a torque-adjustable motor, and its fingertips are equipped with optical sensor modules based on photoelectric sensing principles. The optical sensor module is used to acquire the texture grayscale image of the surface of the grasped object and the micro-displacement information caused by the relative sliding of the grasped object and the mechanical fingertip; A robotic arm is used to adjust the position and posture of a robotic hand. The front-end processor is used to acquire the texture grayscale image and sliding micro-displacement information of the optical sensor module and transmit them to the host computer controller. The host computer controller is used to process the texture grayscale image and sliding micro-displacement information, identify the object material and match the friction coefficient, calculate the optimal clamping force increment ΔF based on the sliding micro-displacement information and friction coefficient, and control the motor to achieve adaptive gripping.

2. The adaptive grasping system based on photoelectric sensing and material recognition according to claim 1, characterized in that: The mechanical fingertip is provided with a receiving cavity, and a sensing hole is opened in the cavity wall facing the inner side of the mechanical hand. The sensing hole is closed by an optical lens. An optical sensor module is set in the receiving cavity and collects the texture grayscale image of the surface of the grasped object and the sliding micro-displacement information caused by the relative sliding between the grasped object and the mechanical fingertip through the optical lens of the sensing hole. The optical sensor module is connected to the front-end processor.

3. The adaptive grasping system based on photoelectric sensing and material recognition according to claim 2, characterized in that: The optical sensor module refers to the optical mouse chip; the distance between the optical sensor module and the contact surface of the object being grasped is 2.2mm to 2.6mm.

4. The adaptive grasping system based on photoelectric sensing and material recognition according to claim 2, characterized in that: The optical sensor module is connected to the front-end processor via SPI communication, and the front-end processor communicates with the host computer controller via USB-to-serial and USB isochronous transmission.

5. The adaptive grasping system based on photoelectric sensing and material recognition according to claim 1, characterized in that: The host computer controller has a built-in lightweight convolutional neural network model for material classification of the texture grayscale image.

6. An adaptive grasping method based on photoelectric sensing and material recognition, characterized in that: The adaptive grasping system based on photoelectric sensing and material recognition as described in claim 1 includes the following steps: The robotic arm is controlled to move to the vicinity of the object to be grasped, and the robotic arm is adjusted to contact the surface of the object to be grasped. The optical sensor module collects grayscale images of the surface texture of the object to be grasped. The data is transmitted from the front-end processor to the host computer controller. The host computer controller inputs the received surface texture grayscale image into the trained lightweight convolutional neural network model to complete the material classification and obtain the material classification result. Based on the material classification result, the host computer controller retrieves the friction coefficient that matches the material of the grasped object from the established friction coefficient database. The robotic arm is controlled to lift the robotic hand upwards; the optical sensor module detects the micro-displacement information caused by the relative sliding between the grasped object and the robotic fingertips, and transmits it to the host computer controller for processing via the front-end processor; the host computer controller calculates the optimal clamping force increment ΔF based on the micro-displacement information and the friction coefficient that matches the material of the grasped object, and controls the motor to increase the clamping force by the optimal clamping force increment ΔF until the sliding stops, thereby achieving adaptive and stable grasping of the object while minimizing the deformation of the grasped object.

7. The adaptive grasping method based on photoelectric sensing and material recognition according to claim 6, characterized in that: The method for calculating the optimal clamping force increment ΔF is as follows: ； in, K These are the fitting coefficients. For safety factor; f To match the coefficient of friction to the material of the object being grasped, x g This refers to the sliding micro-displacement obtained within one sliding detection cycle.

8. The adaptive grasping method based on photoelectric sensing and material recognition according to claim 7, characterized in that: The sliding detection cycle is 5ms.

9. The adaptive grasping method based on photoelectric sensing and material recognition according to claim 6, characterized in that: Also includes: During the grasping process, the optical sensor module continuously detects the sliding micro-displacement information, calculates the optimal clamping force increment ΔF, and dynamically adjusts the clamping force to cope with external interference during the grasping process.

Images