Interactive robotic arm control method based on tactile feedback analysis

Abstract

The present invention discloses an interactive manipulator control method based on tactile feedback analysis, which relates to the field of manipulator control technology. The method comprises: configuring a high-precision tactile sensor on the end effector of a target manipulator to sense and acquire tactile data; obtaining a tactile feedback signal, and judging the properties of the contact object based on the tactile feedback signal; analyzing and identifying the tactile pattern, predicting the first operation of the manipulator, and generating a manipulator control strategy; based on the manipulator visual information, fusing the tactile feedback signal, and generating the operation path of the target manipulator; and controlling the target manipulator through a manipulator control update strategy. The method solves the technical problems that the existing manipulator control relies on vision and position sensors, cannot react to subtle external contacts in real time and accurately, and results in insufficient control precision, lack of flexibility and safety, and achieves accurate perception of the target object, thereby achieving the technical effect of improving the accuracy, flexibility and safety of the manipulator operation control.

Description

Technical Field

[0001] The present application relates to the technical field related to robot arm control, and in particular to an interactive robot arm control method based on tactile feedback analysis. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing technology, robotic arms have become an indispensable and important equipment in the modern production process and are widely used in various industrial scenarios such as assembly, processing, and handling. Traditional robotic arm operation methods mostly rely on preset motion trajectories and visual feedback for control. However, control methods that rely solely on visual feedback have certain limitations, especially in high-precision operation tasks. It is difficult to accurately perceive subtle changes in contact force and object properties, resulting in uncertainty and lack of flexibility in operation. For example, in complex grasping, assembly or fine operations, robotic arms need to have "tactile" capabilities similar to human fingers to enhance their ability to perceive and adapt to the environment. For example, when handling fragile objects, soft materials or irregular objects, visual sensors alone cannot accurately judge the physical properties of the object, which leads to grasping failure or object damage. However, most current robotic arms have not yet integrated or fully utilized tactile feedback, and cannot achieve sensitive perception and precise control of complex external environments.

[0003] At present, there are technical problems in the interactive robotic arm control related technologies that rely on vision and position sensors and cannot respond to subtle external contacts in real time and accurately, resulting in insufficient control precision and lack of flexibility and safety. Summary of the invention

[0004] The present application provides an interactive robotic arm control method based on tactile feedback analysis, thereby solving the technical problems of existing robotic arm control, which relies on vision and position sensors and cannot respond to slight external contacts in real time and accurately, resulting in insufficient control, lack of flexibility and safety. The application achieves accurate perception of target objects and achieves the technical effect of improving the accuracy, flexibility and safety of robotic arm operation control.

[0005] The present application provides an interactive robotic arm control method based on tactile feedback analysis, comprising: configuring a high-precision tactile sensor through the end effector of a target robotic arm to sense and acquire tactile data, wherein the tactile sensor integrates a multi-layer pressure sensing element; performing feature analysis on the tactile data to obtain a tactile feedback signal, and based on the tactile feedback signal, traversing a robotic arm tactile model library for comparison to determine the properties of a contact object, wherein the properties of the contact object include the hardness and surface texture of the contact object; analyzing and identifying the tactile pattern according to the tactile feedback signal combined with the properties of the contact object, predicting the first operation of the robotic arm, and generating a robotic arm control strategy; based on the visual information of the robotic arm, integrating the tactile feedback signal, generating an operation path of the target robotic arm, correcting a preset operation trajectory, adding it to the robotic arm control strategy, and obtaining a robotic arm control update strategy; controlling the target robotic arm through the robotic arm control update strategy, and entering a robotic arm safety mode if the robotic arm operation status monitoring data is abnormal.

[0006] In a possible implementation, the tactile data is feature analyzed to obtain a tactile feedback signal, and the following processing is also performed: the tactile data is preprocessed; feature analysis is performed based on the preprocessing result, including analyzing the contact force frequency domain characteristics, contact force time domain characteristics and contact force instantaneous characteristics to obtain the contact force, duration and surface vibration of the object; and a tactile feedback signal is generated based on the feature analysis result.

[0007] In a possible implementation, the robotic arm tactile model library is constructed, and the following processing is also performed: multiple categories of target objects are acquired, and the end effector of the target robotic arm is contacted with each category of target objects, including pressing, sliding, tapping and grasping; multiple initial tactile data are collected through the tactile sensor, and the multiple initial tactile data are standardized to obtain multiple standard tactile data; multiple key tactile features are extracted according to the analysis of the multiple standard tactile data, and the target object is labeled based on the multiple key tactile features to obtain a labeled object; a mapping relationship between the tactile feedback signal and the labeled object is established, and multiple tactile feature models are trained, each tactile feature model stores the key tactile features of the same type of objects; the multiple tactile feature models constitute the robotic arm tactile model library.

[0008] In a possible implementation, the analysis and identification of tactile patterns also performs the following processing: obtaining historical cases of the target robotic arm, extracting historical operation records and corresponding tactile pattern classification results; based on a convolutional neural network, training a tactile perception model using the historical operation records and the corresponding tactile pattern classification results; analyzing the tactile feedback signal to obtain tactile feedback analysis results, including contact force size, pressure distribution and friction coefficient; and inputting the tactile feedback analysis results into the tactile perception model to identify the tactile pattern.

[0009] In a possible implementation, the first operation of the robotic arm is predicted, and a control strategy of the robotic arm is generated, and the following processing is performed: based on the tactile pattern and in combination with the properties of the contact object, the first operation type of the robotic arm is predicted; the tactile data is time-series analyzed to determine the movement trend of the contact object; based on the current posture of the target robotic arm and the position of the end effector, the first operation of the robotic arm is obtained, and the control strategy of the robotic arm is generated.

[0010] In a possible implementation, the generation of the operation path of the target robotic arm also performs the following processing: using a binocular camera to collect image data of the target object; performing feature point recognition on the image data, smoothing the movement trajectory of the target object, and extracting the spatial position information of the target object; synchronously fusing the spatial position information and the tactile feedback signal to generate the operation path of the target robotic arm.

[0011] In a possible implementation, the preset operation trajectory is corrected and added to the robot arm control strategy to obtain the robot arm control update strategy, and the following processing is also performed: the preset operation trajectory includes the starting position, target position and multiple path key points of the target robot arm; the operation path of the target robot arm is compared with the preset operation trajectory to detect whether the trajectory deviates; if the detected trajectory deviates, the multiple path key points are corrected and adjusted according to the operation path of the target robot arm to obtain the corrected operation path of the target robot arm; the corrected operation path of the target robot arm is added to the robot arm control strategy to obtain the robot arm control update strategy; wherein, the robot arm control update strategy also includes a first force range and a first motion speed.

[0012] In a possible implementation, if the robot arm operation status monitoring data is abnormal, the robot arm safety mode is entered, and the following processing is also performed: the operation status of the target robot arm is monitored by multiple sensors to obtain multiple key operation data; if the multiple key operation data exceed the preset threshold, it is regarded as the target robot arm operation abnormality, and the robot arm safety mode is triggered; according to the current position and posture of the target robot arm, a retreat path is planned, and the target robot arm retreats to a safe position along the retreat path.

[0013] It is intended to adopt the interactive robotic arm control method based on tactile feedback analysis proposed in the present application, configure a high-precision tactile sensor on the end effector of the target robotic arm to sense and acquire tactile data, wherein the tactile sensor integrates multiple layers of pressure sensing elements; perform feature analysis on the tactile data to obtain a tactile feedback signal, and based on the tactile feedback signal, traverse the robotic arm tactile model library for comparison to determine the properties of the contact object, wherein the properties of the contact object include the hardness and surface texture of the contact object; analyze and identify the tactile pattern according to the tactile feedback signal combined with the properties of the contact object, predict the first operation of the robotic arm, and generate a robotic arm control strategy; based on the robotic arm visual information, fuse the tactile feedback signal, generate an operation path of the target robotic arm, correct the preset operation trajectory, add it to the robotic arm control strategy, and obtain a robotic arm control update strategy; control the target robotic arm through the robotic arm control update strategy, and if the robotic arm operation status monitoring data is abnormal, enter the robotic arm safety mode. It solves the technical problems of existing robotic arm control, which relies on vision and position sensors and cannot respond to slight external contacts in real time and accurately, resulting in insufficient control, lack of flexibility and safety. It achieves accurate perception of target objects and achieves the technical effect of improving the accuracy, flexibility and safety of robotic arm operation control. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the accompanying drawings of the embodiment of the present invention will be briefly introduced below. A flow chart is used in the present application to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps can be processed in reverse order or simultaneously as needed. At the same time, other operations can also be added to these processes, or one or more operations can be removed from these processes.

[0015] Figure 1 A schematic flow chart of an interactive robotic arm control method based on tactile feedback analysis provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of the process of analyzing and identifying tactile patterns in an interactive robotic arm control method based on tactile feedback analysis provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below.

[0018] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0019] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict, and the terms "first\second" involved are merely to distinguish similar objects and do not represent a specific ordering of objects. The terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products, or devices. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field of this application. The terms used herein are for the purpose of describing the embodiments of the present application only.

[0020] The present application embodiment provides an interactive robot arm control method based on tactile feedback analysis, such as Figure 1 As shown, the method includes:

[0021] Step S100, configuring a high-precision tactile sensor through the end effector of the target robot arm to sense and acquire tactile data, wherein the tactile sensor integrates multiple layers of pressure sensing elements.

[0022] Preferably, the end effector is the part of the robot arm that interacts directly with external objects, usually a gripper, pliers or other tools. Its position determines the contact point between the robot arm and the object. The high-precision tactile sensor is the core sensing device installed on the end effector of the robot arm, which can detect the contact information between the robot arm and the external object in real time and detect tiny force or pressure changes. The tactile sensor can sense a variety of contact characteristics, such as the magnitude, direction, and pressure distribution of the force. The tactile sensor contains multiple pressure sensing layers, which can be distributed at different positions or in different directions to sense the pressure applied to the end of the robot arm. Each layer of sensing elements usually uses different materials or designs to achieve accurate perception of different dimensions (such as vertical and lateral forces), for example The surface sensing elements sense changes in surface pressure and friction, and are used to detect the initial force or sliding information of contact with the object. The middle-layer sensing elements sense deeper pressure changes or complex contact situations (such as object deformation), and provide richer feedback information for the applied force. The bottom-layer sensing elements are used to detect a wider range of contact forces or pressures, and provide a global pressure distribution between the robot arm and the object. The tactile sensor can distinguish and perceive different types of forces (such as pressure, shear force, and friction) through a multi-layer structural design, and provide more sophisticated and diverse tactile feedback than a single sensor. The tactile sensor can obtain tactile data between the robot arm and external objects in real time, and transmit these tactile data to the control system to help the robot arm make more flexible and precise operational decisions.

[0023] Step S200, performing feature analysis on the tactile data to obtain a tactile feedback signal, and based on the tactile feedback signal, traversing the robotic arm tactile model library for comparison to determine the properties of the contact object, wherein the properties of the contact object include the hardness and surface texture of the contact object.

[0024] Preferably, by processing the collected tactile data, key information that can reflect the physical properties of the object is extracted, which usually includes analyzing the rate of change of force during contact, understanding the reaction of the object under different contact conditions, analyzing the pressure difference at different parts, extracting features related to the surface shape and texture of the object, and analyzing the relationship between the contact force and the time of force application, which is used to determine whether the object is a flexible material or a rigid material. The tactile feedback signal is a signal generated after feature analysis that can reflect the current contact state between the robot arm and the object. This signal includes feature representations of tactile information such as force, pressure, and friction. The system uses these signals as feedback basis for the perception of external objects. These feedback signals contain the real-time physical reaction of the object and can be used to determine the properties of the object during contact (such as hardness and surface texture); the tactile model library is a database that stores tactile characteristics of various objects. These models are established through historical operations, training data, etc., and cover tactile characteristics of different types of objects, including the hardness, elasticity, friction coefficient, and surface roughness of the object. After the robot arm obtains the tactile feedback signal, it traverses the tactile model library to find the object type that best matches the current tactile signal. Through this comparison, the properties of the current contact object are judged. For example, by analyzing the response speed of the force and the relationship between the applied force and the deformation, it can be judged whether the object is a hard material (such as metal, plastic) or a soft material (such as rubber, cloth). Hardness is the ability of an object to resist deformation. Objects with higher hardness have smaller changes in force when in contact, while objects with lower hardness may produce obvious deformation as the contact force increases. If the deformation of the object is small when a large force is applied, it can be judged that the object has a higher hardness; otherwise, it means that the object is softer; by analyzing the pressure distribution and friction characteristics, it can be judged whether the surface of the object is smooth or rough, and whether there are fine textures or particles. Surface texture refers to the characteristics of the roughness or fine shape of the surface of the object, which may include unevenness, granularity, sliding resistance, etc. When the friction force fluctuates during sliding, it may indicate that there is a granular texture on the surface; if the friction force is stable, it may indicate that the surface is smooth. Through the feature analysis and model comparison process of tactile feedback signals, the robotic arm can accurately judge the hardness and surface texture of the contacted object, so that it can flexibly adjust the operation mode according to the different object properties, thereby improving the operation accuracy and adaptability.

[0025] In a possible implementation, step S200 further includes step S210, preprocessing the tactile data; step S220, performing feature analysis based on the preprocessing result, including analyzing the frequency domain characteristics, time domain characteristics and instantaneous characteristics of the contact force, and obtaining the contact force, duration and surface vibration of the object; and step S230, generating a tactile feedback signal based on the feature analysis result.

[0026] Preferably, the tactile data is preprocessed, such as low-pass filtering, high-pass filtering or Kalman filtering, to remove redundant high-frequency or low-frequency noise signals to ensure the accuracy of the data, and then the tactile data is normalized to adjust the data values ​​to the same magnitude range to improve the comparability and analysis accuracy between different features; feature analysis is performed on the preprocessed tactile data to extract key information that can reflect the contact process, which is usually divided into three dimensions: frequency domain features, time domain features and instantaneous features. The tactile signal is converted from the time domain to the frequency domain by performing Fourier transform or wavelet transform on the contact force to capture information on periodic force changes during the contact process, which is particularly suitable for analyzing small vibrations or frequency fluctuations on the surface of an object. The amplitude and phase of different frequency components are extracted from the frequency domain, which can identify different vibration frequencies during the contact process and help determine the surface characteristics of the object. If the frequency spectrum contains higher frequency components, it may indicate that the surface of the object is relatively rough and produces small vibrations during contact; Domain analysis is mainly used to obtain key information such as the magnitude of force, speed of change and contact time, calculate the average force, peak force and force change rate applied during the contact process, and determine whether the force applied by the robot is stable, too large or too small; record the duration of each contact during the contact process to determine the length of time of the operation; by analyzing the transient changes of tactile signals, capture sudden events in the contact process, such as a sharp increase or decrease in instantaneous contact force. The transient characteristics can identify rapid changes that occur during the contact process, such as a slight collision or sliding when the robot contacts the object; the tactile feedback signal is based on the result of feature analysis extraction, reflecting the real-time information of the contact process between the robot and the object. By analyzing the time domain, frequency domain and transient characteristics of the contact force, a feedback signal describing the contact state of the robot and the object is generated. For example, when the robot contacts an object with a rough surface, the feedback signal may contain higher frequency domain vibration components and force change rate information in the time domain, prompting the robot to adjust the force.

[0027] In a possible implementation, step S200 further includes step S241, acquiring target objects of multiple categories, and performing contact operations between the end effector of the target robotic arm and the target objects, including pressing, sliding, tapping and grasping; step S242, collecting multiple initial tactile data through the tactile sensor, and standardizing the multiple initial tactile data to obtain multiple standard tactile data; step S243, extracting multiple key tactile features based on the analysis of the multiple standard tactile data, and labeling the target object based on the multiple key tactile features to obtain a labeled object; step S244, establishing a mapping relationship between the tactile feedback signal and the labeled object, and training multiple tactile feature models, each tactile feature model storing the key tactile features of the same type of objects; step S245, the multiple tactile feature models constitute the robotic arm tactile model library.

[0028] Preferably, objects of different types, shapes, and materials are selected as target objects to ensure that they cover a wide range of physical properties (such as hardness, elasticity, surface roughness, etc.), and a variety of tactile operations are performed on different objects through the end effector of the robotic arm, including pressing, sliding, light touching, and grasping; during the contact operation, the physical feedback data between the object and the robotic arm, including contact force, pressure, friction, etc., are collected in real time through the tactile sensor, and the initial tactile data is standardized and converted into a unified scale and dimension to facilitate subsequent analysis and comparison, including converting tactile data of different ranges into a unified numerical range and removing the tactile signal. Noise and abnormal data are collected to obtain multiple standard tactile data. Based on the standardized tactile data, the key tactile features of the object in different operation modes are extracted. These features can reflect the physical properties of the object, such as hardness, surface friction coefficient, elasticity, and deformation ability. According to the extracted key tactile features, the target object is classified and labeled. For example, according to properties such as hardness and surface smoothness, the object is labeled as soft, hard, rough surface, etc. The target object is given a specific label to form a labeled object with a feature label. Then, a mapping relationship between the tactile feedback signal and the labeled object is established, which describes the tactile feedback signal (such as For example, if the tactile signal shows a higher hardness feedback, it is mapped to the category labeled "hard object" to be used by the robot arm to match the tactile feedback signal and the labeled object category in real time during the operation, so as to facilitate the robot arm to identify and process objects according to the feedback signal in different tasks; based on multiple labeled objects and their corresponding tactile feedback signals, machine learning algorithms (such as support vector machines (SVMs), neural networks, etc.) are used to train models on these data to generate models that can identify specific object categories. Each tactile feature model stores key tactile features related to a certain type of object, such as The model trained for "soft objects" can recognize the feedback features of different soft objects, and the model trained for "rough surface objects" can recognize objects with different surface roughness; the multiple trained tactile feature models are combined into a tactile model library, each model contains the key tactile information of the object. When the robotic arm performs a task, it can use the real-time collected tactile feedback signal to find and match the feature model in the tactile model library, identify the category and attributes of the currently operated object, and adjust the operation strategy accordingly. For example, when the robotic arm senses that the contact force matches a certain model, the system can infer that it is a "soft object" and adjust the force application method accordingly.

[0029] Step S300 , analyzing and identifying the tactile pattern according to the tactile feedback signal and the contact object attribute, predicting the first operation of the robot arm, and generating a robot arm control strategy.

[0030] Preferably, by combining the tactile feedback signal with the known object properties, the physical properties of the object can be understood more accurately. For example, if an object is judged to be "hard" and "rough", then measures such as a smaller contact area and increased buffering need to be taken when applying force. By analyzing the force changes, pressure distribution, friction characteristics, etc. in the feedback signal, the tactile pattern corresponding to the current operation can be identified. For example, if the robotic arm detects that the pressure is gradually increasing and the friction force remains stable, it may be a grasping action. If it is detected that the contact area is expanding and the force is increasing in a certain direction, it may be a pressing or pushing action. The tactile pattern refers to the operation mode of the robot arm in different contact situations, which may include pressing, grasping, sliding, rotating, light touching and other operation modes. Each mode reflects how the robot arm physically interacts with the object. The purpose of identifying the tactile pattern is to determine the execution stage of the current task. For example, when the robot arm contacts the object, it may first touch it lightly, and then gradually increase the force to complete the grasping or fixing. This pattern recognition can intelligently adjust the contact mode to ensure safe and effective operation; according to the hardness, surface texture and other properties of the object, predict the action that the robot arm needs to take. For example, when it is recognized that the object is a flexible object, it is predicted that the next operation needs to reduce the force to avoid damage to the object. Referring to the operation history of similar objects and combining the current tactile feedback signal, the optimal first operation is predicted. This means that after the robot arm obtains tactile feedback and recognizes the current tactile pattern, it predicts and executes the optimal Physical actions, for example, when the robot arm contacts a soft object, it may predict that the next step requires gentle grasping; when it contacts a hard object, it predicts that the first operation may be to further apply force or more precise grasping. If a new tactile feedback signal is received during the first operation, the next operation can be adjusted in real time. For example, if the object is found to slide during the grasping process, the robot arm can reduce the force or readjust the contact position according to the feedback signal; based on the identified tactile pattern and object properties, a control strategy adapted to the current object is generated. For example, when grasping a hard object, the strategy may be to slowly apply a large force while avoiding sliding; when handling a soft object, the strategy may be to grasp gently and prevent excessive pressure. The robot arm control strategy refers to a comprehensive plan of the motion path, force range, movement speed, etc. performed by the robot arm in a specific operation task. The purpose of generating a control strategy is to ensure that the robot arm can complete the task in the best way and has flexible adjustment capabilities during the operation.

[0031] In one possible implementation, Figure 2As shown, step S300 further includes step S310, obtaining historical cases of the target robotic arm, extracting historical operation records and corresponding tactile pattern classification results; step S320, based on a convolutional neural network, using the historical operation records and the corresponding tactile pattern classification results to train a tactile perception model; step S330, analyzing the tactile feedback signal to obtain a tactile feedback analysis result, including contact force size, pressure distribution and friction coefficient; step S340, inputting the tactile feedback analysis result into the tactile perception model to identify the tactile pattern.

[0032] Preferably, historical operation records are collected from past operation tasks performed by the robotic arm, including the movements, forces, properties of contact objects, and tactile feedback data of the robotic arm in different operation scenarios, covering a variety of operation modes (such as grasping, pressing, sliding, etc.) and characteristics of different objects. The historical operation records include parameters such as joint angles, force conditions, operation trajectories, and speeds of the robotic arm when performing tasks, as well as tactile data obtained during the operation (such as force magnitude, pressure distribution, friction, etc.). The tactile pattern classification result refers to the tactile pattern corresponding to each operation (such as pressing, grasping, Sliding, etc.); build a recognition model based on a convolutional neural network (CNN), input the collected historical operation data into CNN as training data, including the operating parameters of the robot in different operation tasks (such as contact force, pressure distribution, friction, etc.), each historical operation record has a corresponding tactile pattern classification result, these classification results are the target output of the model, and are used to train CNN for tactile pattern recognition. Specifically, through the convolution operation, local features in the tactile data are extracted to capture the spatial relationship of different tactile data, such as force changes, distribution of friction coefficients, etc. The pooling layer is used to reduce the dimension of the data and reduce the computational complexity. After the extraction is completed, the extracted features are mapped to the tactile pattern classification results through the fully connected layer, and the final model output is the predicted value of the tactile pattern classification, such as pressing, sliding, grabbing, etc. After multiple rounds of training and optimization, the CNN model can learn the key features of each tactile pattern based on the input tactile data, so as to accurately identify and classify different tactile patterns; analyze the tactile feedback signal to obtain the tactile feedback analysis results, including the contact force, pressure distribution and friction coefficient. For example, by analyzing the force applied to the object, the strength of the force applied by the robot arm during the operation is judged, and the contact between the robot arm and the object is obtained through the tactile sensor. The robot uses the tactile feedback obtained by the analysis (contact force, pressure distribution, friction coefficient, etc.) as input and passes it into the trained tactile perception model. The model processes the input data, extracts the features, and recognizes and classifies the tactile patterns based on the historical operation patterns learned in the model. The tactile perception model matches the corresponding tactile patterns based on the input feedback analysis results, helping the robot arm to dynamically adjust the operation mode based on real-time tactile feedback, thereby achieving more intelligent, flexible and precise operation.

[0033] In a possible implementation, step S300 further includes step S350, predicting the first operation type of the robotic arm according to the tactile pattern and the properties of the contact object; step S360, performing a time series analysis on the tactile data to determine the movement trend of the contact object; and step S370, obtaining the first operation of the robotic arm based on the current posture of the target robotic arm and the position of the end effector, and generating the robotic arm control strategy.

[0034] Preferably, the type of operation that the robotic arm should perform next is predicted based on the identified tactile pattern and the physical properties of the object. For example, if the tactile pattern is identified as pressing and the object is hard, it is predicted that the robotic arm should apply greater force to complete the pressing operation; if the tactile pattern is identified as grasping and the object is soft, it is predicted that a smaller force should be used for gentle grasping to avoid damaging the object; if the tactile pattern is sliding and the surface of the object is smooth, it is predicted that the robotic arm should adjust the friction and contact angle to prevent uncontrolled sliding; a time series analysis is performed on the tactile data, wherein the time series analysis is performed by analyzing the time series data to identify the changing trends of parameters such as force, position, and speed during the contact process between the robotic arm and the object, and by analyzing the changes in contact force over time, the changing trend of the force applied by the robotic arm is judged, and by analyzing the changes in friction over time, it is judged whether the object has a tendency to slide. If If the timing analysis results show that the contact force and friction force remain stable, the object is judged to be at rest; if the analysis shows that the force or friction force changes greatly and has a certain directionality, it may indicate that the object is moving under the action of the robot arm, and the operation mode is adjusted according to the movement trend of the object, such as changing the force direction or adjusting the speed; by analyzing the drastic changes in friction force, it is determined whether the object is sliding, and then corresponding operations (such as increasing friction or adjusting the contact angle) are taken to prevent the object from losing control; the optimal first operation is selected based on the current posture and end effector position of the robot arm, as well as the object movement trend obtained from the timing analysis and tactile pattern, where the posture of the robot arm refers to the angle and direction of each joint of the robot arm, which determines the relative position and direction of the robot arm in space; finally, the robot arm control strategy is generated, including the force size, movement path, speed and contact angle.

[0035] Step S400, based on the robot vision information, the tactile feedback signal is integrated to generate the operation path of the target robot, correct the preset operation trajectory, add it to the robot control strategy, and obtain the robot control update strategy.

[0036] Preferably, a visual sensor (such as a camera, a lidar or a depth sensor) installed on the robot arm obtains real-time environmental data and information such as the three-dimensional position, shape, and posture of the target object, provides key data such as the spatial position, contour, and possible obstacles of the target object, and uses image processing technology (such as edge detection, target recognition, and three-dimensional reconstruction) to convert two-dimensional or three-dimensional visual data into operational data that can be used by the robot arm, such as the precise position, size, and surface features of the object. By combining visual data with tactile feedback signals, the operating environment can be understood more accurately. Visual information provides a global perception at a long distance, while tactile feedback provides detailed information at close range and during contact. Through data fusion algorithms (such as Kalman filtering and Bayesian reasoning), different data sources are comprehensively analyzed to improve the accuracy and stability of the operation path. The position and environmental conditions of the target object are determined based on visual data. Combined with the motion model of the robotic arm, a preliminary operation path is generated, that is, the motion trajectory of the robotic arm is calculated to ensure that the robotic arm can reach the target position from the current state. During the path execution process, real-time tactile feedback signals can help adjust the path. For example, if the robotic arm senses that the force is too large or the surface friction changes during the movement, it may mean that the hardness or texture of the object is inconsistent with expectations. The path and operation method are adjusted according to the tactile feedback signal, such as reducing the speed or adjusting the force. force; the preset trajectory refers to the preliminary running path generated according to the target task and environment before the operation task begins. This trajectory is planned based on preliminary visual data and task requirements. By continuously monitoring visual data and tactile feedback signals during the operation, the deviation between the actual running trajectory of the robot arm and the preset trajectory is detected. For example, a slight change in the position of an object or sliding on an irregular surface may cause the robot arm's trajectory to deviate. According to the detected trajectory deviation, the running trajectory is adjusted and corrected in real time. If the object position changes, the relative position of the robot arm and the target object is recalculated based on the visual feedback, and the running trajectory is adjusted to ensure that the robot arm reaches the target position accurately; if the tactile feedback signal The signal shows that the hardness or surface friction of the object is not consistent with expectations, and the size and direction of the force applied by the robotic arm are adjusted to ensure safe and precise operation; when obstacles or friction changes are detected in the environment, the operating speed of the robotic arm is reduced to prevent excessive contact between the robotic arm and the object or damage to the object; the real-time corrected path and motion parameters are updated to the control strategy of the robotic arm. The control strategy not only includes path planning, but also parameters such as force range, speed, and joint angle. By continuously updating the control strategy, the robotic arm can adjust the operation mode according to real-time changes in the environment to ensure the accuracy and safety of the operation, and finally generate a robotic arm control update strategy, which significantly improves the flexibility and operational reliability of the robotic arm.

[0037] In a possible implementation, step S400 further includes step S410, using a binocular camera to collect image data of the target object; step S420, performing feature point recognition on the image data, smoothing the movement trajectory of the target object, and extracting the spatial position information of the target object; step S430, synchronously fusing the spatial position information and the tactile feedback signal to generate an operation path of the target robotic arm.

[0038] Preferably, the binocular camera captures the target object in real time, and generates two image data from different angles by synchronous capture; through feature point recognition algorithms (such as SIFT, SURF, ORB, etc.), distinguishing feature points, that is, image areas with unique visual features, such as edges, corners or surface textures of objects, are extracted from the images collected by the binocular camera; by tracking the feature points of the target object, the movement trajectory of the object in space can be identified. For example, if the object is displaced or rotated during the operation of the robotic arm, the movement trajectory of the target object is smoothed by a filtering algorithm (such as Kalman filtering, mean filtering or Bessel smoothing, etc.), noise and short-term abnormal changes are eliminated, so that the trajectory is more continuous and stable, ensuring that the robotic arm makes reasonable movements according to the continuous trajectory without being disturbed by image noise, and the target object is calculated by matching the image data captured by the binocular camera with the feature points. The three-dimensional coordinates of the marked object, that is, the precise position of the object in space and the distance relative to the camera, are used to extract the spatial position information of the object's X, Y, and Z axes; synchronous fusion processing refers to ensuring that the image data collected from the binocular camera and the feedback signal obtained by the tactile sensor are synchronized within the same time period, and fusing the spatial position information and tactile feedback data obtained by the binocular camera to obtain a comprehensive perception result of the object's physical position and physical properties. For example, by combining the spatial position of the object (such as X, Y, and Z coordinates) and the tactile feedback signal during contact (such as friction or contact pressure), it is inferred whether the surface of the object moves or slides, whether the contact force is appropriate, etc. Based on the fused spatial position information and tactile feedback signal, an operation path is generated for the robotic arm to ensure that the robotic arm can move from the current position to the target position. At the same time, parameters such as force, speed, and contact angle are adjusted according to the tactile feedback to avoid object damage or contact failure.

[0039] In a possible implementation, step S400 further includes step S440, wherein the preset operation trajectory includes the starting position, target position and multiple path key points of the target robotic arm; step S450, comparing the operation path of the target robotic arm with the preset operation trajectory to detect whether the trajectory deviates; step S460, if the detected trajectory deviates, correcting and adjusting the multiple path key points according to the operation path of the target robotic arm to obtain a corrected operation path of the target robotic arm; step S470, adding the corrected operation path of the target robotic arm to the robotic arm control strategy to obtain a robotic arm control update strategy; step S480, wherein the robotic arm control update strategy also includes a first force range and a first motion speed.

[0040] Preferably, the preset operation trajectory includes the movement path of the robot arm from the starting position to the target position, and the change of the trajectory is defined by multiple path key points. These key points are the preset control points that the robot arm must pass through, ensuring that the robot arm can perform the operation task according to the expected route. During the execution of the task by the robot arm, the actual operation path of the robot arm is monitored in real time and compared with the preset operation trajectory, that is, the deviation between the actual path and each key point in the preset path is calculated. If it is detected that the actual operation path deviates from the preset trajectory, it is judged whether the current operation of the robot arm meets expectations based on the size and direction of the deviation. If not, the actual movement trajectory of the robot arm is readjusted to a trajectory close to the preset trajectory to ensure that the task can be executed as planned, which may include repositioning key points, adjusting the path direction, optimizing key points with large deviations, and generating new key path points; re-planning the corrected operation path of the robot arm, and the corrected operation path is added to the control strategy of the robot arm to form a new control scheme, that is, the robot arm control update strategy, which includes the optimal force range and the optimal movement speed, to ensure that the robot arm can dynamically adjust its movements according to real-time feedback to complete efficient and accurate operation tasks.

[0041] Step S500, controlling the target robotic arm through the robotic arm control update strategy, and if the robotic arm operation status monitoring data is abnormal, entering the robotic arm safety mode.

[0042] Preferably, the robot arm performs operation tasks according to the update strategy, such as grasping, carrying or assembling, etc. The update strategy can dynamically adjust the movement of the robot arm in real time so that it can respond flexibly when facing complex or changing environments. For example, when the tactile feedback of the robot arm detects that the hardness of the object is high, the control strategy is used to reduce the force or adjust the movement path. The robot arm continuously monitors various operation data of the robot arm during the execution of the task to ensure that its operation status is within the normal range, including monitoring whether the force applied by the robot arm when performing the operation exceeds the safety range, monitoring the position, angle, and relative position relationship of each joint of the robot arm to ensure that the joint movement conforms to the preset path, detecting whether the movement speed and acceleration of the robot arm are within the safety threshold range to prevent the robot arm from colliding with or being damaged by the object due to too fast operation, and using the tactile sensor to perceive the pressure, friction, sliding, etc. when the robot arm contacts the object in real time to ensure that there is no loss of control or abnormal contact during the operation; when the operating parameters of the robot arm (such as force, position, speed, etc.) exceed the preset safety threshold, the abnormal situation is identified, for example, If the force applied by the robotic arm exceeds the object's tolerance range, causing damage to the object, that is, excessive force is applied; if the joint position or path of the robotic arm deviates significantly from the preset trajectory, it indicates that there is a problem with the operation accuracy, that is, the motion trajectory deviates; when grasping or moving an object, if the tactile feedback detects a sudden increase or decrease in friction, it may indicate that the object is sliding or in poor contact, that is, an abnormal change in friction; if the movement speed of the robotic arm suddenly increases or decreases, exceeding the safe range, causing loss of control or damage to the object, that is, abnormal speed; when the operating status monitoring data of the robotic arm shows an abnormality, the current task is immediately terminated and the safety mode is triggered. After entering the safety mode, the robotic arm will first immediately stop all current movements to avoid continuing operations that may cause damage or danger. After safely stopping, the robotic arm will return to a safe position to ensure that the robotic arm will not collide with surrounding objects or be further damaged. This position is usually a planned position that can effectively avoid interference with other objects, ensuring the operational safety of the robotic arm and greatly improving the reliability and safety of the operation.

[0043] In a possible implementation, step S500 further includes step S510, monitoring the operating status of the target robotic arm through multiple sensors to obtain multiple key operating data; step S520, if the multiple key operating data exceed a preset threshold, it is regarded as an abnormal operation of the target robotic arm, and the robotic arm safety mode is triggered; step S530, planning a retreat path according to the current position and posture of the target robotic arm, and the target robotic arm retreats to a safe position along the retreat path.

[0044] Preferably, multiple types of sensors are used for status monitoring, including force sensors to monitor whether the force applied by the robot arm exceeds a set range, position sensors to monitor the position and angle of each joint of the robot arm, speed sensors to detect the movement speed of the robot arm, tactile sensors to sense in real time the pressure, friction and vibration when the robot arm contacts an object, and visual sensors to obtain information about the environment around the robot arm in real time through cameras or other visual devices, and obtain multiple key operation data. By analyzing these data in real time, it is determined whether the operating state of the robot arm is normal, and whether the interaction between the robot arm and the target object is as expected; when any key operation data exceeds its preset threshold, it is determined that the operating state is abnormal, for example, the robot arm Exerting too much force may damage the target object, joint position deviation from the preset trajectory may cause misoperation, and abnormal increase in speed may indicate that the robot arm has lost control. Among them, the preset threshold is a set of operating parameter ranges set to ensure the safety of robot arm operation. When it is detected that the operating data exceeds the threshold and the system is judged to be abnormal, the robot arm's safety mode is immediately triggered. First, the current position and posture information of the robot arm is obtained, and a safe return path is planned to return the robot arm from the current state to a safe position to prevent the robot arm from continuing to make erroneous operations in an abnormal state. The robot arm moves step by step along the path until it returns to a safe position, and continuously monitors the operating status of the robot arm's return to ensure that no new abnormalities occur during the return.

[0045] The above specific implementation manner does not constitute a limitation to the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application. In some cases, the actions or steps recorded in the present application can be performed in an order different from that in the embodiment and can still achieve the desired results. In addition, the process depicted in the accompanying drawings does not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An interactive robotic arm control method based on tactile feedback analysis, characterized in that: The method comprises: A high-precision tactile sensor is configured on the end effector of the target robot arm to sense and acquire tactile data, wherein the tactile sensor integrates a multi-layer pressure sensing element; Performing feature analysis on the tactile data to obtain a tactile feedback signal, and based on the tactile feedback signal, traversing the robotic arm tactile model library for comparison to determine the properties of the contact object, wherein the properties of the contact object include the hardness and surface texture of the contact object; Analyzing and identifying the tactile pattern according to the tactile feedback signal and the contact object attribute, predicting the first operation of the robotic arm, and generating a robotic arm control strategy; Based on the visual information of the robot arm, the tactile feedback signal is integrated to generate the operation path of the target robot arm, correct the preset operation trajectory, add it to the robot arm control strategy, and obtain the robot arm control update strategy; The target robotic arm is controlled by the robotic arm control update strategy, and if the robotic arm operation status monitoring data is abnormal, the robotic arm safety mode is entered; The method for generating an operation path of a target robotic arm comprises: Use a binocular camera to collect image data of the target object; Performing feature point recognition on the image data, smoothing the moving trajectory of the target object, and extracting the spatial position information of the target object; Synchronously fusing the spatial position information and the tactile feedback signal to generate an operation path of the target robotic arm; The method of correcting the preset running trajectory and adding it to the robot arm control strategy to obtain the robot arm control update strategy includes: The preset running trajectory includes the starting position, target position and multiple path key points of the target robot arm; Compare the operation path of the target robot arm with the preset operation trajectory to detect whether the trajectory deviates; If the detected trajectory deviates, the plurality of path key points are corrected and adjusted according to the operation path of the target robotic arm to obtain a corrected operation path of the target robotic arm; Adding the corrected operation path of the target robot arm to the robot arm control strategy to obtain a robot arm control update strategy; Wherein, the robot arm control update strategy also includes a first force application range and a first motion speed.

2. The interactive robotic arm control method based on tactile feedback analysis according to claim 1, characterized in that: Performing feature analysis on the tactile data to obtain a tactile feedback signal, the method comprising: Preprocessing the tactile data; Perform feature analysis based on the preprocessing results, including analyzing the contact force frequency domain characteristics, contact force time domain characteristics and contact force instantaneous characteristics, and obtain the contact force, duration and surface vibration of the object; Based on the feature analysis results, a tactile feedback signal is generated.

3. The interactive robotic arm control method based on tactile feedback analysis according to claim 2, characterized in that: The method for constructing the robotic arm tactile model library includes: Acquire multiple categories of target objects and perform contact operations between the end effector of the target robot and various target objects, including pressing, sliding, tapping and grasping; Collecting a plurality of initial tactile data by means of the tactile sensor, and performing standardization processing on the plurality of initial tactile data to obtain a plurality of standard tactile data; Analyzing and extracting a plurality of key tactile features according to the plurality of standard tactile data, and labeling the target object based on the plurality of key tactile features to obtain a labeled object; Establishing a mapping relationship between the tactile feedback signal and the labeled object, training a plurality of tactile feature models, each tactile feature model storing key tactile features of the same type of objects; The multiple tactile feature models constitute the robotic arm tactile model library.

4. The interactive robotic arm control method based on tactile feedback analysis according to claim 1, characterized in that: The analyzing and identifying tactile patterns comprises: Obtain historical cases of the target robotic arm, extract historical operation records and corresponding tactile pattern classification results; Based on a convolutional neural network, the tactile perception model is trained using the historical operation records and the corresponding tactile pattern classification results; Analyzing the tactile feedback signal to obtain tactile feedback analysis results, including contact force magnitude, pressure distribution, and friction coefficient; The tactile feedback analysis result is input into the tactile perception model to identify the tactile pattern.

5. The interactive robotic arm control method based on tactile feedback analysis according to claim 4, characterized in that: The method of predicting the first operation of the robotic arm and generating a robotic arm control strategy includes: predicting a first operation type of the robotic arm according to the tactile pattern and in combination with the contact object attribute; Performing time series analysis on the tactile data to determine the movement trend of the contact object; Based on the current posture of the target robotic arm and the position of the end effector, a first operation of the robotic arm is acquired, and the robotic arm control strategy is generated.

6. The interactive robotic arm control method based on tactile feedback analysis according to claim 1, characterized in that: If the robot arm operation status monitoring data is abnormal, the robot arm enters the safety mode, the method includes: Monitor the operating status of the target robot arm through multiple sensors to obtain multiple key operating data; If the multiple key operation data exceed the preset threshold, it is considered that the target robot arm operation is abnormal, and the robot arm safety mode is triggered; A retraction path is planned according to the current position and posture of the target robot arm, and the target robot arm retracts to a safe position along the retraction path.

Images