Robot finger touch sensing method and system based on artificial intelligence

Abstract

The invention discloses a robot finger tactile perception method and system based on artificial intelligence, and the method comprises the following steps: S1, obtaining visual data and tactile perception initial parameters of a to-be-interacted object of a robot finger, carrying out the preprocessing of the visual data, and extracting the appearance features of the object, performing initial configuration on the touch sensing initial parameters; and S2, object recognition is conducted on the preprocessed visual data through a machine learning model, and the material category, morphological characteristics and surface attributes of an object are determined. According to the robot finger tactile perception method and system based on artificial intelligence, by combining a visual system and tactile sensor data and utilizing a convolutional neural network and Transform fusion architecture to extract local textures and global morphological features of an object, high-precision recognition of material categories, morphological features and surface attributes of the object is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot control, more particularly, to a robot finger tactile perception method and system based on artificial intelligence. BACKGROUND

[0002] With the continuous development of robot technology, robots have been widely applied in many fields such as industrial production, logistics transportation, medical care, etc. In these application scenarios, the interaction operation between robot fingers and objects is crucial, such as grasping, assembling, operating tools, etc. At present, the interaction ability of robot fingers mainly depends on the visual system to obtain the appearance information of the object, but it is difficult to comprehensively and accurately perceive the physical characteristics of the object, such as the material, surface texture, softness and hardness of the object, etc., only relying on visual information.

[0003] Tactile perception, as one of the important ways for humans to perceive the external environment, is also indispensable for robots to complete complex interaction tasks. However, the existing robot finger tactile perception technology has many limitations. On the one hand, the sensitivity and data acquisition response frequency of the tactile sensor are usually fixed, which cannot be adaptively adjusted according to the characteristics of different objects, resulting in the accuracy and stability of tactile perception being affected when facing objects with different materials and shapes. For example, the same tactile sensor parameters may not be able to accurately capture the tactile characteristics of hard smooth objects and flexible porous objects. On the other hand, there is a lack of effective tactile perception parameter optimization mechanism, which makes it difficult to dynamically adjust the perception parameters according to the real-time acquisition of tactile feedback data, so that the robot fingers cannot timely and accurately adapt to the changes of the objects during the interaction process, thereby affecting the precision and reliability of the interaction.

[0004] In addition, the existing robot finger tactile perception method often processes visual and tactile information in isolation, fails to fully integrate the advantages of both, and cannot fully play the role of multi-modal perception in object recognition and interaction control. Therefore, it is of great practical significance to develop a tactile perception method and system that can adaptively adjust tactile perception parameters, effectively integrate visual and tactile information, and realize precise interaction between robot fingers and objects. SUMMARY

[0005] The present application relates to the field of robot control, more particularly, to a robot finger tactile perception method and system based on artificial intelligence.

[0006] The present application achieves the above-mentioned purpose through the following technical solutions: a robot finger tactile perception method based on artificial intelligence, the method comprising the following steps: S1, acquire visual data of an object to be interacted by a robot finger and initial parameters of tactile perception, preprocess the visual data to extract features of the object, and initialize the initial parameters of tactile perception; S2, identify the object through a machine learning model based on the preprocessed visual data to determine a material category, a morphological feature, and a surface attribute of the object; S3, adaptively adjust sensitivity thresholds and data acquisition response frequencies of tactile sensors of the robot finger based on the identification result of the object; S4, control the robot finger to contact the object, collect contact force, pressure distribution, and vibration signals in real time through the tactile sensors, and generate tactile feedback data; S5, input the tactile feedback data into the machine learning model for feature matching and error correction, dynamically optimize tactile perception parameters, and realize precise interaction between the robot finger and the object.

[0007] Further, in step S1, the visual data of the object to be interacted by the robot finger and the initial parameters of tactile perception include: acquire image data of the object through a robot vision system to construct a visual data set, and perform denoising, enhancement, and feature normalization processing on the visual data set; initialize tactile perception parameters, which at least include initial sensitivity, basic response frequency, and contact force threshold of the tactile sensors.

[0008] Further, in step S2: the machine learning model adopts a hybrid architecture combining convolutional neural network and Transformer, and the tactile sensor at least includes a multi-axis force sensor and a flexible piezoresistive array sensor; extract local texture features and global morphological features of the object through the machine learning model, output the material category, morphological feature, and surface attribute of the object after feature fusion, and establish a mapping relationship database between the object attributes and the tactile parameters.

[0009] Further, in step S3, adaptively adjust parameters of the tactile sensor of the robot finger based on the identification result of the object, including: if the identified object is a hard and smooth material, reduce the sensitivity threshold of the multi-axis force sensor and increase its data acquisition response frequency, and maintain the initial parameters of the flexible piezoresistive array sensor; if the identified object is a flexible and porous material, increase the sensitivity threshold of the flexible piezoresistive array sensor and reduce its data acquisition response frequency, and adjust the sensitivity threshold of the multi-axis force sensor; if the identified object is an irregular shape, dynamically adjust the local sensitivity of the flexible piezoresistive array sensor and adjust the sensitivity threshold of the multi-axis force sensor.

[0010] Further, in step S4, the robot finger is controlled to contact the object and collect tactile feedback data, including: The robot finger is controlled to approach the object according to a preset trajectory, and when the force sensor detects that the contact force reaches a preset contact force threshold, data collection is triggered; The contact force and torque signals are collected by a multi-axis force sensor, and the pressure distribution data and vibration signals are collected by a flexible piezoresistive array sensor; The collected tactile data is filtered to remove noise interference, and a standardized tactile feedback data set is generated.

[0011] Further, the robot vision system includes an image acquisition device for acquiring color images and depth images of the object; The feature normalization process is achieved through a preset normalization expression, which uses the mean and standard deviation of each channel pixel as parameters to standardize the pixel values in the visual data.

[0012] Further, the convolutional neural network includes at least multiple convolution stages and feature pooling layers, each convolution stage being composed of several residual blocks for extracting local texture features of the object; The Transformer module includes at least multiple encoder layers, each encoder layer being composed of an attention mechanism and a feedforward neural network for capturing global shape features of the object; The feature vectors output by the convolutional neural network are dimensionally reduced and fused with the global features output by the Transformer module, and then input into a classifier to output the object recognition result.

[0013] Further, the filtering process uses a low-pass filtering algorithm to filter out environmental noise and mechanical vibration interference in the tactile data by setting a preset cutoff frequency. The standardized tactile feedback data set includes at least force / torque signal data collected by the multi-axis force sensor and pressure signal data collected by the flexible piezoresistive array sensor.

[0014] Further, in step S5, the tactile feedback data is input into a machine learning model for optimization, including: The standardized tactile feedback data is compared with preset parameters corresponding to the visual recognition result, and an error value is calculated; If the error value exceeds a preset error threshold, the feature fusion layer weight parameters in the machine learning model are adjusted through an optimization algorithm to update the sensitivity of the tactile sensor and the data collection response frequency configuration; The contact, feedback, and optimization process is repeated until the matching degree of the tactile feedback data and the actual properties of the object reaches a preset standard.

[0015] The application discloses an artificial intelligence-based robot finger tactile perception system for realizing the intelligent noise reduction method based on the mine earphone. A visual data acquisition module is used for acquiring visual data of an object to be interacted with the robot finger. A tactile data acquisition module is used for acquiring tactile data when the robot finger contacts the object, and the tactile data at least includes contact force, pressure distribution and vibration signals. A data processing module is used for pre-processing, feature extraction and object recognition of the visual data and the tactile data. A parameter adjustment module is used for adaptively adjusting parameters of the tactile sensor based on the object recognition result. An interaction control module is used for controlling the robot finger to contact the object and realizing accurate interaction between the robot finger and the object based on the optimized tactile perception parameters.

[0016] The application has the following beneficial effects: 1. By combining the visual system and the tactile sensor data, the local texture and global morphological features of the object are extracted by using the convolutional neural network and the Transformer fusion architecture, and high-precision recognition of the object material category, morphological features and surface properties is realized.

[0017] 2. In the process that the robot finger contacts the object, the real-time collected tactile feedback data is input into the machine learning model for feature matching and error correction, the tactile perception parameters are dynamically optimized, and through the repeated process of contact, feedback and optimization, the matching degree of the tactile feedback data and the actual properties of the object reaches the preset standard, and real-time adjustment and optimization of the tactile perception parameters are realized.

[0018] 3. Through the continuous interaction of the tactile feedback data and the machine learning model, the application can real-time correct the perception error, update the model weight, realize dynamic optimization of the perception accuracy, and improve the interaction stability and authenticity between the robot finger and the object.

[0019] 4. The application can stably run in various complex environments, keep good perception consistency of the object under different light, material and deformation conditions, and significantly improve the reliability of the robot in fine operation tasks. DETAILED DESCRIPTION

[0020] The accompanying drawings used to provide further understanding of the present application, constitute a part of the present application, and the illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute improper limitations on the present application. In the drawings: Fig. 1 It is a method flowchart of the application; Fig. 2 It is a parameter adaptive adjustment logic diagram of the application; Fig. 3 System block diagram of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application; obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0022] Embodiment 1: Please refer to Figs. 1-3 The present application provides a technical solution: a robot finger tactile perception method based on artificial intelligence, the method comprising: S1, acquiring visual data of a robot finger object to be interacted and tactile perception initial parameters, preprocessing the visual data to extract object appearance features, and initializing and configuring the tactile parameters; Wherein, the visual data refers to the image or video information related to the robot finger object to be interacted obtained by the visual sensor, which contains the visual features of the object such as appearance, color, shape, etc. The tactile perception initial parameters are the initial values set when the robot finger tactile perception system starts to work, which are used to define the basic working state and measurement range of the tactile sensor, such as the initial sensitivity of the sensor, the acquisition frequency, etc. The preprocessing is a series of operations on the acquired visual data to remove noise, enhance features, etc., so that it is more suitable for subsequent analysis and processing. The object appearance features are the attributes extracted from the visual data that can describe the appearance of the object, such as color distribution, texture, edge shape, etc. The initialization configuration is to set and adjust the tactile perception initial parameters to achieve an initial state suitable for starting work; S2, identifying the object through a machine learning model on the preprocessed visual data to determine the material category, morphological features and surface properties of the object, the machine learning model adopts a hybrid architecture of ResNet-50 and Transformer, and the tactile perception adopts a 6-axis force sensor and a 16x16 flexible piezoresistive array sensor; The machine learning model is a mathematical model based on data and algorithms, which can learn patterns and rules from input data and be used for prediction, classification, etc. Object recognition is the use of machine learning models to analyze visual data and determine the material category of objects in images or videos, such as metal, plastic, wood, etc. The shape feature is the shape of the object, such as round, square or irregular, and the surface attribute is the smoothness, roughness or texture of the object. ResNet-50 is a deep residual network architecture with 50 layers, which solves the gradient vanishing problem in deep neural network training by introducing residual connections and can learn more complex feature representations, performing well in image classification tasks. The Transformer hybrid architecture combines the Transformer architecture with ResNet-50. Transformer is originally used in natural language processing, which can capture long-range dependencies in data through self-attention mechanisms. By combining it with ResNet-50, the advantages of both can be combined to improve feature extraction and recognition of visual data. The 6-axis force sensor can measure forces and torques in six directions, three translational directions and three rotational directions. The 16x16 flexible piezoresistive array sensor is composed of a flexible piezoresistive sensor array with 16 rows and 16 columns. S3, based on the object recognition result, adaptively adjusting the sensitivity threshold and data acquisition response frequency of the robot finger tactile sensor; The adaptive adjustment is to automatically change the sensitivity threshold and data acquisition response frequency of the robot finger tactile sensor according to the object recognition result. The sensitivity threshold is the minimum force or pressure change value that the tactile sensor can detect. When the contact force or pressure exceeds this threshold, the sensor will produce a corresponding signal output. The data acquisition response frequency is the number of times the tactile sensor collects data per unit time. Higher response frequency can obtain more detailed tactile information, but also increases the burden of data processing. S4, controlling the robot finger to contact the object and collecting the contact force, pressure distribution and vibration signals in real time through the tactile sensor to generate tactile feedback data; Wherein, the control of the robot finger contacting the object is realized by the robot control system, so that the robot finger physically contacts the object to be interacted according to a predetermined trajectory and force; the contact force is the interaction force generated at the contact point when the robot finger contacts the object; the pressure distribution is the distribution of the pressure size and direction of different positions on the surface of the object, and the pressure distribution information of the surface of the object can be obtained through the 16x16 flexible piezoresistive array sensor; the vibration signal is the vibration generated in the process of the robot finger contacting the object due to the elasticity, friction and other factors of the object, and the vibration signal can reflect the internal structure and surface characteristics of the object; the tactile feedback data is the comprehensive information composed of the contact force, the pressure distribution and the vibration signal and other data collected by the tactile sensor, and is used to feedback the contact condition of the robot finger and the object; S5, inputting the tactile feedback data into the machine learning model for feature matching and error correction, dynamically optimizing the tactile perception parameters, and realizing the precise interaction of the robot finger and the object; Wherein, the feature matching is to compare and match the features in the collected tactile feedback data with the known features in the machine learning model, so as to determine which condition in the model is similar to the current contact condition; the error correction is to correct the possible errors in the tactile perception process according to the result of the feature matching; the dynamic optimization of the tactile perception parameters is to adjust the parameters of the tactile perception system in real time, such as the sensitivity threshold, the data acquisition response frequency and the like, according to the result of the error correction, so as to improve the accuracy and adaptability of the tactile perception; the precise interaction is to make the robot finger accurately perceive the tactile characteristics of the object, and to accurately operate, such as grasping, carrying and operating, according to these characteristics, so as to realize the precise interaction with the object.

[0023] It should be noted that during work, the visual data is acquired and preprocessed to extract the appearance features, the initial tactile perception parameters are configured, comprehensive basic information is provided for subsequent operations, the machine learning model with the mixed architecture of ResNet-50 and Transformer is used for object recognition, the object material, shape and surface property are accurately determined, the identification accuracy is improved, the tactile sensor parameters are adaptively adjusted based on the identification result, the adaptability to different objects is enhanced, the tactile feedback data is generated by real-time acquisition of the contact force and other signals, the interaction condition is grasped in time, the tactile feedback data is input into the model for feature matching and error correction, the parameters are dynamically optimized, the precise interaction of the robot finger and the object can be realized, the stability and reliability of the operation are improved, and the robot can better complete the grasping, operation and other tasks in a complex environment.

[0024] In an embodiment, the visual data of the object to be interacted by the robot finger and the initial tactile perception parameters are acquired, including: The RGB image and the depth image of the object are collected by a robot vision system (a 200-megapixel RGB camera + a depth camera) to construct a vision data set wherein , , , corresponding to 30 fps frame rate The vision data is subjected to Gaussian denoising, histogram equalization enhancement and feature normalization processing, and the normalization expression is:

[0025] wherein is the mean value of the pixels in the channel, and the mean values of the RGB channels are 128, 125 and 120 respectively is the standard deviation of the pixels in the channel, and the standard deviations of the RGB channels are 35, 32 and 30 respectively The tactile perception parameters are initialized, including the initial sensitivity of the 6-axis force sensor , the initial sensitivity of the flexible piezoresistive array sensor , the basic response frequency and the contact force threshold .

[0026] In this way, the RGB and depth images of the object are collected by the robot vision system to construct a vision data set, and are subjected to Gaussian denoising and other preprocessing and feature normalization, and the tactile perception parameters are initialized, so that the high-precision vision system can comprehensively obtain the object information, the preprocessing can improve the data quality, the normalization processing can unify the feature scale, which is conducive to subsequent model analysis, the initialization of the tactile parameters can provide a reference for subsequent adaptive adjustment, so that the robot finger can quickly and accurately adjust the tactile perception according to different tasks and object characteristics, enhance the adaptability to various objects, and lay a solid foundation for precise interaction.

[0027] In an embodiment, the preprocessed vision data is subjected to object recognition by a machine learning model, including: A machine learning model based on the combination of a convolutional neural network (CNN) and a Transformer is constructed, the CNN adopts a ResNet-50 architecture, includes 5 convolution stages and 1 global average pooling layer, the 1st-4th convolution stages respectively include 3, 4, 6 and 3 residual blocks, each residual block is composed of 2 3×3 convolution layers and 1 1×1 shortcut connection, and is used to extract local texture features of the object The Transformer module includes 6 encoder layers, each of which is composed of a multi-head self-attention mechanism with 8 heads and a feature dimension of 256, and a feedforward neural network with a hidden layer dimension of 1024, for capturing global shape features of the object. The 2048-dimensional feature vector output by the CNN is reduced to 256 dimensions by a fully connected layer, and is element-wise added to the 256-dimensional global feature output by the Transformer, and is input into a Softmax classifier to output the object material category, such as metal, plastic, fabric, etc., the shape type, such as flat, curved, irregular shape, and the surface attribute, such as smoothness, hardness, etc. A database for mapping object attributes and tactile parameters is established to store the optimal tactile perception configuration parameters corresponding to different categories of objects, for example, the sensitivity of the 6-axis force sensor , the response frequency for a metal material, and the sensitivity of the flexible piezoresistive array sensor , the response frequency for a fabric material.

[0028] In this way, a machine learning model based on the combination of CNN and Transformer is designed for object recognition, and a database for mapping object attributes and tactile parameters is established. The ResNet-50 architecture of the CNN can effectively extract local texture features of the object, and the Transformer module can capture global shape features. The fusion of the two can more comprehensively and accurately identify the object, establish the mapping database, quickly obtain the optimal tactile perception configuration parameters according to the identification result, so that the robot finger does not need to explore from scratch, but directly adjusts the tactile parameters according to the object attributes, improves the interaction efficiency and accuracy, and enhances the processing capability of the robot for different objects.

[0029] In an embodiment, based on the object recognition result, the parameters of the tactile sensor of the robot finger are adaptively adjusted, including: If the identified object is a hard and smooth material, the sensitivity threshold of the 6-axis force sensor is reduced from to , by 20%-30%, the response frequency is increased from to 115-125 , by 15%-25%, and the flexible piezoresistive array sensor maintains the initial parameters to accurately capture subtle changes in contact force; If the identified object is a flexible and porous material, the sensitivity threshold of the flexible piezoresistive array sensor is increased from 5 to 3-3.5 , by 20%-40%, the response frequency is reduced from 100 to 80-90 , by 10%-20%, and the sensitivity of the 6-axis force sensor is adjusted to , avoid signal interference caused by material deformation; If the object is irregular, dynamically adjust the local sensitivity of the flexible piezoresistive array sensor, increase the sensitivity of the sensor unit corresponding to the convex and angular area by 25-35%, from 5 to 3.25-3.75 , the response frequency remains 100 , the sensitivity of the 6-axis force sensor is adjusted to .

[0030] In this way, according to the different materials and shapes of the identified object, the parameters of the robot finger tactile sensor are adaptively adjusted. For hard and smooth materials, the sensitivity threshold of the force sensor is reduced and the response frequency is increased to accurately capture subtle force changes. For flexible porous materials, the parameters of the flexible piezoresistive array sensor are adjusted to avoid signal interference. For irregular objects, the local sensitivity is dynamically adjusted so that the robot finger can flexibly adjust the tactile perception according to the characteristics of the object, improve the contact perception accuracy of objects with different materials and shapes, and more accurately complete tasks such as grasping and operation, and improve the stability and reliability of interaction.

[0031] In an embodiment, the robot finger is controlled to contact the object and collect tactile feedback data, including: The robot finger is controlled to gradually approach the object according to a preset contact trajectory, and when the 6-axis force sensor detects that the contact force reaches the initial threshold , data collection is triggered; Through the 6-axis force sensor, the measurement range is 0- , the accuracy is ±0.1% , the contact force and torque signals in X, Y, Z axis directions are collected in real time, and the sampling interval is determined according to the adjusted response frequency; Through the flexible piezoresistive array sensor, the array size is 16x16, the measurement range of a single unit is 0-50 , the resolution , the pressure distribution data is collected, and the vibration signal is recorded synchronously, and the frequency range is 0-500 ; The collected tactile data is subjected to 5th order Butterworth low-pass filtering, the cutoff frequency is set to 50 , the environmental noise and mechanical vibration interference are removed, and the standardized tactile feedback data set is generated.

[0032] In this way, the robot finger is designed to contact the object along a preset trajectory, and when the contact force reaches a threshold value, data acquisition is triggered, and tactile data is collected through multiple sensors and filtered. Contacting along a preset trajectory can ensure the stability and repeatability of the contact, triggering acquisition can avoid invalid data, multiple sensors can comprehensively obtain information such as contact force, pressure distribution and vibration signals, providing rich data for subsequent analysis, filtering can remove noise interference and generate standardized data sets to improve data quality, so that subsequent models can be based on accurate data for analysis and optimization, and the accuracy and reliability of tactile perception are enhanced.

[0033] In an embodiment, the tactile feedback data is input into a machine learning model for optimization, including: The standardized tactile feedback data, including the 6-dimensional force / torque signal of the 6-axis force sensor and the 256-dimensional pressure signal of the flexible piezoresistive array sensor, are compared with the preset parameters corresponding to the visual recognition result, and the error value is calculated, and the calculation formula is:

[0034] Wherein, is the actual collected force signal, is the preset force parameter, is the actual pressure distribution matrix, is the preset pressure distribution matrix; If the error value (preset threshold), the learning rate is set to , the number of iterations is set to 100, the weight parameters of the feature fusion layer in the machine learning model are adjusted, and the sensitivity of the 6-axis force sensor and the response frequency configuration of the flexible piezoresistive array sensor are updated. Repeat the contact-feedback-optimization process until the tactile feedback data matches the actual properties of the object by more than 90%, that is, , to achieve stable tactile perception interaction.

[0035] In this way, the standardized tactile feedback data is compared with the preset parameters to calculate the error, and if the error is large, the model parameters are adjusted through the gradient descent algorithm, and the process is repeated until the matching degree meets the standard. By comparing and calculating the error, the deviation between the actual collected data and the preset parameters can be clearly understood, the gradient descent algorithm can effectively adjust the weight parameters of the feature fusion layer of the model, optimize the tactile perception configuration, and repeat the contact-feedback-optimization process to enable the robot finger to continuously adapt to the characteristics of the object, improve the matching degree of tactile perception and the actual properties of the object, and achieve stable tactile perception interaction, enhancing the operation ability and adaptability of the robot in complex environments.

[0036] The embodiment is described as follows: 1. Experimental environment and device deployment (1) Hardware device configuration ; Hardware device configuration table (2) Sensor deployment schematic, using textual description Visual system: "side view + overhead view" dual-view deployment, side camera (RGB + depth) captures object side shape and distance, overhead camera (RGB) supplements object top surface texture features, both data are transmitted synchronously to the computing unit through Ethernet.

[0037] Tactile sensor: 6-axis force sensor is connected in series between the finger joint and the end of the robot arm, which can collect real-time three-dimensional force ( 、 、 ) and three-dimensional torque ( 、 、 ) when the finger contacts the object; a flexible piezoresistive array is attached to the finger contact surface with medical-grade adhesive, the array elements are arranged in a matrix, with a spacing of 2mm between adjacent elements to ensure complete coverage of the contact area.

[0038] 2. Model training process (1) Data set construction Visual data set: 10 common objects (metal block, plastic cup, fabric, sponge, wooden block, glass sheet, rubber ball, paper box, ceramic bowl, foam board) are collected, 500 groups of samples are taken for each object under different light and angle, a total of 5000 groups of RGB + depth images, divided into training set, validation set and test set according to the ratio of 7:2:1.

[0039] Tactile data set: control the robot finger to contact the above 10 types of objects with different forces ( ), collect 1000 groups of force signal and pressure distribution data for each type of object, label the object material and shape, and correspond to the visual data.

[0040] (2) Training steps Pre-training stage: use ImageNet data set to pre-train ResNet-50 model, freeze the first 3 convolution stages, only train the 4-5 convolution stages and fully connected layer, learning rate is set to , iterate 50 times, so that the model has the ability to extract object texture.

[0041] Fusion training stage: combine the pre-trained ResNet-50 with 6-layer Transformer (head number 8, feature dimension 256), input the visual data set to train the feature fusion layer, use cross-entropy loss function to optimize the classification result, dynamically adjust the learning rate (initial 5e-5, decay to 0.8 times of the previous round every 10 rounds), after 100 rounds of training, the object recognition accuracy of the model reaches 96.3%.

[0042] End-to-end optimization phase: input the "visual data - tactile parameter" paired data set, take the tactile feedback error (E) as the optimization objective, adjust the model weight with the Adam optimizer (β1=0.9, β2=0.999), and after 80 iterations, the error threshold stabilizes at E≤0.1N+2kPa, meeting the interaction accuracy requirements.

[0043] 3. Experimental results and analysis (1) Object recognition and parameter adjustment effect ; Note: The interaction success rate is defined as the proportion of the number of experiments in which the robot finger successfully grasps / contacts the object without damage to the total number of experiments, and each group of objects is repeated 100 times.

[0044] (2) Comparison of key performance indicators ; 4. Conclusion of the embodiment This embodiment realizes precise tactile perception of the robot finger for objects of different materials and shapes through the closed-loop process of "visual recognition - tactile adaptive adjustment - feedback optimization". The experimental results show that this method is superior to traditional methods in terms of parameter adjustment response speed, force control accuracy, and interaction success rate, and can meet the high-precision requirements of robot tactile perception in industrial grasping, medical care, and other scenarios.

[0045] Sensor calibration process The sensor needs to be calibrated regularly after deployment to avoid errors caused by mechanical wear and tear and changes in environmental temperature. The specific process is as follows: (1) 6-axis force sensor calibration, once every 30 days.

[0046] ① Zero calibration: Place the robot finger in an unloaded state (no contact, no external force), start the calibration program, collect 100 sets of unloaded data from the sensor, calculate the zero point offset value of each axis , , , , , , and store it in the system compensation library. The subsequent data collection automatically subtracts the corresponding offset value.

[0047] ② Range calibration: Use standard weights with an accuracy of ±0.001 , apply standard forces of 5 , 10 , 20 , 30 , and 50 in the X, Y, and Z axis directions, and apply a force of 0.5 , 1 , 2 , 3 , 5 The standard torque of the standard moment is recorded, the deviation of the sensor output value from the standard value is recorded, a calibration curve is fitted by least squares method, and the sensor accuracy compensation parameter is updated.

[0048] ③ Temperature compensation: in the environment temperature range of 0-40℃, collect 1 time of empty load and full load 50 Force data every 5℃, establish temperature-error mapping model, adjust output value according to environment temperature sensor data in real time, ensure that error≤±0.05%FS when temperature changes±10℃.

[0049] (2) Flexible piezoresistive array sensor calibration, once every 15 days.

[0050] ① Unit calibration: use micro-pressure calibrator, accuracy ±0.01kPa, apply 0kPa, 5kPa, 10kPa, 20kPa, 30kPa, 50kPa standard pressure to 256 units of 16×16 array one by one, record resistance value change of each unit, establish "pressure-resistance" corresponding relationship table, mark failed units, deviation>5%, and enable standby units.

[0051] ② Consistency calibration: apply 20kPa standard pressure to all normal units, calculate standard deviation of each unit output value, if standard deviation>1kPa, adjust gain coefficient of each unit through software, so that array output consistency error≤3%.

[0052] ③ Fatigue calibration: simulate 1000 times of contact action of robot finger, contact force 0.5-5N, after ending, repeat steps ①-②, compensate sensitivity attenuation caused by material fatigue, ensure that sensor accuracy remains stable within 1000 cycles after calibration.

[0053] An artificial intelligence-based robot finger tactile perception system for implementing the above-mentioned mine earphone-based intelligent noise reduction method, comprising: A visual data acquisition module for acquiring visual data of an object to be interacted with the robot finger; A tactile data acquisition module for acquiring tactile data when the robot finger contacts the object, the tactile data at least including contact force, pressure distribution and vibration signal; A data processing module for pre-processing, feature extraction and object recognition of the visual data and the tactile data; A parameter adjustment module for adaptively adjusting parameters of the tactile sensor based on the object recognition result; An interaction control module is configured to control the robot finger to contact the object and to realize accurate interaction between the robot finger and the object based on the optimized tactile perception parameters.

[0054] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of being implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0055] The above embodiments have been described in detail, and the principles and embodiments of the present application have been described by applying specific examples. The above embodiment descriptions are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific embodiments and application scope will be changed, and the above description should not be understood as limiting the present application.

Claims

1. An artificial intelligence-based robot finger tactile perception method, characterized by, The method comprises the following steps: S1, acquiring visual data of an object to be interacted by a robot finger and initial parameters of tactile perception, preprocessing the visual data to extract object appearance features, and initializing and configuring the initial parameters of tactile perception; S2, identifying the object through a machine learning model based on the preprocessed visual data to determine the material category, morphological features and surface properties of the object; S3, adaptively adjusting the sensitivity threshold and data acquisition response frequency of the tactile sensor of the robot finger based on the object recognition result; S4, controlling the robot finger to contact the object and collecting tactile feedback data in real time through the tactile sensor; S5, inputting the tactile feedback data into the machine learning model for feature matching and error correction, dynamically optimizing the tactile perception parameters, and realizing precise interaction between the robot finger and the object. 2.The artificial intelligence-based robot finger haptic perception method of claim 1, wherein, In step S1, the visual data of the object to be interacted by the robot finger and the initial parameters of tactile perception are acquired, including: Collecting image data of the object through a robot vision system to construct a visual data set, and performing denoising, enhancement and feature normalization processing on the visual data set; Initializing tactile perception parameters, which at least include the initial sensitivity, basic response frequency and contact force threshold of the tactile sensor. 3.The artificial intelligence-based robot finger haptic perception method of claim 1, wherein, In step S2: The machine learning model adopts a hybrid architecture combining convolutional neural network and Transformer, and the tactile sensor at least includes a multi-axis force sensor and a flexible piezoresistive array sensor; The local texture features and global morphological features of the object are extracted through the machine learning model, and the material category, morphological features and surface properties of the object are output after feature fusion, and a mapping relationship database of object properties and tactile parameters is established. 4.The artificial intelligence-based robot finger haptic perception method of claim 1, wherein, In step S3, the parameters of the tactile sensor of the robot finger are adaptively adjusted based on the object recognition result, including: If the identified object is a hard and smooth material, the sensitivity threshold of the multi-axis force sensor is reduced and the data acquisition response frequency is increased, and the initial parameters of the flexible piezoresistive array sensor are maintained; If the identified object is a flexible and porous material, the sensitivity threshold of the flexible piezoresistive array sensor is increased and the data acquisition response frequency is reduced, and the sensitivity threshold of the multi-axis force sensor is adjusted; If the identified object is irregular in shape, the local sensitivity of the flexible piezoresistive array sensor is dynamically adjusted, and the sensitivity threshold of the multi-axis force sensor is adjusted. 5.The artificial intelligence-based robot finger haptic perception method of claim 1, wherein, In step S4, the robot finger is controlled to contact the object and collect tactile feedback data, including: The robot finger is controlled to approach the object according to a preset trajectory, and when the tactile sensor detects that the contact force reaches a preset contact force threshold, data acquisition is triggered; The contact force and torque signals are collected through the multi-axis force sensor, and the pressure distribution data and vibration signals are collected through the flexible piezoresistive array sensor; The collected tactile data is filtered to remove noise interference, and a standardized tactile feedback data set is generated.

6. The artificial intelligence-based robot finger tactile perception method according to claim 2, characterized in that: The robot vision system comprises an image acquisition device for collecting color images and depth images of the object; The feature normalization processing is realized by a preset normalization expression, and the normalization expression is parameterized by the mean and standard deviation of each channel pixel, and the pixel value in the visual data is standardized and converted.

7. The artificial intelligence-based robot finger tactile perception method according to claim 3, characterized in that: The convolutional neural network at least comprises a plurality of convolution stages and feature pooling layers, each convolution stage being composed of a plurality of residual blocks for extracting local texture features of the object. The Transformer module at least comprises a plurality of encoder layers, each encoder layer being composed of an attention mechanism and a feedforward neural network for capturing global shape features of the object. The feature vectors output by the convolutional neural network are dimensionally reduced and fused with the global features output by the Transformer module, and then input into a classifier to output object recognition results.

8. The artificial intelligence based robotic finger haptics perception method of claim 5, wherein: The filtering processing adopts a low-pass filtering algorithm, sets a preset cutoff frequency, filters out environmental noise and mechanical vibration interference in the tactile data, and the standardized tactile feedback data set at least comprises force / torque signal data collected by a multi-axis force sensor and pressure signal data collected by a flexible piezoresistive array sensor. 9.The artificial intelligence-based robot finger haptic perception method of claim 1, wherein, In step S5, the tactile feedback data is input into a machine learning model for optimization, including: Comparing the standardized tactile feedback data with preset parameters corresponding to the visual recognition result to calculate an error value; If the error value exceeds a preset error threshold, adjusting the feature fusion layer weight parameters in the machine learning model through an optimization algorithm to update the sensitivity of the tactile sensor and the data acquisition response frequency configuration; Repeating the contact, feedback and optimization process until the matching degree of the tactile feedback data and the actual properties of the object reaches a preset standard.

10. An artificial intelligence-based robot finger tactile perception system for implementing the artificial intelligence-based intelligent noise reduction method based on mine earphones according to any one of claims 1-9, characterized in that, It comprises: a visual data acquisition module for acquiring visual data of an object to be interacted with by a robot finger; a tactile data acquisition module for acquiring tactile data when the robot finger contacts the object, the tactile data at least including contact force, pressure distribution and vibration signal; a data processing module for pre-processing, feature extraction and object recognition of the visual data and the tactile data; a parameter adjustment module for adaptively adjusting parameters of the tactile sensor based on the object recognition result; an interaction control module for controlling the robot finger to contact the object and realizing precise interaction between the robot finger and the object based on the optimized tactile perception parameters.

Images