Robotic hardness prediction method and system based on haptic information and continuous learning

Abstract

The application discloses a robot hardness prediction method and system based on tactile information and continuous learning, and the method comprises the following steps: acquiring hardness classification data sets of different objects collected by a robot, and dividing the hardness classification data sets into a training set and a test set after preprocessing; training and testing an integrated playback model by using the training set and the test set to obtain a trained conditional tactile continuous learning ordered network model; acquiring tactile data of a target object grabbed by the robot, inputting the tactile data into the trained conditional tactile continuous learning ordered network model, and outputting a hardness prediction result. The application reduces the neural network parameter quantity, accelerates the training speed of the model, increases the conditional information in the middle layer of the model to control the type of samples generated by the model, balances the distribution of the generated samples, has good generalization in different neighborhood incremental scenes, and improves the hardness prediction accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of robotics, and in particular to a method, system, terminal, and computer-readable storage medium for predicting robot hardness based on tactile information and continuous learning. Background Technology

[0002] Currently, tactile sensation, as an important intelligent sensory mode, has attracted the attention of researchers at home and abroad. Researchers use different types of tactile sensors to perceive, identify, and classify the surface roughness, material, contact position, contact pressure, and hardness of objects.

[0003] However, in practical applications, it is difficult to collect and label large-scale data for all application scenarios. Training can only be performed using data collected in specific environments. Therefore, the training data is in a closed, static environment, and these data are independent and identically distributed (the data are independent of each other and have the same probability distribution) with a constant distribution. In actual applications, the test scenarios are open, and the objects, temperature conditions, environmental background factors, etc., will cause the data to be completely different from the training data. This results in the model trained with a limited dataset being limited when facing open scenarios. Therefore, finding the optimal learning model and continuous learning algorithm for tactile data in different scenarios is an objective need for the practical application of tactile robots.

[0004] In robotics, researchers typically employ three continuous learning methods to address the catastrophic forgetting problem: memory replay, regularization, and knowledge distillation. Memory replay mitigates forgetting by replaying past task data. For example, a Growing Dual Memory (GDM) architecture is introduced, enabling robots to progressively recognize each individual's facial expressions. A generative learning network is designed to generate pseudo-samples for incremental tactile recognition tasks. A generative model is used for experience replay, allowing robots to continuously learn in various physical environments. A basic generative method is proposed to simulate tasks such as opening doors and latching latches in an environment. Regularization introduces additional regularization terms during neural network training, penalizing significant changes in important parameters while learning new knowledge. For example, Elastic Weight Consolidation (EWC) is applied to enable autonomous mobile robots to continuously and accurately predict pedestrian movements. EWC is also used to enable soft robots to continuously adapt to different external loads. Knowledge distillation transfers knowledge from a teacher model to a student model. A multi-task knowledge distillation method is proposed, refining large datasets for each task into smaller datasets, significantly reducing the required training samples. For example, a distillation loss constraint method is introduced, which uses knowledge distillation loss to constrain the probability distribution on new tasks to be close to the distribution learned on previous tasks, thus realizing cross-modal material perception from vision to touch.

[0005] Currently, research on continuous learning and domain adaptation in the field of robotic tactile perception is limited. Most related algorithms are applied primarily to publicly available visual datasets, and few demonstrate equally excellent performance in robotics. Therefore, designing experimental scenarios and studying domain increment in robotic tactile perception is a crucial and urgent research topic. In other words, existing technologies suffer from catastrophic forgetting phenomena resulting from continuous neural network learning during tactile robot perception, redundant model parameters, and long training times, leading to low prediction accuracy when forecasting object hardness.

[0006] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0007] The main objective of this invention is to provide a robot hardness prediction method, system, terminal, and computer-readable storage medium based on tactile information and continuous learning. This invention aims to solve the problems in the prior art where the catastrophic forgetting phenomenon caused by continuous learning of neural networks during the perception process of tactile robots, redundant model parameters, long training time, and low prediction accuracy of the model when predicting the hardness of objects are all problems.

[0008] To achieve the above objectives, the present invention provides a robot hardness prediction method based on tactile information and continuous learning, the robot hardness prediction method based on tactile information and continuous learning comprising the following steps:

[0009] Obtain a hardness classification dataset of different objects collected by the robot, and divide the hardness classification dataset into a training set and a test set after preprocessing;

[0010] The ensemble playback model is trained and tested using the training set and the test set to obtain a trained conditional haptic continuous learning ordered network model.

[0011] The robot acquires tactile data of grasping a target object, inputs the tactile data into a trained conditional tactile continuous learning ordered network model, and outputs the hardness prediction result of the target object.

[0012] Optionally, the robot hardness prediction method based on tactile information and continuous learning, wherein acquiring a hardness classification dataset of different objects collected by the robot, and preprocessing the hardness classification dataset into a training set and a test set, specifically includes:

[0013] The robot acquires tactile array data and torque data of multiple objects through capacitive tactile array sensors and torque sensors.

[0014] The tactile array data and the torque data are preprocessed to obtain a hardness classification dataset. The preprocessing includes downsampling and size transformation. The hardness classification dataset is divided proportionally to obtain a training set and a test set.

[0015] Optionally, the robot hardness prediction method based on tactile information and continuous learning, wherein the step of training and testing the ensemble playback model using the training set and the test set to obtain a trained conditional tactile continuous learning ordered network model specifically includes:

[0016] An integrated playback model is constructed, and conditional information is added to the middle layer of the integrated playback model to obtain a conditional haptic continuous learning network model. The spatial features of the conditional haptic continuous learning network model are sorted to obtain a conditional haptic continuous learning ordered network model.

[0017] The conditional tactile continuous learning ordered network model is trained using the training set, and the test set is input into the conditional tactile continuous learning ordered network model for testing. If the test results meet the preset requirements, the trained conditional tactile continuous learning ordered network model is obtained.

[0018] Optionally, in the robot hardness prediction method based on tactile information and continuous learning, the integrated playback model includes a discriminator and a generator.

[0019] The loss function minimized by the integrated playback model for:

[0020]

[0021] in, This represents the loss function of the discriminator; α represents the loss function of the generator; α represents the hyperparameter used to balance the weights between the discrimination loss and the generation loss.

[0022] Optionally, in the robot hardness prediction method based on tactile information and continuous learning, the conditional information is used to control the type of samples generated by the integrated playback model and to unify the category features output by the encoder into a standard normal distribution.

[0023] Optionally, in the robot hardness prediction method based on tactile information and continuous learning, the conditional tactile continuous learning ordered network model combines the feature distribution in the latent space with the order between hardness categories to sort the feature space.

[0024] The loss function minimized by the conditional tactile continuous learning ordered network model for:

[0025]

[0026] in, and μ i This refers to the mean and variance of the normal distribution of the feature distribution of sample i in the latent space; step represents the hyperparameter used to adjust the distance between different class distributions in the latent space; y i This represents the label information for sample i.

[0027] Optionally, the robot hardness prediction method based on tactile information and continuous learning, wherein acquiring tactile data of the robot grasping the target object, inputting the tactile data into a trained conditional tactile continuous learning ordered network model, and outputting the hardness prediction result of the target object, specifically includes:

[0028] The robot acquires target tactile array data and target torque data of the target object through capacitive tactile array sensors and torque sensors.

[0029] The target tactile array data and the target torque data are preprocessed to obtain target tactile data. The target tactile data is then input into a trained conditional tactile continuous learning ordered network model to predict the hardness of the target object, and the predicted hardness of the target object is output.

[0030] Furthermore, to achieve the above objectives, the present invention also provides a robot hardness prediction system based on tactile information and continuous learning, wherein the robot hardness prediction system based on tactile information and continuous learning includes:

[0031] The data acquisition module is used to acquire hardness classification datasets of different objects collected by the robot, and to preprocess the hardness classification datasets into training sets and test sets.

[0032] The model training module is used to train and test the ensemble playback model using the training set and the test set to obtain a trained conditional haptic continuous learning ordered network model.

[0033] The hardness prediction module is used to acquire tactile data of the robot grasping the target object, input the tactile data into a trained conditional tactile continuous learning ordered network model, and output the hardness prediction result of the target object.

[0034] Furthermore, to achieve the above objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a robot hardness prediction program based on tactile information and continuous learning stored in the memory and executable on the processor, wherein when the robot hardness prediction program based on tactile information and continuous learning is executed by the processor, it implements the steps of the robot hardness prediction method based on tactile information and continuous learning as described above.

[0035] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a robot hardness prediction program based on tactile information and continuous learning, and the robot hardness prediction program based on tactile information and continuous learning, when executed by a processor, implements the steps of the robot hardness prediction method based on tactile information and continuous learning as described above.

[0036] In this invention, a hardness classification dataset of different objects collected by a robot is obtained. This dataset is preprocessed and divided into a training set and a test set. The ensemble playback model is trained and tested using the training and test sets to obtain a trained conditional tactile continuous learning ordered network model. Tactile data of the robot grasping the target object is obtained and input into the trained conditional tactile continuous learning ordered network model, outputting a hardness prediction result for the target object. This invention reduces the number of neural network parameters while accelerating model training. Furthermore, by adding conditional information to the intermediate layers of the model, the type of samples generated by the model is controlled, resulting in a more balanced distribution of generated samples. This leads to good generalization under different neighborhood increment scenarios, improving the accuracy of the model in predicting hardness. Attached Figure Description

[0037] Figure 1 This is a flowchart of a preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning of the present invention;

[0038] Figure 2 This is a schematic diagram of the incremental experimental scenario setup in the field of hardness recognition in a preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning of the present invention.

[0039] Figure 3 This is a schematic diagram of the integrated playback model architecture in a preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning of the present invention.

[0040] Figure 4 This is a schematic diagram of the conditional tactile continuous learning network model architecture in a preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning of the present invention.

[0041] Figure 5This is a schematic diagram of the conditional tactile continuous learning ordered network model architecture in a preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning of the present invention.

[0042] Figure 6 This is a schematic diagram of the continuous learning algorithm processing framework in a preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning of the present invention.

[0043] Figure 7 This is a schematic diagram illustrating the principle of a preferred embodiment of the robot hardness prediction system based on tactile information and continuous learning of the present invention.

[0044] Figure 8 This is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0045] This invention primarily proposes a deep learning method for incremental scenarios in the field of tactile robotics (a classic scenario in continuous learning problems, similar to domain adaptation tasks, referring to neural networks continuously learning the same task in different scenarios). Based on a generative playback network, it uses pseudo-samples of old tasks generated for training to mitigate the catastrophic forgetting phenomenon caused by continuous learning of neural networks during the tactile robot's perception process. By integrating the generative and discriminative models, the number of neural network parameters is reduced while accelerating the model's training speed. Furthermore, by adding conditional information to the intermediate layers of the model, the type of samples generated by the model is controlled, resulting in a more balanced distribution of generated samples. In addition, this method introduces a special loss function that combines the feature distribution in the latent space with the order between hardness categories, sorting the feature space and utilizing the spatial relationships between different categories to improve the quality of the generated pseudo-images, further enhancing the model's prediction accuracy.

[0046] This invention can maintain accuracy on old tasks while continuously learning multiple tasks, and it has a smaller number of network parameters, shorter training time, and can achieve higher accuracy.

[0047] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0048] The preferred embodiment of the robot hardness prediction method based on tactile information and continuous learning described in this invention, such as... Figure 1 As shown, the robot hardness prediction method based on tactile information and continuous learning includes the following steps:

[0049] Step S10: Obtain the hardness classification dataset of different objects collected by the robot, and divide the hardness classification dataset into training set and test set after preprocessing.

[0050] Specifically, the robot acquires tactile array data and torque data of multiple objects through capacitive tactile array sensors and torque sensors; the tactile array data and torque data are preprocessed to obtain a hardness classification dataset (i.e., a robot hardness classification dataset based on tactile information is obtained through autonomous acquisition, and the hardness classification dataset is processed to simulate a continuous learning scenario of incremental tactile robot domain). The preprocessing includes downsampling and size transformation, etc. The hardness classification dataset is divided proportionally to obtain a training set and a test set.

[0051] This invention employs a tactile robot to collect hardness data from silicone blocks of varying hardness. Tactile array data and torque data are collected using a capacitive tactile array sensor and a torque sensor integrated into a BarrettHand, respectively. For the capacitive tactile array sensor, the data sampling frequency is 200Hz. Each capacitive tactile array sensor has 24 sensing units, including 15 units with an area of ​​0.3 cm². 2 The unit and 9 with an area of ​​0.15 cm² 2 The experiment used silicone samples in three shapes: 1) cuboid, 2) cuboid with protrusions, and 3) flower. These shapes were designated as shapes A, B, and C, respectively. Each shape had 13 different hardness grades, and a total of 39 silicone samples were made, each with 13 hardness grades.

[0052] The data was preprocessed using a BarrettHand to collect tactile images. The BarrettHand has three fingers, each with a tactile sensor and a torque sensor. Each tactile sensor provides an 8×3 tactile array, and the three fingers provide a 72 (9×8) tactile image. To utilize the torque sensor information, the three torque values ​​from the three sensors were concatenated into a 1×3 matrix, which was then copied three times. The tactile matrix and the torque matrix were then concatenated to form a 9×9 tactile image. During the pressing process, a time-step sequence T=67 was captured. The starting point of the sequence was set to the moment the finger just touched the silicone sample. Next, the sequence was downsampled to a step size of 12. Therefore, a tactile image sequence with a step size of 12 was obtained, with each frame being a 9×9 tactile image.

[0053] Experimental scenarios were designed based on the collected dataset, and the continuous learning experimental design for the tactile hardness recognition task was as follows: Figure 2As shown: Shapes A, B, and C are trained in chronological order (that is, the hardness data of three different surfaces are sequentially fed into the model for training and testing. When learning a new task, the old task data cannot be used for training, but the performance is still tested on the old task. The purpose is to enable the model to update online and not forget the old task while learning a new task). The task objective remains unchanged (identify 13 hardness levels), and the surface of the silicone sample is different for each task.

[0054] Step S20: Use the training set and the test set to train and test the integrated playback model to obtain a trained conditional haptic continuous learning ordered network model.

[0055] Specifically, an integrated playback model is constructed, and conditional information is added to the intermediate layer of the integrated playback model to obtain a conditional haptic continuous learning network model. The spatial features of the conditional haptic continuous learning network model are sorted to obtain a conditional haptic continuous learning ordered network model. The conditional haptic continuous learning ordered network model is trained using the training set, and the test set is input into the conditional haptic continuous learning ordered network model for testing. If the test results meet the preset requirements, the trained conditional haptic continuous learning ordered network model is obtained.

[0056] Compared to standard playback methods, this invention focuses more on the number of model parameters in the robotics field. Playback methods that simultaneously incorporate a discriminative model (discriminator) and a generative model (generator) are highly effective, but significantly increase model size and training time. The model improvements used in this invention are as follows: Figure 3 As shown, the discriminative model (discriminator) and the generative model (generator) are combined into one model, allowing the ensemble replay model to train the generative function simultaneously, thus compressing the model size and training time. Figure 3 In this context, FC represents a fully connected layer, and z is the structure in the autoencoder, representing the feature distribution in the latent space after the data passes through the encoder.

[0057] For the ensemble playback model, the loss function minimized by the ensemble playback model is... for:

[0058]

[0059] in, This represents the loss function of the discriminator; α represents the loss function of the generator; α represents the hyperparameter used to balance the weights between the discrimination loss and the generation loss.

[0060] This invention improves upon conventional playback methods by proposing an integrated model based on a variational autoencoder, which integrates generation and discrimination capabilities within a single model. This method offers smaller model size and higher accuracy compared to other playback methods.

[0061] Since the generation of ensemble replay models is uncontrollable, this invention attempts to add conditional information (which involves including labels in the model training and allowing custom sample labels to be input later to control the type of generated samples) to the intermediate layer of the ensemble replay model. This controls the type of samples generated by the model, resulting in a more balanced distribution of generated samples. The basic structure of the conditional haptic continuous learning network model is as follows: Figure 4 As shown.

[0062] This invention introduces label information to make the feature space of the autoencoder controllable, and controls the type distribution of generated pseudo samples through labels, thus solving the problem of unbalanced data distribution generated by the autoencoder and further improving the performance of the model.

[0063] By introducing conditional information into continuous learning, the types of model generation can be controlled, and the categorical features output by the encoder can be unified into a standard normal distribution. Next, the feature distribution in the latent space is combined with the order among hardness categories to sort the feature space, thereby utilizing the spatial relationships between different categories to improve the quality of the generated pseudo-images. Figure 5 As shown, for a conditional haptic continuous learning ordered network model, the loss function minimized by the conditional haptic continuous learning ordered network model is... for:

[0064]

[0065] in, and μ i This refers to the mean and variance of the normal distribution of the feature distribution of sample i in the latent space (Z); step represents the hyperparameter used to adjust the distance between different class distributions in the latent space; y i This represents the label information for sample i.

[0066] This invention introduces a modified loss function into the model and utilizes the potential linear relationship between hardness information to improve sample quality and classification accuracy.

[0067] Specifically, such as Figure 6As shown, the experimental scenario is designed based on the collected dataset. A hardness classification dataset is collected and preprocessed, including downsampling and size transformation, and divided into training and testing sets. The process of continuously learning multiple tasks by the model is completed according to the training and testing sets. The present invention is used to generate a playback algorithm for the continuous learning method of tactile robots. It uses a deep neural network for playback. When the model learns a new task, old pseudo-samples are played back, thereby mitigating the catastrophic forgetting problem in continuous learning. At the same time, by optimizing the network, the number of model parameters is reduced, the training time is shortened, and the model performance is improved.

[0068] Step S30: Obtain tactile data of the target object grasped by the robot, input the tactile data into the trained conditional tactile continuous learning ordered network model, and output the hardness prediction result of the target object.

[0069] Specifically, the robot acquires target tactile array data and target torque data of the target object through capacitive tactile array sensors and torque sensors; the target tactile array data and the target torque data are preprocessed to obtain target tactile data; the target tactile data is input into a trained conditional tactile continuous learning ordered network model to predict the hardness; and the hardness prediction result of the target object is output.

[0070] Beneficial effects:

[0071] (1) The present invention proposes a continuous learning method for tactile robots to generate playback algorithms, which is used to solve the domain incremental learning scenario faced by tactile robots in open environments, ensuring that the robot has the ability to continuously learn multiple tasks, and can have good generalization in different neighborhood incremental scenarios without changing the basic framework.

[0072] (2) The present invention designs an integrated model of generation and discrimination. The present invention can perform both discrimination and generation functions through an integrated model. Compared with the mainstream playback method model, the model of the present invention has fewer parameters, faster training speed and higher performance.

[0073] (3) The present invention adds conditional information to the model, which makes the types of replay samples controllable, solves the problem of uneven distribution of sample types generated by mainstream models, and improves the performance and stability of the model.

[0074] (4) This invention introduces the idea of ​​ordered networks into the network, introduces a special loss function, and combines the characteristics of the dataset to serialize the distribution of sample features in the latent space, thereby further improving the model performance.

[0075] Compared with existing continuous learning algorithms, this invention improves model performance and reduces model parameters and training time by integrating a discriminative model and a generative model. This invention introduces conditional information in feature extraction, which helps to solve the long-tail problem of replay sample types and makes the distribution of pseudo sample types in replay controllable. This invention improves model performance by introducing the idea of ​​sequence networks and using a special loss function to serialize the distribution of features in the latent space.

[0076] The network structure proposed in this invention has been well validated on a hardness dataset, showing improved performance compared to current mainstream models, and reducing the number of model parameters and training time, proving the feasibility of the method.

[0077] Furthermore, the integrated network used in this invention, whose encoder, decoder and classification layer network architecture can be replaced by CNN, LSTM or attention module, and the domain incremental dataset based on tactile robots as input in this invention can theoretically be replaced by other modal datasets, such as visual robot datasets or computer vision datasets.

[0078] Furthermore, such as Figure 7 As shown, based on the above-mentioned robot hardness prediction method based on tactile information and continuous learning, the present invention also provides a robot hardness prediction system based on tactile information and continuous learning, wherein the robot hardness prediction system based on tactile information and continuous learning includes:

[0079] Data acquisition module 51 is used to acquire hardness classification datasets of different objects collected by the robot, and divide the hardness classification datasets into training set and test set after preprocessing.

[0080] Model training module 52 is used to train and test the integrated playback model using the training set and the test set to obtain a trained conditional haptic continuous learning ordered network model.

[0081] The hardness prediction module 53 is used to acquire tactile data of the robot grasping the target object, input the tactile data into the trained conditional tactile continuous learning ordered network model, and output the hardness prediction result of the target object.

[0082] Furthermore, such as Figure 8 As shown, based on the above-mentioned robot hardness prediction method and system based on tactile information and continuous learning, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 8 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0083] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal. Further, the memory 20 may include both internal and external storage devices. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code installed on the terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a robot hardness prediction program 40 based on tactile information and continuous learning. This robot hardness prediction program 40 based on tactile information and continuous learning can be executed by the processor 10, thereby implementing the robot hardness prediction method based on tactile information and continuous learning in this application.

[0084] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, used to run program code stored in the memory 20 or process data, such as executing the robot hardness prediction method based on tactile information and continuous learning.

[0085] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0086] In one embodiment, when the processor 10 executes the robot hardness prediction program 40 based on tactile information and continuous learning in the memory 20, it implements the steps of the robot hardness prediction method based on tactile information and continuous learning as described above.

[0087] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a robot hardness prediction program based on tactile information and continuous learning, and the robot hardness prediction program based on tactile information and continuous learning, when executed by a processor, implements the steps of the robot hardness prediction method based on tactile information and continuous learning as described above.

[0088] In summary, this invention provides a robot hardness prediction method and related equipment based on tactile information and continuous learning. The method includes: acquiring a hardness classification dataset of different objects collected by the robot; preprocessing the hardness classification dataset and dividing it into a training set and a test set; training and testing an ensemble playback model using the training set and the test set to obtain a trained conditional tactile continuous learning ordered network model; acquiring tactile data of the robot grasping the target object; inputting the tactile data into the trained conditional tactile continuous learning ordered network model; and outputting the hardness prediction result of the target object. This invention reduces the number of neural network parameters while accelerating the model's training speed. Furthermore, by adding conditional information to the intermediate layers of the model to control the type of samples generated by the model, the distribution of generated samples becomes more balanced, exhibiting good generalization ability in different neighborhood incremental scenarios and improving the accuracy of the model in predicting hardness.

[0089] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, object, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, object, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, object, or terminal that includes that element.

[0090] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0091] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A robot hardness prediction method based on tactile information and continuous learning, characterized in that, The robot hardness prediction method based on tactile information and continuous learning includes: Obtain a hardness classification dataset of different objects collected by the robot, and divide the hardness classification dataset into a training set and a test set after preprocessing; The ensemble playback model is trained and tested using the training set and the test set to obtain a trained conditional haptic continuous learning ordered network model. The robot acquires tactile data of grasping a target object, inputs the tactile data into a trained conditional tactile continuous learning ordered network model, and outputs the hardness prediction result of the target object. The step of training and testing the ensemble playback model using the training set and the test set to obtain a trained conditional haptic continuous learning ordered network model specifically includes: An integrated playback model is constructed, and conditional information is added to the intermediate layer of the integrated playback model to obtain a conditional haptic continuous learning network model. The spatial features of the conditional haptic continuous learning network model are sorted to obtain a conditional haptic continuous learning ordered network model. The conditional tactile continuous learning ordered network model is trained using the training set, and the test set is input into the conditional tactile continuous learning ordered network model for testing. If the test results meet the preset requirements, the trained conditional tactile continuous learning ordered network model is obtained. The conditional tactile continuous learning ordered network model combines the feature distribution in the latent space with the order between hardness categories to sort the feature space. The loss function minimized by the conditional tactile continuous learning ordered network model for: ； in, and Refers to samples The mean and variance information of the normal distribution of the feature distribution in the latent space; This represents a hyperparameter used to adjust the distance between different class distributions in the latent space; Indicates sample Tag information.

2. The robot hardness prediction method based on tactile information and continuous learning according to claim 1, characterized in that, The process of acquiring hardness classification datasets of different objects collected by the robot, and then preprocessing these datasets into training and testing sets, specifically includes: The robot acquires tactile array data and torque data of multiple objects through capacitive tactile array sensors and torque sensors. The tactile array data and the torque data are preprocessed to obtain a hardness classification dataset. The preprocessing includes downsampling and size transformation. The hardness classification dataset is then divided proportionally to obtain a training set and a test set.

3. The robot hardness prediction method based on tactile information and continuous learning according to claim 1, characterized in that, The integrated playback model includes a discriminator and a generator; The loss function minimized by the integrated playback model for: ； in, This represents the loss function of the discriminator; Represents the loss function of the generator; This represents a hyperparameter used to balance the weights between the discriminative loss and the generative loss.

4. The robot hardness prediction method based on tactile information and continuous learning according to claim 3, characterized in that, The conditional information is used to control the type of samples generated by the integrated playback model and to unify the class features output by the encoder into a standard normal distribution.

5. The robot hardness prediction method based on tactile information and continuous learning according to claim 1, characterized in that, The process of acquiring tactile data of the robot grasping the target object, inputting the tactile data into a trained conditional tactile continuous learning ordered network model, and outputting a hardness prediction result of the target object specifically includes: The robot acquires target tactile array data and target torque data of the target object through capacitive tactile array sensors and torque sensors. The target tactile array data and the target torque data are preprocessed to obtain target tactile data. The target tactile data is then input into a trained conditional tactile continuous learning ordered network model to predict the hardness of the target object, and the predicted hardness of the target object is output.

6. A robot hardness prediction system based on tactile information and continuous learning, characterized in that, The robot hardness prediction system based on tactile information and continuous learning is used to implement the robot hardness prediction method based on tactile information and continuous learning as described in any one of claims 1-5, wherein the robot hardness prediction system based on tactile information and continuous learning comprises: The data acquisition module is used to acquire hardness classification datasets of different objects collected by the robot, and to preprocess the hardness classification datasets into training sets and test sets. The model training module is used to train and test the ensemble playback model using the training set and the test set to obtain a trained conditional haptic continuous learning ordered network model. The hardness prediction module is used to acquire tactile data of the robot grasping the target object, input the tactile data into a trained conditional tactile continuous learning ordered network model, and output the hardness prediction result of the target object.

7. A terminal, characterized in that, The terminal includes: a memory, a processor, and a robot hardness prediction program based on tactile information and continuous learning stored in the memory and executable on the processor. When the robot hardness prediction program based on tactile information and continuous learning is executed by the processor, it implements the steps of the robot hardness prediction method based on tactile information and continuous learning as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a robot hardness prediction program based on tactile information and continuous learning, which, when executed by a processor, implements the steps of the robot hardness prediction method based on tactile information and continuous learning as described in any one of claims 1-5.

Images