GRU-based social robot hand motion control method and system

Abstract

The invention provides a GRU-based social robot hand motion control method and system in the technical field of social robots. The method comprises the steps of S1, collecting a large amount of historical motion interaction data to construct a data set; s2, creating a control instruction generation model, and setting a loss function of the control instruction generation model; s3, training the control instruction generation model through the data set and the loss function, and deploying the trained control instruction generation model to the social robot; s4, the social robot collects environment sensing data in real time, the environment sensing data is input into the deployed control instruction generation model, and a hand motion control instruction is obtained; and S5, the social robot converts the hand motion control instruction into a motor parameter through a nonlinear normalization algorithm, and controls the hand to move based on the motor parameter. The method has the advantages that the precision, the adaptability and the robustness of the hand motion control of the social robot are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of social robots, and in particular to a GRU-based social robot hand motion control method and system. Background Art

[0002] As an important carrier of human-computer interaction technology, social robots realize social functions such as companionship and care, life services, and educational assistance through multimodal interaction methods such as touch, vision, and motion feedback. During the interaction process, the hand is the most core anthropomorphic motion component of the social robot. Its motion control accuracy directly affects the naturalness and credibility of interactive behaviors such as emotional expression and object manipulation. Currently, social robot hand motion control mainly uses solutions based on traditional control theory, which exposes technical bottlenecks in three dimensions in engineering practice:

[0003] 1. Low control precision: It is difficult to accurately capture and predict the timing characteristics of human gestures, resulting in a discrepancy between hand movement control commands and actual needs. 2. Poor adaptability: Relying on precise mathematical models and preset parameters, it struggles to adapt to dynamically changing environments and user behaviors, leading to poor performance in complex scenarios. 3. Lack of robustness: Sensitivity to noisy data and abnormal inputs can easily lead to erroneous control commands, impacting the stability and reliability of social robots.

[0004] Therefore, how to provide a GRU-based social robot hand motion control method and system to improve the accuracy, adaptability and robustness of social robot hand motion control has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a GRU-based social robot hand motion control method and system to improve the accuracy, adaptability and robustness of the social robot hand motion control.

[0006] In a first aspect, the present invention provides a GRU-based social robot hand motion control method, comprising the following steps:

[0007] Step S1: collecting a large amount of historical motion interaction data, and constructing a data set after pre-processing each of the historical motion interaction data;

[0008] Step S2: creating a control instruction generation model based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and setting a loss function for the control instruction generation model;

[0009] The input layer, the first GRU layer, the Dropout layer, the second fully connected layer, and the output layer are connected in sequence; the input layer, the second GRU layer, the first fully connected layer, and the second fully connected layer are connected in sequence;

[0010] Step S3: training a control instruction generation model using the data set and the loss function, and deploying the trained control instruction generation model to the social robot;

[0011] Step S4: The social robot collects environmental perception data in real time, inputs the environmental perception data into the deployed control instruction generation model, and obtains hand movement control instructions;

[0012] Step S5: The social robot converts the hand motion control instruction into motor parameters through a nonlinear normalization algorithm, and controls the hand to move based on the motor parameters;

[0013] Step S6: The social robot collects status information of the robot in real time and dynamically adjusts the working mode based on the status information;

[0014] Step S7: The social robot records the motion control log in real time, and iteratively optimizes the control instruction generation model based on the motion control log.

[0015] Furthermore, the step S1 is specifically as follows:

[0016] A large amount of historical motion interaction data including robot hand movements, human interaction audio and human gesture images are collected, and each of the historical motion interaction data is preprocessed with data cleaning, noise reduction and feature extraction. After preprocessing, each of the historical motion interaction data is subjected to data enhancement operations including at least time domain enhancement, spatial enhancement, inverse kinematic transformation and noise injection. After verifying the data enhancement, each of the historical motion interaction data is constructed.

[0017] Furthermore, in step S2, the input layer is used to convert the input human interaction audio or human gesture image into a feature vector; the first GRU layer is used to extract a first temporal dependency feature from the feature vector; the Dropout layer is used to regularize the output of the first GRU layer; the second GRU layer is used to extract a second temporal dependency feature from the feature vector; the first fully connected layer is used to perform a nonlinear transformation on the second temporal dependency feature to extract a high-level abstract feature; the second fully connected layer is used to fuse the outputs of the Dropout layer and the first fully connected layer to obtain a comprehensive representation; the output layer is used to output a hand motion control instruction based on the comprehensive representation;

[0018] The loss function is constructed based on mean square error loss and adversarial loss weighting.

[0019] Furthermore, the step S3 is specifically as follows:

[0020] The data set is divided into a training set, a validation set and a test set based on a preset ratio, and the control instruction generation model is trained by the training set until the loss value of the loss function is less than the preset loss threshold; the trained control instruction generation model is verified by calculating the gradient norm by the validation set, and the verified control instruction generation model is tested by calculating the instruction accuracy by the test set. The training is terminated if the test passes, and the trained control instruction generation model is deployed to the social robot.

[0021] Furthermore, in step S4, the environmental perception data is human interaction audio or human gesture image;

[0022] In step S5, the nonlinear normalization algorithm adopts the posMatch algorithm;

[0023] In step S6, the status information includes at least motor temperature, robot position, and robot load.

[0024] In a second aspect, the present invention provides a GRU-based social robot hand motion control system, comprising the following modules:

[0025] A data set construction module is used to collect a large amount of historical motion interaction data, and construct a data set after pre-processing each of the historical motion interaction data;

[0026] A control instruction generation model creation module is used to create a control instruction generation model based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and set a loss function for the control instruction generation model;

[0027] The input layer, the first GRU layer, the Dropout layer, the second fully connected layer, and the output layer are connected in sequence; the input layer, the second GRU layer, the first fully connected layer, and the second fully connected layer are connected in sequence;

[0028] A control instruction generation model training module is used to train the control instruction generation model using the data set and the loss function, and deploy the trained control instruction generation model to the social robot;

[0029] A hand motion control instruction generation module is used for the social robot to collect environmental perception data in real time, input the environmental perception data into the deployed control instruction generation model, and obtain hand motion control instructions;

[0030] A hand motion control module, configured for the social robot to convert the hand motion control instructions into motor parameters through a nonlinear normalization algorithm, and to control the hand to move based on the motor parameters;

[0031] A status information monitoring module is used for the social robot to collect status information of the robot in real time and dynamically adjust the working mode based on the status information;

[0032] The control instruction generation model optimization module is used for the social robot to record the motion control log in real time and iteratively optimize the control instruction generation model based on the motion control log.

[0033] Furthermore, the dataset construction module is specifically used to:

[0034] A large amount of historical motion interaction data including robot hand movements, human interaction audio and human gesture images are collected, and each of the historical motion interaction data is preprocessed with data cleaning, noise reduction and feature extraction. After preprocessing, each of the historical motion interaction data is subjected to data enhancement operations including at least time domain enhancement, spatial enhancement, inverse kinematic transformation and noise injection. After verifying the data enhancement, each of the historical motion interaction data is constructed.

[0035] Furthermore, in the control instruction generation model creation module, the input layer is used to convert the input human interaction audio or human gesture image into a feature vector; the first GRU layer is used to extract the first temporal dependency feature from the feature vector; the Dropout layer is used to regularize the output of the first GRU layer; the second GRU layer is used to extract the second temporal dependency feature from the feature vector; the first fully connected layer is used to perform a nonlinear transformation on the second temporal dependency feature to extract high-level abstract features; the second fully connected layer is used to fuse the outputs of the Dropout layer and the first fully connected layer to obtain a comprehensive representation; the output layer is used to output the hand motion control instruction based on the comprehensive representation;

[0036] The loss function is constructed based on mean square error loss and adversarial loss weighting.

[0037] Furthermore, the control instruction generation model training module is specifically used to:

[0038] The data set is divided into a training set, a validation set and a test set based on a preset ratio, and the control instruction generation model is trained by the training set until the loss value of the loss function is less than the preset loss threshold; the trained control instruction generation model is verified by calculating the gradient norm by the validation set, and the verified control instruction generation model is tested by calculating the instruction accuracy by the test set. The training is terminated if the test passes, and the trained control instruction generation model is deployed to the social robot.

[0039] Furthermore, in the hand motion control instruction generation module, the environmental perception data is human interaction audio or human gesture image;

[0040] In the hand motion control module, the nonlinear normalization algorithm adopts the posMatch algorithm;

[0041] In the state information monitoring module, the state information includes at least motor temperature, robot position and robot load.

[0042] The advantages of the present invention are:

[0043] 1. By collecting a large amount of historical motion interaction data, each of the historical motion interaction data is pre-processed to construct a data set; then, a control instruction generation model is created based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and the loss function of the control instruction generation model is set. The control instruction generation model is trained through the data set and the loss function, and the trained control instruction generation model is deployed to the social robot; the social robot collects environmental perception data in real time, inputs the environmental perception data into the deployed control instruction generation model to obtain hand motion control instructions, and converts the hand motion control instructions into motor parameters through a nonlinear normalization algorithm. , controlling the hand movements based on motor parameters; the social robot collects the state information of the machine in real time, dynamically adjusts the working mode based on the state information, and records the motion control log in real time, and iteratively optimizes the control instruction generation model based on the motion control log; that is, the control instruction generation model constructed based on GRU is used to generate hand motion control instructions, which can accurately capture and predict the timing characteristics of human gestures, and can adapt to dynamically changing environments and user behaviors. In addition, data enhancement operations are performed on the data set used to train the control instruction generation model, which reduces the sensitivity of the trained control instruction generation model to noise data and abnormal inputs, and ultimately greatly improves the accuracy, adaptability and robustness of the social robot's hand motion control.

[0044] 2. Through the branching structure of the first and second GRU layers, different levels of temporal dependency features (such as short-term dynamics and long-term trends) are extracted respectively, and fused through the fully connected layer to enhance the model's ability to represent complex temporal data and improve the accuracy of control instruction generation.

[0045] 3. By introducing a weighted combination of adversarial loss and mean square error loss into the loss function, it not only optimizes the instruction generation accuracy, but also improves the model's robustness to noise or abnormal inputs and prevents overfitting.

[0046] 4. The output of the first GRU layer is regularized through the Dropout layer, and the second fully connected layer fuses the features of the Dropout layer and the first fully connected layer to achieve the dual effects of feature fusion and regularization, balancing the model complexity and generalization ability.

[0047] 5. By collecting multimodal data such as human interaction audio and gesture images, and combining them with diverse data enhancement methods such as time domain enhancement (such as time stretching), spatial enhancement (such as image rotation), and inverse kinematic transformation (reverse derivation of joint parameters), the model's adaptability to real-world scenarios is significantly improved.

[0048] 6. By adding noise injection in the data enhancement stage to simulate actual environmental interference, the model is ensured to be more tolerant to sensor noise, environmental interference, etc. after deployment.

[0049] 7. A nonlinear normalization algorithm is used to convert control instructions into motor parameters. Compared with the linear method, it is more in line with the motor response characteristics, reduces mechanical delays and errors, and improves the smoothness of motion control.

[0050] 8. Dynamically adjust the working mode through motor temperature, load and other status information to prevent hardware overload damage. At the same time, optimize the motion trajectory based on the robot position information to achieve a balance between safety and efficiency.

[0051] 9. During the model verification phase, the model convergence is monitored by calculating the gradient norm to avoid gradient explosion or vanishing problems and ensure training stability. Through phased verification of the training set, validation set, and test set, the model performance is strictly screened, the risk of overfitting is reduced, and the actual effect after deployment is improved.

[0052] 10. By recording control logs in real time and feeding them back into the model optimization process, the model can continuously adapt to new scenarios and new user interaction patterns, forming a closed-loop learning system.

[0053] 11. The control instruction generation model is constructed through GRU. Compared with the traditional LSTM, it has fewer parameters and higher computational efficiency. It is suitable for real-time operation in embedded hardware and reduces the dependence on robot computing power.

[0054] 12. By supporting command generation in multiple input forms such as audio, gestures, and images, it can flexibly adapt to different interaction scenarios (such as voice control and visual guidance) and expand the scope of robot application.

[0055] 13. By independently designing the first GRU layer, the second GRU layer, the fully connected layer and other modules in the model architecture, it is possible to upgrade a single module (such as replacing a more efficient timing network) in the future without having to reconstruct the entire system, thereby improving the efficiency of technology iteration.

[0056] 14. The nonlinear mapping of the posMatch algorithm is used to adapt the response curves of different types of motors (such as stepper motors and servo motors), reducing the parameter tuning cost during hardware replacement.

[0057] 15. Feature extraction and model inference are completed locally on the robot through environmental perception data (audio, image), avoiding uploading of original data to the cloud and reducing the risk of privacy leakage.

[0058] 16. Through dual-GRU layer parallel time series modeling and adversarial loss function optimization, combined with multimodal data enhancement (audio, gesture images) and dynamic noise injection, the accuracy and environmental adaptability of hand motion control are significantly improved; the nonlinear normalization algorithm (posMatch) is used to achieve efficient mapping of motor parameters, and the working mode is dynamically adjusted through a real-time state feedback closed loop (such as motor temperature and load monitoring), taking into account both control smoothness and hardware security; at the same time, its modular design supports flexible expansion to multi-task scenarios, localized data processing and adversarial training mechanisms enhance privacy protection capabilities, and the iterative optimization of motion control logs and the compatibility of incremental learning form a closed-loop self-optimization system, ultimately achieving an anthropomorphic, low-latency social interaction experience based on low energy consumption and high robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0060] Figure 1 This is a flow chart of a GRU-based social robot hand motion control method of the present invention.

[0061] Figure 2 It is a structural diagram of a GRU-based social robot hand motion control system of the present invention. DETAILED DESCRIPTION

[0062] The technical solution in the embodiments of the present application has the following overall idea: a control instruction generation model constructed based on GRU is used to generate hand motion control instructions, which can accurately capture and predict the temporal characteristics of human gestures, and can adapt to dynamically changing environments and user behaviors. In addition, data enhancement operations are performed on the data set used to train the control instruction generation model, thereby reducing the sensitivity of the trained control instruction generation model to noise data and abnormal inputs, thereby improving the accuracy, adaptability and robustness of the social robot's hand motion control.

[0063] Please refer to Figures 1 to 2 As shown, a preferred embodiment of the social robot hand motion control method based on GRU of the present invention includes the following steps:

[0064] Step S1: collecting a large amount of historical motion interaction data, and constructing a data set after pre-processing each of the historical motion interaction data;

[0065] Step S2: creating a control instruction generation model based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and setting a loss function for the control instruction generation model;

[0066] The input layer, the first GRU layer, the Dropout layer, the second fully connected layer, and the output layer are connected in sequence; the input layer, the second GRU layer, the first fully connected layer, and the second fully connected layer are connected in sequence;

[0067] The control instruction generation model constructed by GRU has fewer parameters and higher computational efficiency than the traditional LSTM. It is suitable for real-time operation in embedded hardware and reduces the dependence on the robot's computing power.

[0068] Step S3: training a control instruction generation model using the data set and the loss function, and deploying the trained control instruction generation model to the social robot;

[0069] Step S4: The social robot collects environmental perception data in real time, inputs the environmental perception data into the deployed control instruction generation model, and obtains hand movement control instructions;

[0070] By supporting command generation in multiple input forms such as audio, gestures, and images, it can flexibly adapt to different interaction scenarios (such as voice control and visual guidance) and expand the scope of robot application.

[0071] Feature extraction and model inference are completed locally on the robot through environmental perception data (audio, image), avoiding uploading of original data to the cloud and reducing the risk of privacy leakage.

[0072] Step S5: The social robot converts the hand motion control instructions into motor parameters through a nonlinear normalization algorithm, and controls the hand movement based on the motor parameters; that is, the nonlinear normalization algorithm is used to achieve accurate conversion of user instructions to motor parameters, and the control amount of 0-100% is mapped to the actual position value of the motor. The input is the normalized position parameter position∈[0,100], and the output is a PWM control signal.

[0073] Step S6: The social robot collects status information of the robot in real time and dynamically adjusts the working mode based on the status information;

[0074] The social robot provides a 17-degree-of-freedom motion control interface, supports single-motor position control (motorMove) and multi-joint coordinated motion (fingerMove), implements a position mapping algorithm (posMatch) to convert 0-100% control variables into actual motor pulse values ​​(0-4095), integrates a preset gesture action library (such as grasping and stretching), and stores motion parameters in a JSON configuration file.

[0075] Step S7: The social robot records motion control logs in real time and iteratively optimizes the control command generation model based on the motion control logs. When the control command generation model is updated, backpropagation is performed using the loss function, with the gradient clipped to 1.0. In specific implementations, the social robot's performance can be evaluated based on multiple dimensions, such as motion accuracy, stability, and response speed.

[0076] By recording control logs in real time and feeding them back into the model optimization process, the model can continuously adapt to new scenarios and new user interaction patterns, forming a closed-loop learning system.

[0077] Through dual-GRU layer parallel timing modeling and adversarial loss function optimization, combined with multimodal data enhancement (audio, gesture images) and dynamic noise injection, the accuracy and environmental adaptability of hand motion control are significantly improved; the nonlinear normalization algorithm (posMatch) is used to achieve efficient mapping of motor parameters, and the working mode is dynamically adjusted through a real-time state feedback closed loop (such as motor temperature and load monitoring), taking into account both control smoothness and hardware security; at the same time, its modular design supports flexible expansion to multi-task scenarios, localized data processing and adversarial training mechanisms enhance privacy protection capabilities, and the iterative optimization of motion control logs and incremental learning compatibility form a closed-loop self-optimization system, ultimately achieving an anthropomorphic, low-latency social interaction experience based on low energy consumption and high robustness.

[0078] The step S1 is specifically as follows:

[0079] A large amount of historical motion interaction data including robot hand movements, human interaction audio and human gesture images are collected, and each of the historical motion interaction data is preprocessed with data cleaning, noise reduction and feature extraction. After preprocessing, each of the historical motion interaction data is subjected to data enhancement operations including at least time domain enhancement, spatial enhancement, inverse kinematic transformation and noise injection. After verifying the data enhancement, each of the historical motion interaction data is constructed.

[0080] In specific implementation, an 8-dimensional feature space can be constructed by feature extraction, including sine / cosine basis functions (including random phase offset φ~U[0,2π]), linear trend terms (slope κ~N(0,0.2)) and composite noise components (Gaussian noise σ 2=0.25+5% probability impulse noise δ=5), and introduce time-correlated noise (logistic noise cumulative sum) and nonlinear modulation (Beta distribution scaling α=2, β=5).

[0081] Joint motion modeling: A hierarchical motion generator is designed for a 12-DOF manipulator, where each joint trajectory consists of a fundamental harmonic (frequency f ~ U[3,7] Hz) and an independent phase modulation φi ~ U[0,2π], and kinematic constraints are added, such as finger motion range mapping (such as [500,3500] raw values ​​for the thumb) and amplitude modulation (γi ~ U[0.8,1.2]).

[0082] Anomaly injection mechanism: 1% probability of introducing a sudden offset of ±[20,20].

[0083] Data augmentation: including temporal enhancement (random time warping ±10% duration variation, dynamic frame sampling 15-30 frame subsequences), spatial enhancement (joint coupling perturbation Δθ~N(0,2°), inverse kinematic transformation 10% sample inversion normalization direction) and noise injection (sensor layer SNR = 15dB frequency domain mask noise, control layer proportional differential noise Kp~N(1,0.1),Kd~N(0.1,0.02)).

[0084] Dataset validation: The feature distribution was confirmed by KS test (α = 0.01), joint kinematics examination (vmax = 120° / s), three-dimensional motion trajectory plot, feature-joint response surface, and closed-loop control test in a simulated environment (MSE < 1.2% of the target range).

[0085] By collecting multimodal data such as human interaction audio and gesture images, and combining them with diverse data enhancement methods such as time domain enhancement (such as time stretching), spatial enhancement (such as image rotation), and inverse kinematic transformation (reverse derivation of joint parameters), the model's adaptability to real-world scenarios is significantly improved.

[0086] By adding noise injection in the data enhancement stage to simulate actual environmental interference, the model is ensured to be more tolerant to sensor noise, environmental interference, etc. after deployment.

[0087] In step S2, the input layer is used to convert the input human interaction audio or human gesture image into a feature vector; the first GRU layer is used to extract the first time-dependent feature from the feature vector; the Dropout layer is used to regularize the output of the first GRU layer; the second GRU layer is used to extract the second time-dependent feature from the feature vector; the first fully connected layer is used to perform a nonlinear transformation on the second time-dependent feature to extract high-level abstract features; the second fully connected layer is used to fuse the outputs of the Dropout layer and the first fully connected layer to obtain a comprehensive representation; the output layer is used to output hand movement control instructions based on the comprehensive representation; in specific implementation, a ReLU activation function is used after the first fully connected layer;

[0088] The loss function is constructed based on a weighted combination of mean squared error (MSE) and adversarial loss. MSE measures the difference between the predicted value and the true value, while adversarial loss introduces a discriminator to evaluate the authenticity of the generated hand motion trajectory.

[0089] The GRU network is an improved recurrent neural network (RNN) that is particularly suitable for processing time series data. Through its specific structure (including input layer, reset gate, update gate, candidate hidden state layer, and output layer), the interaction of each layer, and the selection of activation functions, it can effectively capture the time series features of hand motion trajectories, such as movement speed, acceleration, and direction changes.

[0090] Through the branch structure of the first GRU layer and the second GRU layer, different levels of temporal dependency features (such as short-term dynamics and long-term trends) are extracted respectively, and fused through the fully connected layer to enhance the model's ability to represent complex temporal data and improve the accuracy of control instruction generation.

[0091] The output of the first GRU layer is regularized through the Dropout layer, and the second fully connected layer fuses the features of the Dropout layer and the first fully connected layer to achieve the dual effects of feature fusion and regularization, balancing the model complexity and generalization ability.

[0092] By independently designing modules such as the first GRU layer, the second GRU layer, and the fully connected layer in the model architecture, it is possible to subsequently upgrade a single module (such as replacing it with a more efficient timing network) without having to reconstruct the entire system, thereby improving the efficiency of technology iteration.

[0093] By introducing a weighted combination of adversarial loss and mean square error loss into the loss function, not only the instruction generation accuracy is optimized, but also the model's robustness to noise or abnormal inputs is improved, preventing overfitting.

[0094] The step S3 is specifically as follows:

[0095] The data set is divided into a training set, a validation set and a test set based on a preset ratio, and the control instruction generation model is trained by the training set until the loss value of the loss function is less than the preset loss threshold; the trained control instruction generation model is verified by calculating the gradient norm by the validation set, and the verified control instruction generation model is tested by calculating the instruction accuracy by the test set. The training is terminated if the test passes, and the trained control instruction generation model is deployed to the social robot.

[0096] By monitoring model convergence by calculating the gradient norm during the model verification phase, we can avoid gradient explosion or vanishing problems and ensure training stability. Through phased verification of training sets, validation sets, and test sets, we can strictly screen model performance, reduce the risk of overfitting, and improve the actual effect after deployment.

[0097] In step S4, the environmental perception data is human interaction audio or human gesture image;

[0098] In step S5, the nonlinear normalization algorithm adopts the posMatch algorithm; the expression of the posMatch algorithm is:

[0099] P_out={

[0100] [(O_max-O_min) / (I_max-I_min)]×(P_in-I_min)+O_min,if O_max>O_min;

[0101] O_max-[O_min / (I_max-I_min)]×(P_in-I_min),otherwise

[0102] }

[0103] Among them, P_in represents the input percentage; I_max / I_min represent the input range extremes; O_max / O_min represent the motor encoder range; the algorithm supports bidirectional motion mapping and adapts to the forward and reverse motion characteristics of the finger joints.

[0104] A nonlinear normalization algorithm is used to convert control instructions into motor parameters. Compared with the linear method, it is more in line with the motor response characteristics, reduces mechanical delays and errors, and improves the smoothness of motion control.

[0105] The nonlinear mapping of the posMatch algorithm is used to adapt the response curves of different types of motors (such as stepper motors and servo motors), reducing the parameter tuning cost when replacing hardware.

[0106] In step S6, the status information includes at least motor temperature, robot position, and robot load.

[0107] The working mode is dynamically adjusted through status information such as motor temperature and load to prevent hardware overload and damage. At the same time, the motion trajectory is optimized in combination with the robot position information to achieve a balance between safety and efficiency.

[0108] A preferred embodiment of the GRU-based social robot hand motion control system of the present invention includes the following modules:

[0109] A data set construction module is used to collect a large amount of historical motion interaction data, and construct a data set after pre-processing each of the historical motion interaction data;

[0110] A control instruction generation model creation module is used to create a control instruction generation model based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and set a loss function for the control instruction generation model;

[0111] The input layer, the first GRU layer, the Dropout layer, the second fully connected layer, and the output layer are connected in sequence; the input layer, the second GRU layer, the first fully connected layer, and the second fully connected layer are connected in sequence;

[0112] The control instruction generation model constructed by GRU has fewer parameters and higher computational efficiency than the traditional LSTM. It is suitable for real-time operation in embedded hardware and reduces the dependence on the robot's computing power.

[0113] A control instruction generation model training module is used to train the control instruction generation model using the data set and the loss function, and deploy the trained control instruction generation model to the social robot;

[0114] A hand motion control instruction generation module is used for the social robot to collect environmental perception data in real time, input the environmental perception data into the deployed control instruction generation model, and obtain hand motion control instructions;

[0115] By supporting command generation in multiple input forms such as audio, gestures, and images, it can flexibly adapt to different interaction scenarios (such as voice control and visual guidance) and expand the scope of robot application.

[0116] Feature extraction and model inference are completed locally on the robot through environmental perception data (audio, image), avoiding uploading of original data to the cloud and reducing the risk of privacy leakage.

[0117] The hand motion control module is used for the social robot to convert the hand motion control instructions into motor parameters through a nonlinear normalization algorithm, and control the hand movement based on the motor parameters; that is, the nonlinear normalization algorithm is used to achieve accurate conversion of user instructions to motor parameters, and the control amount of 0-100% is mapped to the actual position value of the motor. The input is the normalized position parameter position∈[0,100], and the output is a PWM control signal.

[0118] A status information monitoring module is used for the social robot to collect status information of the robot in real time and dynamically adjust the working mode based on the status information;

[0119] The social robot provides a 17-degree-of-freedom motion control interface, supports single-motor position control (motorMove) and multi-joint coordinated motion (fingerMove), implements a position mapping algorithm (posMatch) to convert 0-100% control variables into actual motor pulse values ​​(0-4095), integrates a preset gesture action library (such as grasping and stretching), and stores motion parameters in a JSON configuration file.

[0120] The control command generation model optimization module is used by the social robot to record real-time motion control logs and iteratively optimize the control command generation model based on these motion control logs. When updating the control command generation model, the loss function is backpropagated, with the gradient clipped to 1.0. In specific implementations, the social robot's performance can be evaluated based on multiple dimensions, such as motion accuracy, stability, and response speed.

[0121] By recording control logs in real time and feeding them back into the model optimization process, the model can continuously adapt to new scenarios and new user interaction patterns, forming a closed-loop learning system.

[0122] Through dual-GRU layer parallel timing modeling and adversarial loss function optimization, combined with multimodal data enhancement (audio, gesture images) and dynamic noise injection, the accuracy and environmental adaptability of hand motion control are significantly improved; the nonlinear normalization algorithm (posMatch) is used to achieve efficient mapping of motor parameters, and the working mode is dynamically adjusted through a real-time state feedback closed loop (such as motor temperature and load monitoring), taking into account both control smoothness and hardware security; at the same time, its modular design supports flexible expansion to multi-task scenarios, localized data processing and adversarial training mechanisms enhance privacy protection capabilities, and the iterative optimization of motion control logs and incremental learning compatibility form a closed-loop self-optimization system, ultimately achieving an anthropomorphic, low-latency social interaction experience based on low energy consumption and high robustness.

[0123] The dataset construction module is specifically used for:

[0124] A large amount of historical motion interaction data including robot hand movements, human interaction audio and human gesture images are collected, and each of the historical motion interaction data is preprocessed with data cleaning, noise reduction and feature extraction. After preprocessing, each of the historical motion interaction data is subjected to data enhancement operations including at least time domain enhancement, spatial enhancement, inverse kinematic transformation and noise injection. After verifying the data enhancement, each of the historical motion interaction data is constructed.

[0125] In specific implementation, an 8-dimensional feature space can be constructed by feature extraction, including sine / cosine basis functions (including random phase offset φ~U[0,2π]), linear trend terms (slope κ~N(0,0.2)) and composite noise components (Gaussian noise σ 2 =0.25+5% probability impulse noise δ=5), and introduce time-correlated noise (logistic noise cumulative sum) and nonlinear modulation (Beta distribution scaling α=2, β=5).

[0126] Joint motion modeling: A hierarchical motion generator is designed for a 12-DOF manipulator, where each joint trajectory consists of a fundamental harmonic (frequency f ~ U[3,7] Hz) and an independent phase modulation φi ~ U[0,2π], and kinematic constraints are added, such as finger motion range mapping (such as [500,3500] raw values ​​for the thumb) and amplitude modulation (γi ~ U[0.8,1.2]).

[0127] Anomaly injection mechanism: 1% probability of introducing a sudden offset of ±[20,20].

[0128] Data augmentation: including temporal enhancement (random time warping ±10% duration variation, dynamic frame sampling 15-30 frame subsequences), spatial enhancement (joint coupling perturbation Δθ~N(0,2°), inverse kinematic transformation 10% sample inversion normalization direction) and noise injection (sensor layer SNR = 15dB frequency domain mask noise, control layer proportional differential noise Kp~N(1,0.1),Kd~N(0.1,0.02)).

[0129] Dataset validation: The feature distribution was confirmed by KS test (α = 0.01), joint kinematics examination (vmax = 120° / s), three-dimensional motion trajectory plot, feature-joint response surface, and closed-loop control test in a simulated environment (MSE < 1.2% of the target range).

[0130] By collecting multimodal data such as human interaction audio and gesture images, and combining them with diverse data enhancement methods such as time domain enhancement (such as time stretching), spatial enhancement (such as image rotation), and inverse kinematic transformation (reverse derivation of joint parameters), the model's adaptability to real-world scenarios is significantly improved.

[0131] By adding noise injection in the data enhancement stage to simulate actual environmental interference, the model is ensured to be more tolerant to sensor noise, environmental interference, etc. after deployment.

[0132] In the control instruction generation model creation module, the input layer is used to convert the input human interaction audio or human gesture image into a feature vector; the first GRU layer is used to extract the first time-dependent feature from the feature vector; the Dropout layer is used to regularize the output of the first GRU layer; the second GRU layer is used to extract the second time-dependent feature from the feature vector; the first fully connected layer is used to perform a nonlinear transformation on the second time-dependent feature to extract high-level abstract features; the second fully connected layer is used to fuse the outputs of the Dropout layer and the first fully connected layer to obtain a comprehensive representation; the output layer is used to output the hand movement control instruction based on the comprehensive representation; in specific implementation, the ReLU activation function is used after the first fully connected layer;

[0133] The loss function is constructed based on a weighted combination of mean squared error (MSE) and adversarial loss. MSE measures the difference between the predicted value and the true value, while adversarial loss introduces a discriminator to evaluate the authenticity of the generated hand motion trajectory.

[0134] The GRU network is an improved recurrent neural network (RNN) that is particularly suitable for processing time series data. Through its specific structure (including input layer, reset gate, update gate, candidate hidden state layer, and output layer), the interaction of each layer, and the selection of activation functions, it can effectively capture the time series features of hand motion trajectories, such as movement speed, acceleration, and direction changes.

[0135] Through the branch structure of the first GRU layer and the second GRU layer, different levels of temporal dependency features (such as short-term dynamics and long-term trends) are extracted respectively, and fused through the fully connected layer to enhance the model's ability to represent complex temporal data and improve the accuracy of control instruction generation.

[0136] The output of the first GRU layer is regularized through the Dropout layer, and the second fully connected layer fuses the features of the Dropout layer and the first fully connected layer to achieve the dual effects of feature fusion and regularization, balancing the model complexity and generalization ability.

[0137] By independently designing modules such as the first GRU layer, the second GRU layer, and the fully connected layer in the model architecture, it is possible to subsequently upgrade a single module (such as replacing it with a more efficient timing network) without having to reconstruct the entire system, thereby improving the efficiency of technology iteration.

[0138] By introducing a weighted combination of adversarial loss and mean square error loss into the loss function, not only the instruction generation accuracy is optimized, but also the model's robustness to noise or abnormal inputs is improved, preventing overfitting.

[0139] The control instruction generation model training module is specifically used to:

[0140] The data set is divided into a training set, a validation set and a test set based on a preset ratio, and the control instruction generation model is trained by the training set until the loss value of the loss function is less than the preset loss threshold; the trained control instruction generation model is verified by calculating the gradient norm by the validation set, and the verified control instruction generation model is tested by calculating the instruction accuracy by the test set. The training is terminated if the test passes, and the trained control instruction generation model is deployed to the social robot.

[0141] By monitoring model convergence by calculating the gradient norm during the model verification phase, we can avoid gradient explosion or vanishing problems and ensure training stability. Through phased verification of training sets, validation sets, and test sets, we can strictly screen model performance, reduce the risk of overfitting, and improve the actual effect after deployment.

[0142] In the hand motion control instruction generation module, the environmental perception data is human interaction audio or human gesture image;

[0143] In the hand motion control module, the nonlinear normalization algorithm adopts the posMatch algorithm; the expression of the posMatch algorithm is:

[0144] P_out={

[0145] [(O_max-O_min) / (I_max-I_min)]×(P_in-I_min)+O_min,if O_max>O_min;

[0146] O_max-[O_min / (I_max-I_min)]×(P_in-I_min),otherwise

[0147] }

[0148] Among them, P_in represents the input percentage; I_max / I_min represent the input range extremes; O_max / O_min represent the motor encoder range; the algorithm supports bidirectional motion mapping and adapts to the forward and reverse motion characteristics of the finger joints.

[0149] A nonlinear normalization algorithm is used to convert control instructions into motor parameters. Compared with the linear method, it is more in line with the motor response characteristics, reduces mechanical delays and errors, and improves the smoothness of motion control.

[0150] The nonlinear mapping of the posMatch algorithm is used to adapt the response curves of different types of motors (such as stepper motors and servo motors), reducing the parameter tuning cost when replacing hardware.

[0151] In the state information monitoring module, the state information includes at least motor temperature, robot position and robot load.

[0152] The working mode is dynamically adjusted through status information such as motor temperature and load to prevent hardware overload and damage. At the same time, the motion trajectory is optimized in combination with the robot position information to achieve a balance between safety and efficiency.

[0153] In summary, the advantages of the present invention are:

[0154] 1. By collecting a large amount of historical motion interaction data, each of the historical motion interaction data is pre-processed to construct a data set; then, a control instruction generation model is created based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and the loss function of the control instruction generation model is set. The control instruction generation model is trained through the data set and the loss function, and the trained control instruction generation model is deployed to the social robot; the social robot collects environmental perception data in real time, inputs the environmental perception data into the deployed control instruction generation model to obtain hand motion control instructions, and converts the hand motion control instructions into motor parameters through a nonlinear normalization algorithm. , controlling the hand movements based on motor parameters; the social robot collects the state information of the machine in real time, dynamically adjusts the working mode based on the state information, and records the motion control log in real time, and iteratively optimizes the control instruction generation model based on the motion control log; that is, the control instruction generation model constructed based on GRU is used to generate hand motion control instructions, which can accurately capture and predict the timing characteristics of human gestures, and can adapt to dynamically changing environments and user behaviors. In addition, data enhancement operations are performed on the data set used to train the control instruction generation model, which reduces the sensitivity of the trained control instruction generation model to noise data and abnormal inputs, and ultimately greatly improves the accuracy, adaptability and robustness of the social robot's hand motion control.

[0155] 2. Through the branching structure of the first and second GRU layers, different levels of temporal dependency features (such as short-term dynamics and long-term trends) are extracted respectively, and fused through the fully connected layer to enhance the model's ability to represent complex temporal data and improve the accuracy of control instruction generation.

[0156] 3. By introducing a weighted combination of adversarial loss and mean square error loss into the loss function, it not only optimizes the instruction generation accuracy, but also improves the model's robustness to noise or abnormal inputs and prevents overfitting.

[0157] 4. The output of the first GRU layer is regularized through the Dropout layer, and the second fully connected layer fuses the features of the Dropout layer and the first fully connected layer to achieve the dual effects of feature fusion and regularization, balancing the model complexity and generalization ability.

[0158] 5. By collecting multimodal data such as human interaction audio and gesture images, and combining them with diverse data enhancement methods such as time domain enhancement (such as time stretching), spatial enhancement (such as image rotation), and inverse kinematic transformation (reverse derivation of joint parameters), the model's adaptability to real-world scenarios is significantly improved.

[0159] 6. By adding noise injection in the data enhancement stage to simulate actual environmental interference, the model is ensured to be more tolerant to sensor noise, environmental interference, etc. after deployment.

[0160] 7. A nonlinear normalization algorithm is used to convert control instructions into motor parameters. Compared with the linear method, it is more in line with the motor response characteristics, reduces mechanical delays and errors, and improves the smoothness of motion control.

[0161] 8. Dynamically adjust the working mode through motor temperature, load and other status information to prevent hardware overload damage. At the same time, optimize the motion trajectory based on the robot position information to achieve a balance between safety and efficiency.

[0162] 9. During the model verification phase, the model convergence is monitored by calculating the gradient norm to avoid gradient explosion or vanishing problems and ensure training stability. Through phased verification of the training set, validation set, and test set, the model performance is strictly screened, the risk of overfitting is reduced, and the actual effect after deployment is improved.

[0163] 10. By recording control logs in real time and feeding them back into the model optimization process, the model can continuously adapt to new scenarios and new user interaction patterns, forming a closed-loop learning system.

[0164] 11. The control instruction generation model is constructed through GRU. Compared with the traditional LSTM, it has fewer parameters and higher computational efficiency. It is suitable for real-time operation in embedded hardware and reduces the dependence on robot computing power.

[0165] 12. By supporting command generation in multiple input forms such as audio, gestures, and images, it can flexibly adapt to different interaction scenarios (such as voice control and visual guidance) and expand the scope of robot application.

[0166] 13. By independently designing the first GRU layer, the second GRU layer, the fully connected layer and other modules in the model architecture, it is possible to upgrade a single module (such as replacing a more efficient timing network) in the future without having to reconstruct the entire system, thereby improving the efficiency of technology iteration.

[0167] 14. The nonlinear mapping of the posMatch algorithm is used to adapt the response curves of different types of motors (such as stepper motors and servo motors), reducing the parameter tuning cost during hardware replacement.

[0168] 15. Feature extraction and model inference are completed locally on the robot through environmental perception data (audio, image), avoiding uploading of original data to the cloud and reducing the risk of privacy leakage.

[0169] 16. Through dual-GRU layer parallel time series modeling and adversarial loss function optimization, combined with multimodal data enhancement (audio, gesture images) and dynamic noise injection, the accuracy and environmental adaptability of hand motion control are significantly improved; the nonlinear normalization algorithm (posMatch) is used to achieve efficient mapping of motor parameters, and the working mode is dynamically adjusted through a real-time state feedback closed loop (such as motor temperature and load monitoring), taking into account both control smoothness and hardware security; at the same time, its modular design supports flexible expansion to multi-task scenarios, localized data processing and adversarial training mechanisms enhance privacy protection capabilities, and the iterative optimization of motion control logs and the compatibility of incremental learning form a closed-loop self-optimization system, ultimately achieving an anthropomorphic, low-latency social interaction experience based on low energy consumption and high robustness.

[0170] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A GRU-based social robot hand motion control method, characterized by: The steps include: Step S1: collecting a large amount of historical motion interaction data, and constructing a data set after pre-processing each of the historical motion interaction data; Step S2: creating a control instruction generation model based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and setting a loss function for the control instruction generation model; The input layer, the first GRU layer, the Dropout layer, the second fully connected layer, and the output layer are connected in sequence; the input layer, the second GRU layer, the first fully connected layer, and the second fully connected layer are connected in sequence; Step S3: training a control instruction generation model using the data set and the loss function, and deploying the trained control instruction generation model to the social robot; Step S4: The social robot collects environmental perception data in real time, inputs the environmental perception data into the deployed control instruction generation model, and obtains hand movement control instructions; Step S5: The social robot converts the hand motion control instruction into motor parameters through a nonlinear normalization algorithm, and controls the hand to move based on the motor parameters; Step S6: The social robot collects status information of the robot in real time and dynamically adjusts the working mode based on the status information; Step S7: The social robot records the motion control log in real time, and iteratively optimizes the control instruction generation model based on the motion control log.

2. The GRU-based social robot hand motion control method according to claim 1, characterized in that: The step S1 is specifically as follows: A large amount of historical motion interaction data including robot hand movements, human interaction audio and human gesture images are collected, and each of the historical motion interaction data is preprocessed with data cleaning, noise reduction and feature extraction. After preprocessing, each of the historical motion interaction data is subjected to data enhancement operations including at least time domain enhancement, spatial enhancement, inverse kinematic transformation and noise injection. After verifying the data enhancement, each of the historical motion interaction data is constructed.

3. The GRU-based social robot hand motion control method according to claim 1, characterized in that: In step S2, the input layer is used to convert the input human interaction audio or human gesture image into a feature vector; the first GRU layer is used to extract a first time-dependent feature from the feature vector; the Dropout layer is used to regularize the output of the first GRU layer; the second GRU layer is used to extract a second time-dependent feature from the feature vector; the first fully connected layer is used to perform a nonlinear transformation on the second time-dependent feature to extract a high-level abstract feature; the second fully connected layer is used to fuse the outputs of the Dropout layer and the first fully connected layer to obtain a comprehensive representation; and the output layer is used to output a hand motion control instruction based on the comprehensive representation; The loss function is constructed based on mean square error loss and adversarial loss weighting.

4. The GRU-based social robot hand motion control method according to claim 1, characterized in that: The step S3 is specifically as follows: The data set is divided into a training set, a validation set and a test set based on a preset ratio, and the control instruction generation model is trained by the training set until the loss value of the loss function is less than the preset loss threshold; the trained control instruction generation model is verified by calculating the gradient norm by the validation set, and the verified control instruction generation model is tested by calculating the instruction accuracy by the test set. The training is terminated if the test passes, and the trained control instruction generation model is deployed to the social robot.

5. The GRU-based social robot hand motion control method according to claim 1, characterized in that: In step S4, the environmental perception data is human interaction audio or human gesture image; In step S5, the nonlinear normalization algorithm adopts the posMatch algorithm; In step S6, the status information includes at least motor temperature, robot position, and robot load.

6. A GRU-based social robot hand motion control system, characterized by: Includes the following modules: A data set construction module is used to collect a large amount of historical motion interaction data, and construct a data set after pre-processing each of the historical motion interaction data; A control instruction generation model creation module is used to create a control instruction generation model based on the input layer, the first GRU layer, the second GRU layer, the first fully connected layer, the second fully connected layer, the Dropout layer, and the output layer, and set a loss function for the control instruction generation model; The input layer, the first GRU layer, the Dropout layer, the second fully connected layer, and the output layer are connected in sequence; the input layer, the second GRU layer, the first fully connected layer, and the second fully connected layer are connected in sequence; A control instruction generation model training module is used to train the control instruction generation model using the data set and the loss function, and deploy the trained control instruction generation model to the social robot; A hand motion control instruction generation module is used for the social robot to collect environmental perception data in real time, input the environmental perception data into the deployed control instruction generation model, and obtain hand motion control instructions; A hand motion control module, configured for the social robot to convert the hand motion control instructions into motor parameters through a nonlinear normalization algorithm, and to control the hand to move based on the motor parameters; A status information monitoring module is used for the social robot to collect status information of the robot in real time and dynamically adjust the working mode based on the status information; The control instruction generation model optimization module is used for the social robot to record the motion control log in real time and iteratively optimize the control instruction generation model based on the motion control log.

7. The GRU-based social robot hand motion control system according to claim 6, characterized in that: The dataset construction module is specifically used for: A large amount of historical motion interaction data including robot hand movements, human interaction audio and human gesture images are collected, and each of the historical motion interaction data is preprocessed with data cleaning, noise reduction and feature extraction. After preprocessing, each of the historical motion interaction data is subjected to data enhancement operations including at least time domain enhancement, spatial enhancement, inverse kinematic transformation and noise injection. After verifying the data enhancement, each of the historical motion interaction data is constructed.

8. The GRU-based social robot hand motion control system according to claim 6, characterized in that: In the control instruction generation model creation module, the input layer is used to convert the input human interaction audio or human gesture image into a feature vector; the first GRU layer is used to extract the first time-dependent feature from the feature vector; the Dropout layer is used to regularize the output of the first GRU layer; the second GRU layer is used to extract the second time-dependent feature from the feature vector; the first fully connected layer is used to perform a nonlinear transformation on the second time-dependent feature to extract high-level abstract features; the second fully connected layer is used to fuse the outputs of the Dropout layer and the first fully connected layer to obtain a comprehensive representation; the output layer is used to output the hand movement control instruction based on the comprehensive representation; The loss function is constructed based on mean square error loss and adversarial loss weighting.

9. The GRU-based social robot hand motion control system according to claim 6, characterized in that: The control instruction generation model training module is specifically used to: The data set is divided into a training set, a validation set and a test set based on a preset ratio, and the control instruction generation model is trained by the training set until the loss value of the loss function is less than the preset loss threshold; the trained control instruction generation model is verified by calculating the gradient norm by the validation set, and the verified control instruction generation model is tested by calculating the instruction accuracy by the test set. The training is terminated if the test passes, and the trained control instruction generation model is deployed to the social robot.

10. The GRU-based social robot hand motion control system according to claim 6, characterized in that: In the hand motion control instruction generation module, the environmental perception data is human interaction audio or human gesture image; In the hand motion control module, the nonlinear normalization algorithm adopts the posMatch algorithm; In the state information monitoring module, the state information includes at least motor temperature, robot position and robot load.

Images