An adaptive learning method and system for a body-aware robot

Abstract

The present application relates to the technical field of embodied intelligent robot, and discloses a self-adaptive learning method and system of embodied intelligent robot, wherein the method comprises: acquiring multi-modal sensor data stream; calculating real-time reliability weight of each sensor channel and marking suspected fault state; identifying human collaborator action intention based on reliable human body perception sensor data; estimating multi-dimensional task progress based on reliability weighted environment data; separating human contribution component and robot contribution component from multi-dimensional progress change quantity; generating reshaped reward signal based on reliability weighted contribution fusion; applying decay to the reshaped reward signal during suspected fault period and performing backtracking correction after recovery; adjusting strategy network parameters based on the reward sequence after backtracking correction and generating action instruction.

Description

Technical Field

[0001] This invention relates to the field of embodied intelligent robot technology, and more specifically, to an adaptive learning method and system for embodied intelligent robots. Background Technology

[0002] In unstructured environments where embodied intelligent robots collaborate with humans to perform tasks such as assembly and care, the perception module is equipped with multimodal sensors to perceive the environmental state and the behavior of human collaborators in real time. The policy network in the decision-making module generates robot action instructions based on the perception data. The reward reshaping mechanism evaluates the value of robot actions based on changes in task progress and generates reward signals. The meta-learning algorithm dynamically adjusts the policy network parameters based on the reward signals to adapt to task changes.

[0003] In existing technologies, reward reshaping mechanisms evaluate the value of robot actions based on changes in task progress and generate reward signals, while meta-learning algorithms dynamically adjust strategy network parameters based on these reward signals.

[0004] However, in complex, unstructured environments, multimodal sensors may experience intermittent failures. Examples include sudden changes in lighting causing brief malfunctions of RGB-D cameras, electromagnetic interference triggering signal jumps, and vibration causing sudden drift in the inertial measurement unit. When sensors used to perceive the behavior of human collaborators intermittently fail, the reward reshaping mechanism receives erroneous human behavior data. This leads to either the human collaborator's contribution to task progress being incorrectly attributed to the robot's actions, resulting in inflated rewards, or the robot's effective contributions being incorrectly attributed to humans, resulting in undervalued rewards. The meta-learning algorithm adjusts the policy network parameters based on the distorted reward signal, causing the robot to reinforce ineffective policies or suppress effective policies, severely reducing the quality of adaptive learning and generalization ability. Summary of the Invention

[0005] This invention provides an adaptive learning method and system for embodied intelligent robots, solving the technical problems in related technologies such as intermittent sensor failures leading to human-robot collaborative robots being unable to accurately recognize human intentions, difficulty in correctly separating human and robot contributions to tasks, low strategy learning efficiency, and decreased collaborative security and task completion quality.

[0006] This invention discloses an adaptive learning method for an embodied intelligent robot, comprising the following steps: acquiring environmental state observation data streams from multimodal sensors during task execution, wherein the multimodal sensors include sensor channels for environmental perception and sensor channels for human perception;

[0007] Based on the signal rationality score and mutual consistency score of each sensor channel, the real-time reliability weight of each sensor channel is calculated, and sensor channels with reliability weights lower than a preset threshold are marked as suspected fault states.

[0008] When the confidence weight of the human perception sensor channel is higher than the preset threshold, the hand movement sequence of the human collaborator is input into the intention recognition network, and the intention category of the current human action and its expected impact vector on each sub-task dimension are output.

[0009] The confidence-weighted environmental state observation data is input into the task progress estimation network to obtain a multi-dimensional progress vector and calculate the progress change between adjacent time points.

[0010] Based on the expected influence vector, the human contribution component and the robot contribution component are separated from the progress change; when the human perception sensor channel is in a suspected fault state, the probability distribution of human behavior patterns is inferred based on the intention history buffer and the hidden Markov model to generate a conservatively estimated robot contribution component.

[0011] Based on the human perception credibility weight, interpolation and fusion are performed between the precise contribution separation result and the conservatively estimated robot contribution component to generate a reshaped reward signal.

[0012] An attenuation factor is applied to the remodeling reward signal during suspected faults, and a backtracking correction is performed based on complete and reliable data after the sensor returns to normal.

[0013] Based on the backtracked and corrected reward signal sequence, the policy network parameters are adjusted through a meta-learning algorithm to generate robot action commands.

[0014] Furthermore, the calculation of the real-time reliability weight for each sensor channel based on the signal reasonableness score and mutual consistency score of each sensor channel includes:

[0015] For the signal value output by each sensor channel at the current moment, calculate the signal change rate of adjacent moments, subtract the absolute value of the signal change rate from one to the ratio of the physical limit change rate of the sensor channel, and take the maximum value of zero to obtain the signal rationality score; wherein, the physical limit change rate is the maximum reasonable change amplitude of the sensor within a single sampling period under normal operating conditions and constrained by the dynamic characteristics of the measured physical quantity.

[0016] The data from each sensor channel is input into a Kalman filter to obtain a fusion state estimate. The observation residual between the actual observation value of each sensor channel and the expected observation value obtained by mapping the fusion state estimate through the observation mapping matrix is ​​calculated. Based on the observation residual and the preset standard deviation parameter, a mutual consistency score is generated by a Gaussian kernel function.

[0017] The signal rationality score and the mutual consistency score are weighted and summed according to the fusion coefficient to obtain the real-time reliability weight of each sensor channel.

[0018] Furthermore, it also includes:

[0019] Apply time window smoothing to the real-time credibility weights, and take the moving average of the real-time credibility weights at the most recent preset window length times to obtain the smoothed credibility weights.

[0020] Sensor channels whose smoothed confidence weight is lower than the preset threshold are marked as suspected fault states; in subsequent steps, the smoothed confidence weight is used to replace the real-time confidence weight for judgment and calculation.

[0021] Furthermore, the intent recognition network is a neural network based on temporal convolution and attention mechanisms, and its processing flow includes:

[0022] The hand motion sequence is input into the temporal convolutional layer, and local temporal pattern features are extracted by sliding a one-dimensional convolutional kernel along the time dimension.

[0023] The importance weights of features at each time step are calculated through an attention layer and then weighted and aggregated.

[0024] The classification branch uses a fully connected layer and a softmax activation function to output the intent category, while the regression branch uses a fully connected layer to output the expected impact vector for each subtask dimension under the corresponding intent category.

[0025] Each time the intent recognition network outputs a valid result, the intent category, the expected impact vector, and the corresponding timestamp are stored in the intent history buffer to maintain the most recent preset number of valid records.

[0026] Furthermore, the step of inputting the confidence-weighted environmental state observation data into the task progress estimation network to obtain a multi-dimensional progress vector includes:

[0027] The observation data feature vectors output by each environmental perception sensor channel are multiplied by the corresponding confidence weight of the sensor channel to obtain weighted features. The weighted features of all sensor channels are concatenated or summed to form a weighted environmental state feature vector.

[0028] The weighted environmental state feature vector is input into the task progress estimation network, and the current completion progress value of each sub-task dimension is output to form the multi-dimensional progress vector.

[0029] The difference between the multi-dimensional progress vector at the current moment and the multi-dimensional progress vector at the previous moment is calculated to obtain the progress change.

[0030] Furthermore, the separation of human contribution components and robot contribution components from the progress change based on the expected influence vector includes:

[0031] When the confidence weight of the human perception sensor channel is higher than the preset threshold, the minimum value of the expected influence vector and the progress change in each dimension is taken as the human contribution component, and the difference between the progress change and the human contribution component is taken as the robot contribution component.

[0032] When the confidence weight of the human perception sensor channel is lower than the preset threshold, the intention category sequence in the intention history buffer is input into the Hidden Markov Model, and the posterior probability distribution of each intention category at the current time is output. Based on the posterior probability distribution, the expected impact vector corresponding to each intention category is expected to be calculated to obtain a conservatively estimated human expected impact vector. After applying a conservative amplification factor greater than one to the conservatively estimated human expected impact vector, the conservatively estimated human contribution component and the conservatively estimated robot contribution component are calculated in the same way as the precise contribution separation.

[0033] Furthermore, the generation of the reshaping reward signal includes:

[0034] Based on the human perception credibility weight, the precise contribution separation result and the conservatively estimated robot contribution component are linearly interpolated to obtain the credibility-weighted robot contribution component.

[0035] The transpose of the importance weight vector of each subtask dimension is multiplied by the credibility-weighted robot contribution component to obtain a positive instant reward signal.

[0036] An exploration incentive signal is generated based on the difference between the current progress value and the target progress value of each sub-task dimension. Specifically, when the current human intention category belongs to a predefined set of active operation categories, the exploration incentive component of the corresponding sub-task dimension is set to zero; when the probability of the active operation intention inferred during the fault exceeds a preset probability threshold, the exploration incentive component of the corresponding sub-task dimension is set to zero.

[0037] The positive immediate reward signal is added to the exploration incentive signal weighted by a tradeoff coefficient to output the reshaping reward signal.

[0038] Furthermore, the step of performing backtracking correction based on complete and reliable data after the sensor returns to normal includes:

[0039] The difference between the multi-dimensional progress vector at the fault recovery time and the multi-dimensional progress vector at the fault initiation time is obtained to obtain the cumulative progress change during the suspected fault period.

[0040] Based on the recovered reliable human perception sensor data, the human action intentions during the suspected malfunction are re-identified, the corrected human cumulative contribution component is obtained, and the corrected robot cumulative contribution component is calculated.

[0041] The transpose of the importance weight vector of each subtask dimension is multiplied by the corrected cumulative contribution component of the robot to generate the backtracking correction reward.

[0042] Calculate the proportion of the original reward at each time point during the suspected fault period to the total original reward during the fault period, and redistribute the backtracking correction reward to each time step according to this proportion to obtain the correction reward at each time point.

[0043] When the duration of a suspected fault exceeds a preset maximum duration threshold, only the rewards within half of the preset maximum duration threshold before and after the start and end of the suspected fault are retrospectively corrected, while the attenuated reward signal in the middle period remains unchanged.

[0044] Furthermore, the adjustment of policy network parameters through the meta-learning algorithm includes:

[0045] The meta-learning algorithm adopts the MAML algorithm, which includes an inner loop stage and an outer loop stage. The inner loop stage performs policy gradient updates on each task variant to obtain task-specific policy network parameters. The outer loop stage calculates the meta-gradient of the initial parameters of the policy network based on the performance of the inner loop update on each task variant and updates the initial parameters of the policy network.

[0046] Based on the backtracked and corrected reward signal sequence and the corresponding environment state-action trajectory, the gradient update amount of the policy network parameters is calculated. The gradient update amount is the expected value of the product of the corrected reward and the gradient of the log probability of the policy network outputting the corresponding action under the corresponding observation on the trajectory.

[0047] A progressive integration mechanism is applied to the difference in rewards before and after the backtracking correction. The progressive integration rounds are set, and in each round of parameter updates, the difference in rewards is gradually added to the reward sequence according to the ratio of the current round number to the progressive integration round number.

[0048] This invention provides an adaptive learning system for an embodied intelligent robot, comprising:

[0049] The multimodal data acquisition module is used to acquire environmental state observation data streams from multimodal sensors during task execution.

[0050] The credibility assessment and fault marking module is used to calculate the real-time credibility weight based on the signal rationality score and mutual consistency score of each sensor channel, and mark the sensor channel with the credibility weight below the preset threshold as a suspected fault state.

[0051] The intent recognition module is used to output the intent category and expected impact vector of the current human action based on the intent recognition network when the confidence weight of the human perception sensor channel is higher than the preset threshold.

[0052] The task progress estimation module is used to obtain a multi-dimensional progress vector and calculate the progress change based on the confidence-weighted environmental state observation data.

[0053] The contribution separation module is used to separate the human contribution component and the robot contribution component from the progress change based on the expected influence vector, and to generate a conservative estimate of the robot contribution component based on the intention history buffer and the hidden Markov model when the human perception sensor channel is in a suspected fault state.

[0054] The reward generation module is used to interpolate and fuse the precise contribution separation result and the conservatively estimated robot contribution component based on the human perception credibility weight, and generate a reshaped reward signal.

[0055] The backtracking correction module is used to apply an attenuation factor to the reshaping reward signal during suspected faults and perform backtracking correction based on complete and reliable data after the sensor returns to normal.

[0056] The strategy update and execution module is used to adjust the policy network parameters and generate robot action instructions based on the backtracked and corrected reward signal sequence using a meta-learning algorithm.

[0057] This invention generates real-time reliability weights for each sensor channel by fusing a signal rationality score based on physical characteristic constraints and a mutual consistency score based on multi-source redundancy relationships. This enables the perception module to identify intermittent fault states of multimodal sensors in real time. A reliability-weighted interpolation fusion mechanism facilitates a continuous transition between the precise contribution separation result and the conservatively estimated robot contribution component, ensuring that the reward reshaping mechanism can still output a reshaped reward signal within a reasonable range during intermittent sensor faults. A backtracking correction mechanism after fault recovery recalculates the actual contribution separation result during suspected fault periods using complete reliable data. This allows the meta-learning algorithm to adjust the policy network parameters based on the accurate reward signal sequence after backtracking correction. This solves the technical problem of policy shift caused by incorrect reward signal attribution due to intermittent multimodal sensor faults, achieving the technical effect of maintaining effective adaptive policy learning under intermittent and unreliable sensor conditions. Attached Figure Description

[0058] Figure 1 This is a flowchart of the adaptive learning method for an embodied intelligent robot provided in an embodiment of the present invention;

[0059] Figure 2 This is a schematic diagram illustrating the changing trend of the confidence weight of the multimodal sensor channel provided in an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of the probability distribution of the rehabilitation therapist's intent recognition results provided in an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the multi-dimensional task progress evolution curve provided in the embodiments of the present invention;

[0062] Figure 5 This is a schematic diagram comparing the contributions of the therapist and the robot according to an embodiment of the present invention;

[0063] Figure 6 This is a schematic diagram of reward signal attenuation and backtracking correction during a suspected fault, provided in an embodiment of the present invention.

[0064] Figure 7 This is a schematic diagram of the distribution of sensor signal rationality and mutual consistency scores provided in an embodiment of the present invention;

[0065] Figure 8 This is a schematic diagram illustrating the distribution of the therapist's expected influence vector across the dimensions of each subtask, as provided in an embodiment of the present invention. Detailed Implementation

[0066] In unstructured environments where embodied intelligent robots collaborate with humans to perform tasks such as assembly and care, the perception module is typically equipped with multimodal sensors (RGB-D cameras, joint encoders, torque sensors, inertial measurement units, etc.) to perceive the environmental state and the behavior of human collaborators in real time. The policy network in the decision-making module generates robot action commands based on the perceived data, the reward reshaping mechanism evaluates the value of the robot's actions based on changes in task progress and generates reward signals, and the meta-learning algorithm dynamically adjusts the policy network parameters based on the reward signals to adapt to task changes. It should be understood that in such human-robot collaboration scenarios, task progress is simultaneously affected by both human collaborator behavior and robot actions; therefore, it is necessary to separate the contributions of each party from the overall progress change to correctly attribute the reward signals.

[0067] However, in complex, unstructured environments, multimodal sensors may experience intermittent failures, such as sudden changes in lighting causing brief RGB-D camera malfunctions, electromagnetic interference triggering signal jumps, or vibration causing sudden drift in the inertial measurement unit. These failures occur randomly and the sensors can resume operation after a brief recovery. When sensors used to perceive the behavior of human collaborators intermittently fail, the reward reshaping mechanism receives erroneous human behavior data, leading to the following problems: the progress improvements contributed by human collaborators are incorrectly attributed to robot actions, resulting in inflated rewards, or the robot's effective contributions are incorrectly attributed to humans, resulting in undervalued rewards. Meta-learning algorithms adjust policy network parameters based on distorted reward signals, causing the robot to reinforce ineffective policies or suppress effective policies, severely reducing the quality of adaptive learning and generalization ability. Furthermore, completely discarding all data during intermittent failures leads to inefficient policy learning and an inability to continuously adapt to task changes.

[0068] Therefore, the technical problem that this implementation needs to solve is: in an unstructured human-machine collaborative environment where multimodal sensors have intermittent faults, how to avoid distorted human behavior perception data leading to incorrect attribution of reward signals without completely discarding data during the fault period, thereby maintaining the effective adjustment of policy network parameters by the meta-learning algorithm.

[0069] According to an embodiment of this method, before executing this method, the hardware environment of the embodied intelligent robot includes at least: a perception module equipped with an RGB-D camera, an infrared sensor, a joint encoder, a torque sensor, and an inertial measurement unit; a decision module equipped with a policy network, a reward reshaping mechanism, and a meta-learning algorithm; and an execution module that outputs actions based on the policy network. Data from each sensor channel is synchronously collected and time-aligned at a preset sampling frequency.

[0070] At least one embodiment of the present invention discloses an adaptive learning method for embodied intelligent robots, such as... Figure 1 As shown, it includes the following steps:

[0071] Step 1: Acquire multimodal sensor data streams during task execution;

[0072] The multimodal sensors in the perception module acquire environmental state observation data streams during task execution, including robot joint angle and angular velocity data output by the joint encoder, end-effector contact force data output by the torque sensor, attitude and acceleration data output by the inertial measurement unit, and scene depth images and human infrared thermal images output by the RGB-D camera and infrared sensor. Simultaneously, the system acquires raw data streams of human collaborator limb movement trajectories and hand motion sequences output by sensors used for human perception. The limb movement trajectories are represented by a sequence of 3D coordinates of human key points captured by the RGB-D camera, and the hand motion sequences are represented by a sequence of depth image frames of the hand region.

[0073] Step 2: Calculate the real-time reliability weight of each sensor channel and mark suspected fault states;

[0074] For the data from each sensor channel, a signal rationality score is calculated based on its physical characteristics.

[0075] Specifically, for sensor channels At any moment Output signal value Calculate the rate of change of the signal at adjacent time points. And compare the rate of change of the signal with the physical limit rate of change of this type of sensor. Compare and generate a signal rationality score. :

[0076] in, Indicates sensor channel At any moment Signal rationality score, Indicates sensor channel At any moment The rate of change of the signal, Indicates sensor channel The physical limiting rate of change, This indicates the absolute value operation. This indicates the operation of finding the maximum value. The range of values ​​is When the rate of change of the signal exceeds the physical limit of change, A value of zero indicates that the signal at that moment is completely unreasonable.

[0077] Furthermore, the aforementioned physical limit rate of change This refers to the maximum reasonable rate of change of this type of sensor within a single sampling period, constrained by the dynamic characteristics of the measured physical quantity, under normal operating conditions. For example, for the acceleration channel of an inertial measurement unit, the physical limit rate of change is determined by the product of the robot's maximum acceleration and the sampling period; for the depth channel of an RGB-D camera, the physical limit rate of change is determined by the product of the maximum velocity of objects in the scene and the sampling period.

[0078] Simultaneously, data from each sensor channel is input into a Kalman filter, and a mutual consistency score is calculated based on the redundancy relationship between the sensors. The input to the Kalman filter is the observation sequence of each sensor channel, and the output is the fused state estimate. and the observation residuals of each channel .

[0079] Furthermore, the Kalman filter recursively estimates the system state through a state prediction equation and an observation update equation. The state prediction equation predicts the state at the next time step based on the system dynamics model, while the observation update equation corrects the predicted state based on the actual observations from each sensor channel, thereby obtaining a fused state estimate. Observation residuals For sensor channel The actual observations and the fusion state estimates are mapped through the observation mapping matrix. Calculate the difference between the expected observations. Calculate the sensor channels. The residual between the observed values ​​and the fusion state estimate ,in Indicates sensor channel At any moment The observed values, For sensor channel The observation mapping matrix, Indicates time The fusion state estimate is obtained. A mutual consistency score is generated based on the residuals. :

[0080] in, Indicates sensor channel At any moment The mutual consistency score, Indicates sensor channel At any moment The observation residuals For sensor channel The preset standard deviation parameter, This represents an exponential function.

[0081] Fusion signal rationality score With mutual consistency score Generate real-time reliability weights for each sensor channel. :

[0082]

[0083] in, Indicates sensor channel At any moment Real-time credibility weight, The fusion coefficient has a range of values. .

[0084] Furthermore, to avoid frequent switching of fault states caused by sudden changes in real-time reliability weights due to single signal jitter, the real-time reliability weights are adjusted accordingly. Apply time window smoothing to the most recent The real-time confidence weight at each time point is taken as the moving average as the smoothed confidence weight. :

[0085] in, Indicates sensor channel At any moment The smoothed credibility weights The length of the sliding window. For summation index and The range of values ​​is from arrive The integer. The smoothed confidence weights are used in subsequent steps. Replacement of real-time credibility weight Make judgments and calculations.

[0086] Smoothed credibility weights Below the preset threshold Sensor channels are marked as potentially faulty, with a preset threshold. The range of values ​​is .

[0087] Step 3: Identify the intentions of human collaborators based on reliable human perception sensor data;

[0088] When the smoothed confidence weight of the human perception sensor channel (including the RGB-D camera channel and infrared sensor channel used to capture the limb movement trajectory of human collaborators) is higher than a preset threshold At that time, the hand movement sequence of the human collaborator is input into the intention recognition network, and the network outputs the intention category of the current human action. and its expected impact vector on each sub-task dimension ,in The number of subtask dimensions.

[0089] The intent recognition network is a neural network based on temporal convolution and attention mechanisms, and its input is the most recent... Feature encoding of the hand action sequence in frames, output as the intent category. and expected impact vector ,in This refers to the number of frames in the hand motion sequence. The processing flow of the intent recognition network includes: inputting the hand motion sequence into a temporal convolutional layer to extract temporal features, weighting the features at key time steps through an attention layer, and outputting the intent category through a classification branch. The regression branch outputs the expected impact vector for each subtask dimension under the corresponding intent category. .

[0090] Furthermore, the temporal convolutional layer extracts local temporal pattern features of the hand action sequence by sliding a one-dimensional convolutional kernel along the temporal dimension. The attention layer calculates the importance weights of the features at each time step and performs weighted aggregation of the features, enabling the intent recognition network to focus on keyframes in the hand action sequence. The classification branch uses a fully connected layer to map the aggregated features to the probability distribution of each intent category, and outputs the intent category through a softmax activation function. The regression branch uses a fully connected layer to map the aggregated features to the expected impact values ​​of each subtask dimension, outputting an expected impact vector. .

[0091] Furthermore, the aforementioned expected impact vector Each component of the expected impact vector represents the expected progress contribution of the human's current action intention in the corresponding sub-task dimension. For example, in an assembly task, the sub-task dimension may include the part positioning accuracy, the tightness of the connection, the assembly completion rate, etc. Each component of the expected impact vector corresponds to the expected change in these dimensions.

[0092] Furthermore, in order to preserve temporal context information in the intent recognition results for subsequent inference during fault periods, the intent category is set each time the intent recognition network outputs a valid result. Expected impact vector The corresponding timestamp is stored in the intent history buffer to maintain the most recent history. Valid records, of which This represents the number of records in the intended history buffer.

[0093] Step 4: Estimate multi-dimensional task progress based on credibility-weighted environmental data;

[0094] The credibility-weighted environmental state observation data stream is input into the task progress estimation network to obtain the current multi-dimensional progress vector. Specifically, for the observation data output by each environmental perception sensor channel, its smoothed confidence weight is used. The weighted environmental state features are fused as weighting coefficients to generate weighted environmental state features. These features are then input into the task progress estimation network, which outputs the current progress value for each subtask dimension, forming a multi-dimensional progress vector. .

[0095] Furthermore, the weighted environmental state characteristics are generated as follows: for each sensor channel Output observation data feature vector Compare it with the corresponding smoothed confidence weight Multiplication yields weighted features Then, the weighted features of all sensor channels are concatenated or summed to form a fused weighted environmental state feature vector, which is used as the input to the task progress estimation network.

[0096] Calculate the progress change between the current time and the previous time. :

[0097] in, Indicates time Relative to time The change in progress, For a moment The multi-dimensional progress vector For a moment A multi-dimensional progress vector.

[0098] The task progress estimation network is a multilayer perceptron, whose input is a weighted environmental state feature vector and whose output is the progress value of each subtask dimension.

[0099] Step 5: Separate the human contribution component and the robot contribution component from the multi-dimensional progress change;

[0100] Based on the expected impact vector Progress changes from multiple dimensions The human contribution component was separated from it. And the contribution of robots When the smoothed confidence weight of the human perception sensor channel exceeds a preset threshold... At that time, perform precise contribution separation: divide the expected impact vector With multi-dimensional progress change The minimum value of each element in each dimension is taken as the upper bound estimate of the human contribution component, thus obtaining the human contribution component. :

[0101]

[0102] in, Indicates the weight of human contribution. Indicates the contribution of the robot. This is an operation that takes the minimum value for each element.

[0103] When the smoothed reliability weight of the human perception sensor channel is lower than a preset threshold At that time, based on the intent recognition results of the most recent credible state in the intent history buffer and the Hidden Markov Model, the probability distribution of human behavior patterns in the current time period is inferred. Specifically, the most recent credible state in the intent history buffer is used to infer the probability distribution of human behavior patterns in the current time period. The Hidden Markov Model takes a sequence of intent categories from 10 valid records as input and outputs the posterior probability distribution of each intent category at the current time. ,in For the most recent The timestamp corresponding to each valid record.

[0104] Furthermore, Hidden Markov Models (HMMs) describe the transition patterns between human intention categories using a state transition probability matrix and describe the distribution of observed features corresponding to each intention category using an observation probability matrix. During inference, the HMM calculates the posterior probability distribution of each intention category at the current time using either a forward algorithm or a Viterbi algorithm, based on the historical intention category sequence and the state transition probability matrix.

[0105] Based on the posterior probability distribution, the expected impact vector corresponding to each intent category is calculated to generate a conservatively estimated range of human contribution components:

[0106] in, This represents a conservative estimate of the expected human impact vector. This indicates the category of human intention at the current moment. The posterior probability, For Intent Category The corresponding expected impact vector, The total number of intent categories, For summation index and The range of values ​​is from arrive Integers. Human expected impact vector based on conservative estimates. The conservatively estimated human contribution component was calculated in the same manner as the precise contribution separation. And the conservatively estimated contribution of the robot .

[0107] Furthermore, to prevent conservative estimates from overestimating human contributions and thus avoiding inflated rewards for robots, the influence vector of human expectations on conservative estimates is... Apply conservative amplification factor ( ), which will conservatively estimate the expected human impact vector Replace with Then, contribution separation calculation is performed, which makes the conservative estimate of the robot's contribution component lower, reducing the risk of inflated rewards during failures.

[0108] Step 6: Generate a reshaped reward signal based on credibility-weighted contribution fusion;

[0109] Human perception confidence weight based on human perception sensor channels (Taking the minimum smoothed confidence weight of the human perception-related sensor channels), in accurately contributing to the separation results. Compared with the conservatively estimated robot contribution component Interpolation and fusion are performed between them to calculate the confidence-weighted robot contribution component. :

[0110] in, This represents the credibility-weighted contribution of the robot. This represents the weight of human perception credibility. Indicates the precise contribution separation results. This represents a conservative estimate of the robot's contribution. (The weighting is related to the human perception credibility.) When the confidence weight is close to 1, the robot contribution component is close to the precise contribution separation result; when the human perception confidence weight is close to 1, the robot contribution component is close to the precise contribution separation result. When the value approaches 0, the confidence-weighted robot contribution component is close to the conservatively estimated robot contribution component.

[0111] Credibility-weighted robot contribution components A positive, immediate reward signal is generated through a reward reshaping mechanism. :

[0112] in, This indicates a positive, immediate reward signal. This is a weight vector representing the importance of each subtask dimension. Represents the importance weight vector transpose, This indicates the transpose operation.

[0113] Simultaneously, generate exploration incentive signals. When a human is detected to be in an active operating state (i.e., the current intention category of the human in the intention recognition result), It belongs to the set of active operation categories, or the probability of an active operation intent inferred during a fault exceeds a preset probability threshold. (At that time), suppress the exploration incentive signal of the corresponding subtask dimension and set the exploration incentive component of the active operation dimension to zero.

[0114] Furthermore, the aforementioned set of active operation categories is a predefined subset of intent categories, containing intent categories in which a human collaborator is performing a direct operation on a certain sub-task dimension, such as the tightening operation category corresponding to a human tightening a bolt. The purpose of suppressing exploration stimulus signals for the corresponding sub-task dimension is to prevent the robot from generating redundant exploration actions in sub-task dimensions already actively undertaken by humans.

[0115] Furthermore, in order to enable the exploration stimulus signals to guide the robot to explore sub-task dimensions not covered by humans, based on the current progress value of each sub-task dimension... Generate exploration stimulus signals Specifically, exploring stimulus signals. Dimensional components Calculate as follows:

[0116] For the In the sub-task dimension, if the first If the human intent category corresponding to each sub-task dimension does not belong to the set of active operation categories, then... ,in For the first Target progress values ​​for each sub-task dimension. For the first Current progress value for each subtask dimension. For the subtask dimension index; if the first... If the human intent category corresponding to each sub-task dimension belongs to the set of active operation categories, then... .

[0117] By integrating positive immediate reward signals and exploration incentive signals, the output will reshape the reward signal. :

[0118] in, This indicates a reshaping of reward signals. This is a positive, immediate reward signal. To explore incentive signals, To explore the trade-off coefficients of incentives.

[0119] Step 7: Apply attenuation to the reshaping reward signal during the suspected fault period and perform backtracking correction after recovery; apply an attenuation factor to the reshaping reward signal accumulated during the suspected fault state. ( ), generating a decayed reward signal :

[0120] in, This represents the decayed reward signal. Indicates the attenuation factor. This indicates a reshaping of reward signals. The time interval during which the suspected fault condition persists.

[0121] When the suspected faulty sensor returns to normal and the confidence weight after smoothing rises back to the preset threshold Following this, based on the recovered complete and reliable sensor data, contribution separation and reward calculation are re-performed for the multi-dimensional progress changes during the suspected fault period. Specifically, the fault recovery time is obtained. Multidimensional progress vector and the time of fault onset Multidimensional progress vector Calculate the cumulative schedule change during the suspected failure period. ,in Indicates the time of fault recovery. Indicates the time of fault onset. This represents the cumulative progress change during the suspected failure period. Based on reliable human perception sensor data after recovery, the human's intentional actions during the suspected failure period are re-identified, and the corrected cumulative human contribution component is obtained. Calculate the corrected cumulative contribution component of the robot. Generate backtracking correction rewards :

[0122] in, This indicates a retrospective correction of the reward. This is a weight vector representing the importance of each subtask dimension. Represents the importance weight vector transpose, Accumulate contribution points for the corrected robot. Revert to previous corrected rewards. Replace the decayed reward signal accumulated during the suspected fault period The rewards are redistributed to each time step according to the original reward proportions at each moment during the suspected failure period. Specifically, for each moment during the suspected failure period... Calculate the original reward at that moment. Total original reward during suspected fault period The proportion in The rewards will be retrospectively corrected. Allocate to time according to this ratio Receive the corrected reward at that moment. ,in Indicates time The original reward ratio, Indicates time Correction rewards.

[0123] Furthermore, the re-identification of human intentions during the suspected malfunction in the aforementioned backtracking correction is achieved through joint analysis of reliable sensor data after recovery and the robot's own action records during the suspected malfunction. Since the robot's own actions during the suspected malfunction have been output by the policy network and fully recorded, and the task scenario state after malfunction recovery can be accurately obtained by reliable sensors, the actual contribution of humans can be inferred from the cumulative progress change.

[0124] Furthermore, to address the situation where the accuracy of backtracking correction decreases due to an excessively long suspected fault duration, when the duration of a suspected fault exceeds a preset maximum duration threshold... At that time, only the period before and after the suspected fault begins and ends is considered. Rewards within a given timeframe are retrospectively corrected, while the decayed reward signal in the middle period remains unchanged, preventing the corrected reward deviation from exceeding the decayed reward signal deviation due to excessive accumulated errors.

[0125] Step 8: Adjust the policy network parameters based on the backtracked and corrected reward sequence and generate action instructions;

[0126] The complete reshaping of the reward signal sequence after backtracking and correction. Input meta-learning algorithm, where During non-fault periods, the original remodeling reward signal is provided. The corrected reward is redistributed during the fault period after backtracking correction. The meta-learning algorithm calculates the gradient update of the policy network parameters based on the complete reconstructed reward signal sequence after backtracking correction and the corresponding environment state-action trajectory, dynamically adjusting the policy network parameters. :

[0127] in, Indicates the policy network parameters, The meta-learning rate, To adjust the policy network parameters gradient calculation, For the expected operation, Indicates time Correction rewards, For policy networks in observation Down Output Action The probability, For robots at all times The action, For a moment Environmental condition observation, The total number of time steps for the trajectory. For summation index and The range of values ​​is from arrive Integers.

[0128] Furthermore, the expectation operation in the above gradient update formula... It involves integrating or averaging the probability distribution of the environmental state-action trajectory. In actual calculations, the expected value is approximated by collecting multiple trajectory samples and averaging the gradients of each sample.

[0129] Log probability gradient The gradient is calculated using the backpropagation algorithm of the policy network. Indicates policy network parameters In which direction should the adjustment be made to increase the observation? Select action The probability of.

[0130] Furthermore, the policy network is a multilayer perceptron, whose input is the environmental state observation. The output is the probability distribution of each action. The policy network contains several hidden layers, each using the ReLU activation function. The output layer uses the softmax activation function to convert the policy network output into an action probability distribution.

[0131] Furthermore, the meta-learning algorithm employs the MAML algorithm, whose goal is to learn initial parameters for a policy network that can quickly adapt to new tasks. The training process of the MAML algorithm consists of two phases: an inner loop and an outer loop. The inner loop performs several policy gradient updates on each task variant to obtain task-specific policy network parameters. The outer loop calculates the initial parameters of the policy network based on the performance of the inner loop updates on each task variant. The meta-gradient is calculated, and the initial parameters of the policy network are updated so that it can achieve good performance on all task variants after a small number of inner loop updates.

[0132] The execution module is based on the updated policy network. The robot generates adaptive action commands based on observations of the current environmental state. The output of the policy network is the probability distribution of each action. The probability distribution of actions is converted into specific executable action commands by sampling or selecting the action with the highest probability. These action commands include the target angles and angular velocities of each robot joint and control commands for the end effector. The execution module drives the robot hardware to perform the corresponding actions based on these commands. Furthermore, to avoid drastic oscillations in the policy network parameters caused by the difference between the backtracking correction reward and the original decayed reward signal after fault recovery, the difference in reward before and after backtracking correction is... A gradual integration mechanism is implemented, where the reward difference is gradually added to the reward sequence over several rounds of parameter updates, rather than being replaced all at once. This allows the policy network parameters to smoothly transition to the corrected optimization direction. Specifically, the number of gradual integration rounds is set. In the The reward used in the round parameter update is ,in This indicates the difference in rewards before and after the backtracking correction. To gradually integrate the number of rounds, The current round number and This allows the difference in reward amounts to be gradually integrated.

[0133] This implementation generates real-time reliability weights for each sensor channel by fusing a signal rationality score based on physical characteristic constraints and a mutual consistency score based on multi-source redundancy relationships. This enables the sensing module to identify intermittent fault states of multimodal sensors in real time. Therefore, the contribution separation and reward calculation in subsequent steps can clearly distinguish the reliability of sensor data, avoiding the direct use of unfiltered, erroneous human behavior perception data for reward attribution.

[0134] This implementation employs a confidence-weighted interpolation fusion mechanism to continuously transition between the precise contribution separation result and the conservatively estimated robot contribution component when the human perception sensor is in different confidence states, rather than using a binary, all-or-nothing strategy. Therefore, during intermittent sensor failures, the reward reshaping mechanism can still output a reshaped reward signal within a reasonable range. This avoids both policy learning interruption due to completely discarding data from suspected failure periods and policy deviation caused by incorrect reward attribution due to directly accepting distorted data. The conservative estimation mechanism provides an approximation of the human contribution component through probabilistic inference based on the intent history buffer, making the conservatively estimated robot contribution component during suspected failure periods more conservative, thus reducing the probability of inflated rewards.

[0135] This implementation employs a backtracking correction mechanism after fault recovery. Once the sensor returns to a reliable state, it recalculates the actual contribution separation results during the suspected fault period using complete reliable data, replacing the temporary reward with a decay factor applied during the suspected fault period. Therefore, the meta-learning algorithm ultimately adjusts the policy network parameters based on the accurate reward signal sequence after backtracking correction, eliminating the cumulative bias caused by the decayed reward signal during the suspected fault period. This makes the parameter update direction of the policy network approach the optimization direction under the condition of complete sensor reliability.

[0136] In summary, the synergistic cooperation of credibility assessment, conservative estimation interpolation fusion, and backtracking correction in this implementation method enables the embodied intelligent robot to maintain effective adaptive strategy learning and generalization adaptability to task changes in an unstructured human-machine collaborative environment where multimodal sensors are intermittently unreliable.

[0137] In assisted rehabilitation training scenarios at medical rehabilitation institutions, embodied intelligent robots assist rehabilitation therapists in training patients' upper limb function recovery. The robot is equipped with an RGB-D camera, infrared sensors, joint encoders, torque sensors, and an inertial measurement unit to sense the patient's limb movements, the therapist's assistive actions, and the status of training equipment. During a training task on March 15, 20XX, the patient was required to complete a sequence of grasping-moving-placing movements. The therapist provided assistance or resistance adjustments based on the patient's real-time performance, and the robot dynamically adjusted its assistance strategy based on the patient's completion rate and the therapist's level of intervention. During the training, a sudden change in lighting caused by the patient's movement resulted in intermittent depth data jumps in the RGB-D camera between 14:23:45 and 14:24:12. Simultaneously, the therapist's rapid intervention provided assistance, making it difficult for the robot to accurately distinguish between the progress of the patient's autonomous movements and the progress contributed by the therapist.

[0138] like Figure 2-8 As shown, it includes the following steps:

[0139] Step 1: Acquire multimodal sensor data streams during the rehabilitation training process;

[0140] The robot synchronously collects multimodal sensor data during rehabilitation training through its perception module. The joint encoder outputs angle and angular velocity data for each joint at a frequency of 50Hz, the torque sensor outputs three-dimensional contact force data at the point of contact between the robot's end effector and the patient's arm, and the inertial measurement unit outputs the robot's posture quaternions and three-axis acceleration data. An RGB-D camera captures color and depth images of the training area at a frequency of 30Hz, and an infrared sensor outputs thermal images of the patient's and therapist's body surface temperature distribution. For the rehabilitation therapist's limb movement perception, the RGB-D camera outputs three-dimensional coordinate sequences of key points on the therapist's shoulder, elbow, and wrist. Hand movement sequences are represented by a sequence of depth image frames cropped from the hand region, with each frame having a resolution of 64×64 pixels.

[0141] Table 1. Raw data from multimodal sensors at the initial moment of rehabilitation training:

[0142] Table 2. Raw data fragments of hand movement sequences from rehabilitation therapists:

[0143] The data above shows that the therapist's hand is steadily approaching the patient's training area, the depth value of the center of the hand decreases at a rate of about 0.01m, and the hand posture angle changes smoothly, providing an input sequence for subsequent intention recognition.

[0144] Step 2: Calculate the real-time reliability weight of each sensor channel and mark suspected fault states;

[0145] Calculate the signal plausibility score for the abrupt change in the depth channel of the RGB-D camera at 14:23:45. The previous output value of this depth channel was 1.28m, and the current output value abruptly changes to 0.53m. The signal change rate is:

[0146] The physical limit of the depth channel of the RGB-D camera is determined by the product of the maximum hand movement speed of the therapist in the scene (2.5 m / s) and the sampling period (0.033 s), i.e. m. Substitute into the signal rationality score calculation formula:

[0147]

[0148] Simultaneously, the data from each sensor channel is input into a Kalman filter, and the fused state estimate is the spatial position of the therapist's hand. m. The depth component of the fused state is extracted from the observation mapping matrix of the RGB-D depth channel. The expected observation value is 1.25m, the actual observation value is 0.53m, and the observation residual is:

[0149] Preset standard deviation parameters for RGB-D depth channels Substituting m into the formula for calculating the mutual consistency score:

[0150]

[0151] The fusion coefficient is set to The real-time credibility weight is:

[0152] Table 3. Calculation results of sensor channel reliability weights:

[0153] Apply a sliding window length to the real-time confidence weights Smoothing process, and calculation of the smoothed confidence weight:

[0154] Preset threshold ,because The RGB-D depth channel was marked as a suspected faulty state.

[0155] Step 3: Recognizing the rehabilitation therapist's intentional movements based on trusted human sensor data;

[0156] At 14:23:44, the smoothed confidence weights of the human perception sensor channels (RGB-D camera and infrared sensor) were 0.89 and 0.91, respectively, both higher than the preset threshold of 0.70. The most recent sequence of the therapist's hand movements... The intent recognition network is input with frame depth image features. Temporal convolutional layers extract temporal pattern features of hand movements. Attention layers assign higher weights to keyframes (frame450 to frame452). The classification branch outputs the intent category as "auxiliary support", and the regression branch outputs the expected impact vector on each sub-task dimension.

[0157] Table 4 Results of rehabilitation therapist intent identification:

[0158] Intent recognition results showed that therapists performed assistive support actions between 14:23:44 and 14:23:45, with the largest expected contribution to the "mobility accuracy" subtask dimension (expected impact value 0.42 to 0.45), followed by the "grip stability" dimension (expected impact value 0.35 to 0.38). The intent category "assistive support" and the expected impact vector were then analyzed. The timestamp 14:23:45.000 is stored in the intent history buffer, and the buffer is kept up-to-date. One valid record.

[0159] Step 4: Estimate multi-dimensional task progress based on credibility-weighted environmental data;

[0160] The observation data from each environmental perception sensor channel are weighted and fused using their smoothed confidence weights. The smoothed confidence weight for the joint encoder channel is 0.95, for the torque sensor channel it is 0.91, for the inertial measurement unit channel it is 0.93, and for the RGB-D depth channel it is 0.654. ​​The feature vector output by the joint encoder is... Weighted features are .

[0161] The feature vector output by the torque sensor is Weighted features are All weighted features are concatenated to form a fused weighted environmental state feature vector, which is then input into the task progress estimation network.

[0162] Table 5. Results of Multi-Dimensional Task Progress Estimation:

[0163] Calculate the progress change at 14:23:45:

[0164]

[0165] This change in progress reflects the combined effect of patient voluntary actions, therapist-assisted support, and robot-assisted strategies.

[0166] Step 5: Separate the therapist contribution component and the robot contribution component from the multi-dimensional progress change.

[0167] At 14:23:45, the smoothed confidence weight of the human perception sensor channel is 0.89, which is higher than the preset threshold of 0.70, and precise contribution separation is performed.

[0168] Expected impact vector With progress change Find the minimum value element by element:

[0169]

[0170]

[0171]

[0172] The results show that the progress improvement at 14:23:45 was entirely contributed by the therapist's assistance, and the robot's current strategy did not contribute any additional progress.

[0173] At 14:23:46, the smoothed confidence weight of the RGB-D depth channel is 0.654, which is lower than the preset threshold of 0.70, triggering the conservative estimation mechanism. The intent category sequence of the 20 most recent valid records in the intent history buffer is input into the Hidden Markov Model, and the model outputs the posterior probability distribution of each intent category at the current time.

[0174] Table 6. Posterior probability distribution of therapist intention categories during suspected malfunctions:

[0175] Calculate the conservatively estimated therapist's expected impact vector:

[0176]

[0177]

[0178]

[0179]

[0180] Apply conservative amplification factor :

[0181]

[0182] The change in progress is Calculate the conservatively estimated therapist contribution component:

[0183]

[0184]

[0185]

[0186] Conservative estimates tend to attribute progress improvements to therapists, thus avoiding the possibility of the robot receiving inflated rewards.

[0187] Step 6: Generate a reshaped reward signal based on credibility-weighted contribution fusion;

[0188] Calculate the credibility weight of human perception The minimum smoothed confidence weights of the RGB-D camera channel and the infrared sensor channel are taken. At 14:23:46, the smoothed confidence weight of the RGB-D depth channel is 0.654, and the smoothed confidence weight of the infrared sensor channel is 0.88. Therefore... .

[0189] In precise contribution separation results Compared with the conservatively estimated robot contribution component Interpolation and fusion are performed between them:

[0190]

[0191] Define the importance weight vector for each subtask dimension. Calculate positive immediate reward signals:

[0192] Table 7: Data on the generation of excitation signals:

[0193] Since the therapist's intention category "Assistive Support" belongs to the active operation category set, corresponding to the dimensions of "Grasp Stability" and "Movement Accuracy," the exploration incentive components for these two dimensions are set to zero. The "Placement Accuracy" dimension is not covered by the therapist's active operations, and its exploration incentive component is... The vector form of the excitation signal is explored as follows: After weighted summation, the scalar form of the exploration incentive signal is obtained:

[0194] Set the exploration incentive trade-off coefficient The fusion generates and reshapes the reward signal:

[0195] Step 7: Apply attenuation to the reshaping reward signal during the suspected fault and perform backtracking correction after recovery;

[0196] Between 14:23:46 and 14:24:12, the RGB-D depth channel remained in a suspected fault state, and an attenuation factor was applied to the accumulated reshaping reward signal during this period. :

[0197] Table 8. Reward signal attenuation handling during suspected faults:

[0198] At 14:24:13, the smoothed confidence weight of the RGB-D depth channel rebounded to 0.78, exceeding the preset threshold of 0.70, and the sensor returned to normal. The fault initiation time was... The fault recovery time is Obtain the multi-dimensional progress vectors at two time points:

[0199] Calculate the cumulative schedule change during the suspected failure period:

[0200] Based on the recovered reliable human perception sensor data, the therapist's intentional actions during the suspected malfunction were re-identified. Combined with the robot's own motion recordings, the therapist's actual cumulative contribution was calculated in reverse. Calculate the corrected cumulative contribution component of the robot:

[0201] Generate backtracking correction rewards:

[0202]

[0203] The sum of decayed reward signals accumulated during the suspected fault period is The backtracking correction rewards are redistributed according to the original reward proportions at each time point. For example, the original reward proportion at time 14:23:46 is...

[0204] The reward has been corrected to .

[0205] Step 8: Adjust the policy network parameters based on the backtracked and corrected reward sequence and generate action instructions;

[0206] The backtracked and corrected complete reconstructed reward signal sequence is input into the meta-learning algorithm (MAML algorithm). In the inner loop phase, the policy gradient is calculated based on the state-action trajectory and corrected reward sequence of a single rehabilitation training task.

[0207] The total time steps of the trajectory Meta-learning rate In the outer loop phase, based on the performance of the inner loop update on multiple rehabilitation training task variants (different patients, different rehabilitation stages), the meta-gradient is calculated and the initial parameters of the policy network are updated. .

[0208] To avoid drastic fluctuations in the policy network parameters, a gradual integration mechanism is applied to the difference in rewards before and after backtracking correction, and the number of gradual integration rounds is set. The rewards used in the first round of parameter updates were:

[0209] In the fifth round of parameter updates, the reward was fully transitioned to the adjusted value of 0.016.

[0210] Updated policy network Observation of the current environmental status The robot generates motion commands (including features such as patient arm position, training equipment status, and therapist assistance level). The policy network outputs the probability distribution of each motion and selects the motion with the highest probability, "reduce the assisting torque and guide the patient to complete the placement independently." The motion command includes a target shoulder joint angle of 35.2°, a target elbow joint angular velocity of 6.5° / s, and a target end effector contact force of 1.5N. The execution module drives the robot hardware to execute this motion, assisting the patient to complete the final stage of the placement task independently with minimal therapist intervention.

[0211] Throughout the implementation process, data starts from the initial multimodal sensor raw data stream (Tables 1 and 2), and after confidence weight calculation (Table 3), intermittent faults in the RGB-D depth channel are identified. Under a reliable sensor state, the therapist's intention is identified based on hand movement sequences (Table 4), and multidimensional task progress is estimated by combining confidence-weighted environmental data (Table 5). Precise contribution separation is performed to attribute progress changes to the respective contributions of the therapist and the robot. During suspected fault periods, the probability distribution of the therapist's intention is inferred using a Hidden Markov Model (Table 6), generating a conservatively estimated contribution separation result. A confidence-weighted interpolation fusion mechanism is then used to generate a reshaped reward signal (Table 7). After fault recovery, backtracking correction is performed based on complete reliable data (Table 8), replacing the decayed reward during the suspected fault period with the corrected accurate reward signal. Finally, this is input into a meta-learning algorithm to adjust the policy network parameters. The adjusted policy network generates action instructions adapted to the patient's rehabilitation progress and the therapist's level of intervention, achieving effective human-machine collaboration in rehabilitation training tasks. The entire data flow process maintains logical coherence from raw sensor data to policy network parameter updates. Through a collaborative mechanism of credibility assessment, conservative estimation, and backtracking correction, accurate reward attribution and policy learning quality are maintained even under intermittent sensor failure conditions.

[0212] The embodiments of the present invention have been described above. However, the embodiments are not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make more equivalent embodiments under the guidance of the present embodiments, and all of them are within the protection scope of the present embodiments.

Claims

1. An adaptive learning method for an embodied intelligent robot, characterized in that, Includes the following steps: The system acquires environmental state observation data streams from multimodal sensors during task execution. The multimodal sensors include sensor channels for environmental perception and sensor channels for human perception. Based on the signal rationality score and mutual consistency score of each sensor channel, the real-time reliability weight of each sensor channel is calculated, and sensor channels with reliability weights lower than a preset threshold are marked as suspected fault states. When the confidence weight of the human perception sensor channel is higher than the preset threshold, the hand movement sequence of the human collaborator is input into the intention recognition network, and the intention category of the current human action and its expected impact vector on each sub-task dimension are output. The confidence-weighted environmental state observation data is input into the task progress estimation network to obtain a multi-dimensional progress vector and calculate the progress change between adjacent time points. Based on the expected influence vector, the human contribution component and the robot contribution component are separated from the progress change; when the human perception sensor channel is in a suspected fault state, the probability distribution of human behavior patterns is inferred based on the intention history buffer and the hidden Markov model to generate a conservatively estimated robot contribution component. Based on the human perception credibility weight, interpolation and fusion are performed between the precise contribution separation result and the conservatively estimated robot contribution component to generate a reshaped reward signal. An attenuation factor is applied to the remodeling reward signal during suspected faults, and a backtracking correction is performed based on complete and reliable data after the sensor returns to normal. Based on the backtracked and corrected reward signal sequence, the policy network parameters are adjusted through a meta-learning algorithm to generate robot action commands.

2. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The calculation of the real-time reliability weight for each sensor channel, based on the signal reasonableness score and mutual consistency score of each sensor channel, includes: For the signal value output by each sensor channel at the current moment, calculate the signal change rate of adjacent moments, subtract the absolute value of the signal change rate from one to the ratio of the physical limit change rate of the sensor channel, and take the maximum value of zero to obtain the signal rationality score; wherein, the physical limit change rate is the maximum reasonable change amplitude of the sensor within a single sampling period under normal operating conditions and constrained by the dynamic characteristics of the measured physical quantity. The data from each sensor channel is input into a Kalman filter to obtain a fusion state estimate. The observation residual between the actual observation value of each sensor channel and the expected observation value obtained by mapping the fusion state estimate through the observation mapping matrix is ​​calculated. Based on the observation residual and the preset standard deviation parameter, a mutual consistency score is generated by a Gaussian kernel function. The signal rationality score and the mutual consistency score are weighted and summed according to the fusion coefficient to obtain the real-time reliability weight of each sensor channel.

3. The adaptive learning method for an embodied intelligent robot according to claim 2, characterized in that, Also includes: Apply time window smoothing to the real-time credibility weights, and take the moving average of the real-time credibility weights at the most recent preset window length times to obtain the smoothed credibility weights. Sensor channels whose smoothed confidence weight is lower than the preset threshold are marked as suspected fault states; in subsequent steps, the smoothed confidence weight is used to replace the real-time confidence weight for judgment and calculation.

4. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The intent recognition network is a neural network based on temporal convolution and attention mechanisms, and its processing flow includes: The hand motion sequence is input into the temporal convolutional layer, and local temporal pattern features are extracted by sliding a one-dimensional convolutional kernel along the time dimension. The importance weights of features at each time step are calculated through an attention layer and then weighted and aggregated. The classification branch uses a fully connected layer and a softmax activation function to output the intent category, while the regression branch uses a fully connected layer to output the expected impact vector for each subtask dimension under the corresponding intent category. Each time the intent recognition network outputs a valid result, the intent category, the expected impact vector, and the corresponding timestamp are stored in the intent history buffer to maintain the most recent preset number of valid records.

5. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The process of inputting the credibility-weighted environmental state observation data into the task progress estimation network to obtain a multi-dimensional progress vector includes: The observation data feature vectors output by each environmental perception sensor channel are multiplied by the corresponding confidence weight of the sensor channel to obtain weighted features. The weighted features of all sensor channels are concatenated or summed to form a weighted environmental state feature vector. The weighted environmental state feature vector is input into the task progress estimation network, and the current completion progress value of each sub-task dimension is output to form the multi-dimensional progress vector. The difference between the multi-dimensional progress vector at the current moment and the multi-dimensional progress vector at the previous moment is calculated to obtain the progress change.

6. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The separation of human and robot contribution components from the progress change based on the expected influence vector includes: When the confidence weight of the human perception sensor channel is higher than the preset threshold, the minimum value of the expected influence vector and the progress change in each dimension is taken as the human contribution component, and the difference between the progress change and the human contribution component is taken as the robot contribution component. When the confidence weight of the human perception sensor channel is lower than the preset threshold, the intention category sequence in the intention history buffer is input into the Hidden Markov Model, and the posterior probability distribution of each intention category at the current time is output. Based on the posterior probability distribution, the expected impact vector corresponding to each intention category is expected to be calculated to obtain a conservatively estimated human expected impact vector. After applying a conservative amplification factor greater than one to the conservatively estimated human expected impact vector, the conservatively estimated human contribution component and the conservatively estimated robot contribution component are calculated in the same way as the precise contribution separation.

7. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The generation of the reshaping reward signal includes: Based on the human perception credibility weight, the precise contribution separation result and the conservatively estimated robot contribution component are linearly interpolated to obtain the credibility-weighted robot contribution component. The transpose of the importance weight vector of each subtask dimension is multiplied by the credibility-weighted robot contribution component to obtain a positive instant reward signal. An exploration incentive signal is generated based on the difference between the current progress value and the target progress value of each sub-task dimension. Specifically, when the current human intention category belongs to a predefined set of active operation categories, the exploration incentive component of the corresponding sub-task dimension is set to zero; when the probability of the active operation intention inferred during the fault exceeds a preset probability threshold, the exploration incentive component of the corresponding sub-task dimension is set to zero. The positive immediate reward signal is added to the exploration incentive signal weighted by a tradeoff coefficient to output the reshaping reward signal.

8. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The step of performing backtracking correction based on complete and reliable data after the sensor returns to normal includes: The difference between the multi-dimensional progress vector at the fault recovery time and the multi-dimensional progress vector at the fault initiation time is obtained to obtain the cumulative progress change during the suspected fault period. Based on the recovered reliable human perception sensor data, the human action intentions during the suspected malfunction are re-identified, the corrected human cumulative contribution component is obtained, and the corrected robot cumulative contribution component is calculated. The transpose of the importance weight vector of each subtask dimension is multiplied by the corrected cumulative contribution component of the robot to generate the backtracking correction reward. Calculate the proportion of the original reward at each time point during the suspected fault period to the total original reward during the fault period, and redistribute the backtracking correction reward to each time step according to this proportion to obtain the correction reward at each time point. When the duration of a suspected fault exceeds a preset maximum duration threshold, only the rewards within half of the preset maximum duration threshold before and after the start and end of the suspected fault are retrospectively corrected, while the attenuated reward signal in the middle period remains unchanged.

9. The adaptive learning method for an embodied intelligent robot according to claim 1, characterized in that, The adjustment of policy network parameters through meta-learning algorithms includes: The meta-learning algorithm adopts the MAML algorithm, which includes an inner loop stage and an outer loop stage. The inner loop stage performs policy gradient updates on each task variant to obtain task-specific policy network parameters. The outer loop stage calculates the meta-gradient of the initial parameters of the policy network based on the performance of the inner loop update on each task variant and updates the initial parameters of the policy network. Based on the backtracked and corrected reward signal sequence and the corresponding environment state-action trajectory, the gradient update amount of the policy network parameters is calculated. The gradient update amount is the expected value of the product of the corrected reward and the gradient of the log probability of the policy network outputting the corresponding action under the corresponding observation on the trajectory. A progressive integration mechanism is applied to the difference in rewards before and after the backtracking correction. The progressive integration rounds are set, and in each round of parameter updates, the difference in rewards is gradually added to the reward sequence according to the ratio of the current round number to the progressive integration round number.

10. An adaptive learning system for an embodied intelligent robot, used to execute the adaptive learning method for the embodied intelligent robot according to any one of claims 1 to 9, characterized in that, include: The multimodal data acquisition module is used to acquire environmental state observation data streams from multimodal sensors during task execution. The credibility assessment and fault marking module is used to calculate the real-time credibility weight based on the signal rationality score and mutual consistency score of each sensor channel, and mark the sensor channel with the credibility weight below the preset threshold as a suspected fault state. The intent recognition module is used to output the intent category and expected impact vector of the current human action based on the intent recognition network when the confidence weight of the human perception sensor channel is higher than the preset threshold. The task progress estimation module is used to obtain a multi-dimensional progress vector and calculate the progress change based on the confidence-weighted environmental state observation data. The contribution separation module is used to separate the human contribution component and the robot contribution component from the progress change based on the expected influence vector, and to generate a conservative estimate of the robot contribution component based on the intention history buffer and the hidden Markov model when the human perception sensor channel is in a suspected fault state. The reward generation module is used to interpolate and fuse the precise contribution separation result and the conservatively estimated robot contribution component based on the human perception credibility weight, and generate a reshaped reward signal. The backtracking correction module is used to apply an attenuation factor to the reshaping reward signal during suspected faults and perform backtracking correction based on complete and reliable data after the sensor returns to normal. The strategy update and execution module is used to adjust the policy network parameters and generate robot action instructions based on the backtracked and corrected reward signal sequence using a meta-learning algorithm.

Images