Remote operation data acquisition and data closed-loop system for intelligent humanoid robot with body

Abstract

The invention belongs to the technical field of intelligent humanoid robots, and discloses a remote operation data acquisition and data closed-loop system for an intelligent humanoid robot, which builds an instruction, action and acquisition ternary time sequence synchronization mechanism through a time sequence coordination module, realizes dynamic coordination of the three parts by using a time sensitive network technology, effectively reduces time sequence deviation, and improves the system performance. Meanwhile, a space-time alignment algorithm of the data processing module is combined, so that the consistency of the multi-modal data in time and space dimensions is guaranteed; acquired data can accurately fit human operation logic and a robot action state, the optimization efficiency of a data closed loop is improved, and the requirements of high-precision operation scenes such as complex assembly and flexible material processing are successfully met; a quantitative association system of data quality and control effects is established by means of the evaluation module, invalid data such as sensor faults and action fuzziness can be screened in real time, and occupation of redundant storage and computing resources is reduced.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligent robot technology, specifically a remote control data acquisition and data closed-loop system for embodied intelligent humanoid robots. Background Technology

[0002] The remote-controlled data acquisition and closed-loop data system for embodied intelligent humanoid robots guides the robot to complete tasks through remote operation, simultaneously collecting multimodal data and forming a complete closed loop of acquisition, training, optimization, and application, providing high-quality data support for improving the robot's autonomous capabilities. Current related technologies still face the following technical challenges in adapting to complex scenarios and mining data value: In existing systems, remote control command transmission, robot motion execution, and multi-source sensor data acquisition are mostly independent time-series links, lacking a dynamic coordination mechanism. In high-precision operation scenarios such as complex assembly and flexible material handling, the execution delay of remote control commands will deviate from the timing of sensor data acquisition, making it difficult for the acquired data to accurately match human operation intentions and robot motion states, resulting in operation and data mismatch. This will make it difficult for subsequent motion models trained based on the acquired data to reproduce the real operation logic, thereby reducing the optimization efficiency of data closure. In addition, existing technologies mostly focus on delay optimization of a single link and do not carry out systematic design for the coordinated timing of commands, actions, and data.

[0003] Most current remote control data acquisition systems only complete data collection and storage, without establishing a real-time feedback mechanism between data quality and control strategies. The validity of the collected data can only be verified during subsequent offline training, resulting in invalid data consuming a large amount of storage and computing resources. At the same time, the optimized strategies obtained through training cannot provide real-time feedback for adjusting remote control parameters, causing the remote control process to always rely on human experience and making it difficult to continuously improve accuracy through data closure. This restricts the rapid iteration of embodied intelligent robots from remote control demonstration to autonomous operation, especially in dynamic and unknown environments, where the inability to optimize remote control strategies through real-time data feedback ultimately leads to a decrease in the success rate of operations. Summary of the Invention

[0004] The purpose of this invention is to provide a remote-controlled data acquisition and data closed-loop system for an intelligent humanoid robot to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a remote control data acquisition and data closed-loop system for an embodied intelligent humanoid robot, comprising a timing coordination module, a data processing module, a transmission module, an evaluation module, a driving module, an optimization module, and a fault-tolerant adaptive module; Preferably, the timing coordination module receives multimodal commands output by the operator through various types of remote control terminals such as VR devices, force feedback gloves, and motion capture systems. It employs a multimodal command fusion algorithm based on a lightweight neural network to filter command noise and perform semantic parsing, distinguishing between valid commands and interference signals. Based on a real-time robot dynamics identification model, it introduces an event triggering mechanism to generate adaptive motion adaptation rules according to the operator's dynamic characteristics such as the amplitude and speed of the movements, realizing the conversion from human joint space movements to robot joint space movements. Through standardized interface design, it supports plug-and-play functionality for different remote control terminals. The multimodal command fusion algorithm prioritizes "attitude command > force control command > voice command". Attitude commands and force control commands are weighted and fused with weights of 0.5 and 0.3 respectively. Voice commands are used as auxiliary correction signals with a weight of 0.2. During the fusion process, a sliding window is used to smooth the fluctuations of different modal commands to ensure the stability of the output commands.

[0006] The triggering condition of the event triggering mechanism is that the change in the operator's motion amplitude is ≥15° or the change in motion speed is ≥0.5m / s per unit time. After triggering, the adaptive motion adaptation rules are updated every 20ms. The adaptation rules include joint angle mapping coefficients and motion smooth transition parameters. The joint angle mapping coefficients are dynamically calculated based on the ratio of the robot's degrees of freedom to the range of motion of human joints.

[0007] A three-dimensional timing synchronization engine for commands, actions, and data acquisition is constructed. Time-sensitive networking technology is used to achieve timing alignment of remote command transmission, robot action execution, and sensor data acquisition. The complexity of the operation scenario is monitored in real time and the timing synchronization threshold is dynamically adjusted. At the same time, the timing synchronization signal is synchronized to the data processing module.

[0008] The complexity of the operation scenario is quantified by weighted multi-dimensional indicators. The core indicators include operation accuracy requirements (such as millimeter / centimeter level), action continuity (number of action switching per unit time), and environmental dynamics (speed / number of obstacles). The weights of each indicator are 0.4, 0.3, and 0.3, respectively, and the quantification result is a value of 0-10, which is used for accurate adaptation of the timing synchronization threshold.

[0009] The lightweight neural network adopts a lightweight architecture that integrates CNN and LSTM, retaining only the core feature extraction layer and semantic parsing layer, ensuring that the instruction processing latency is ≤20ms. The dynamic adjustment of the timing synchronization threshold is based on the quantified value of the complexity of the work scenario. The scenario complexity is calculated by weighting the operation accuracy requirements (e.g., ±0.1mm is a high-precision scenario) and the action continuity (e.g., ≥3 action switching times per second is a high-complexity scenario). The threshold range is adaptively matched between 1ms and 5ms, with the threshold for high-precision scenarios ≤2ms and the threshold for conventional scenarios ≤5ms.

[0010] Preferably, the data processing module receives the timing synchronization signal from the timing coordination module, integrates the robot's visual sensor, tactile sensor, inertial sensor, and environmental sensor to construct a multi-source heterogeneous data acquisition network, and uses a spatiotemporal alignment algorithm to achieve real-time synchronous acquisition of multimodal data. Combining the dynamic complexity assessment results of the operation scenario output by the remote control command priority and timing coordination module, the acquisition frequency of each sensor is dynamically adjusted through reinforcement learning algorithm; a data quality pre-assessment subunit is set up to filter the acquired data in real time based on preset multi-dimensional indicators such as action consistency and environmental recognition, and to remove invalid data such as sensor fault data and action ambiguity data; The remote control commands are prioritized into three levels: Level 1 is for emergency operations, such as stop or emergency stop; Level 2 is for core operations, such as grasping or assembling; and Level 3 is for auxiliary operations, such as position fine-tuning or attitude adjustment. The priority is distinguished by the priority identifier field in the command header. The data processing module prioritizes responding to the sensor data acquisition needs corresponding to Level 1 and Level 2 commands.

[0011] The reward function of the reinforcement learning takes data quality compliance rate (weight 0.6), system energy consumption (weight 0.2), and data acquisition real-time performance (weight 0.2) as the core indicators. Positive rewards are given when data quality meets the standards, and negative penalties are given when energy consumption exceeds the standard or acquisition delay exceeds the threshold. The effective data is standardized and its features are initially extracted to generate structured data blocks, which are then transmitted to the transmission module.

[0012] The initial feature extraction targets and extracts core features from different sensor data. Visual sensors extract target contours, key positioning points, and distance information; tactile sensors extract contact pressure distribution, contact area, and peak interaction force; inertial sensors extract joint angular velocity, acceleration, and attitude angle changes; and environmental sensors extract obstacle distance, ambient light intensity, temperature, and humidity data.

[0013] The spatiotemporal alignment algorithm is implemented in two steps: timestamp interpolation calibration and spatial coordinate system. The time dimension is completed by linear interpolation of the timestamp difference of sensor data, and the spatial dimension is based on the robot's base coordinate system, mapping the data of each sensor to a unified coordinate system. Among the multi-dimensional data quality pre-assessment indicators, the threshold for judging action consistency is that the joint angle deviation between the actual action and the commanded action is ≤3°, and the threshold for judging environmental recognition is that the image clarity is ≥85% and the LiDAR point cloud density is ≥100 points / cm². Data that does not meet the threshold is judged as invalid data.

[0014] Preferably, the transmission module receives the structured data blocks output by the data processing module and the instruction transmission requirements of the timing coordination module. It adopts an edge computing architecture, deploys lightweight data transmission nodes locally on the robot, and pushes some data preprocessing tasks to the edge to realize edge preprocessing and low-latency transmission of remote control instructions and collected data. The edge preprocessing includes three types of operations: data pruning, outlier removal, and feature simplification. The preprocessed data volume is reduced by 30%-40% to ensure low-latency transmission. Data pruning refers to removing redundant fields from sensors. Outlier removal refers to filtering data that exceeds a reasonable range based on the 3σ principle. Feature simplification specifically involves retaining core features relevant to the task and discarding secondary features.

[0015] At the transmission layer, a lightweight transmission protocol is adopted, combined with an efficient data compression algorithm to reduce bandwidth consumption; at the verification layer, a two-way intelligent verification mechanism is constructed to perform dual verification of the integrity and validity of the issued remote control commands, identify command loss or errors through hash verification technology, perform secondary quality verification on the uploaded collected data, and comprehensively judge the data credibility by combining sensor working status data. The lightweight transmission protocol is based on an optimized design of the UDP protocol, which trims redundant fields in the protocol header and retains only the core fields of address identifier, data length, and checksum. The data payload adopts a compact format of "sensor type + timestamp + core data". A transmission status monitoring subunit is set up to monitor key indicators such as link bandwidth and latency in real time. When a transmission anomaly is detected, it automatically switches to the backup transmission link and transmits the verified data and instructions to the evaluation module and driver module, respectively.

[0016] The high-efficiency data compression algorithm adopts a differential compression architecture for sensor time-series data, storing only the difference between adjacent data and key feature points, with the compression ratio controlled between 3:1 and 5:1; the monitoring threshold for key link indicators is set as follows: when the bandwidth is less than 1Mbps, the latency is greater than 50ms, and the packet loss rate is greater than 1%, it is judged as a transmission anomaly, triggering redundant link switching, and the backup link switching time is ≤10ms.

[0017] Preferably, the evaluation module receives the verified multi-source collected data transmitted by the transmission module, and uses a cross-modal attention mechanism to deeply mine the correlation features between different modal data, so as to achieve efficient fusion of multimodal data and generate a unified robot operation state description vector. This vector can characterize the robot's action posture, environmental interaction force and surrounding environment features. The operation status description vector is a 128-dimensional vector, containing 32-dimensional motion posture features, 32-dimensional environmental interaction force features, 32-dimensional environmental features, and 32-dimensional status evaluation features. The values ​​of each dimension of the vector are normalized to the interval [0, 1]. The motion posture features include the angles of each joint and the posture of the end effector. The environmental interaction force features include the pressure at each contact point, the magnitude and direction of the resultant force, etc. The environmental features include the distribution of obstacles, the size of the work space, and the lighting conditions, etc. The status evaluation features include the data quality score, the progress of the action completion, and the equipment operating status, etc.

[0018] The cross-modal attention mechanism prioritizes modal features related to the core requirements of the operation. For example, in assembly operations, it prioritizes the allocation of attention weights for tactile sensors (interactive force data) and visual sensors (positioning data) (total weight ≥ 70%), and in environmental perception scenarios, it increases the weight of LiDAR data (weight ≥ 50%).

[0019] The fused data is further denoised using a deep learning model to filter out noise caused by environmental interference, while extracting key features to enhance the data's representation capabilities. The deep learning denoising model adopts an autoencoder architecture. The input layer receives the fused multimodal data, the hidden layer separates the signal and noise through an encoding-decoding process, and the output layer outputs the denoised data. During the model training process, the goal is to improve the data signal-to-noise ratio, focusing on suppressing signal distortion caused by environmental electromagnetic interference and inherent sensor noise.

[0020] A multi-dimensional data quality assessment model is established to quantitatively correlate data quality with the actual control effect of the robot, outputting quality grading results of three levels: excellent, qualified, and poor. The reasons for poor data are accurately labeled, and the excellent and qualified data and quality assessment results are transmitted to the drive module and optimization module.

[0021] The specific dimensions of the multi-dimensional data quality assessment model include data integrity (missing rate ≤5%), temporal consistency (deviation ≤5ms), signal-to-noise ratio (≥30dB), and control effect matching degree (action accuracy deviation ≤0.5mm, operation success rate ≥80%). The weights of each dimension are 0.2, 0.2, 0.1, and 0.5, respectively. After weighted calculation, a score ≥85 is considered high-quality data, 60-84 is considered qualified data, and <60 is considered poor-quality data.

[0022] Preferably, the drive module receives the operation status description vector and data quality evaluation result output by the evaluation module, and at the same time receives the instruction conversion result of the timing coordination module to construct a dynamic weight allocation model and realize the real-time allocation of weights for remote control and robot autonomous control. A collaborative control model is constructed using deep reinforcement learning algorithms. The control strategy is optimized through scenario training, which improves the autonomous control weight in simple repetitive task scenarios and the remote control weight in complex or dynamically unknown scenarios. At the same time, a model predictive control algorithm is introduced to optimize the robot joint control parameters based on the prediction results of the future short-term task state. The model predictive control algorithm first predicts the robot joint motion trajectory for the next 3-5 control cycles based on historical operation data and the current state; then, by comparing the deviation between the predicted trajectory and the expected trajectory, it optimizes control parameters such as joint stiffness and damping coefficient in reverse; finally, it optimizes the parameters in the rolling time domain with the time domain length consistent with the prediction cycle to ensure the real-time performance and adaptability of parameter adjustments.

[0023] The state space of the deep reinforcement learning includes four core parameters: task scenario complexity score, data quality level, robot action deviation value, and remaining task quantity; the action space is the weight adjustment step size of remote control and autonomous control, with each adjustment ranging from 5% to 10%.

[0024] A real-time motion deviation correction subunit is set up. By frequently comparing the deviation between the actual working state and the expected state, an adaptive correction command is generated and fed back to the execution end to further improve the control accuracy. At the same time, the control effect data is fed back to the optimization module.

[0025] The weight calculation of the dynamic weight allocation model is based on the complexity score of the operation scenario (0-10 points) and the data quality level. When the scenario is simple (complexity ≤ 3 points) and the proportion of high-quality data is ≥ 70%, the autonomous control weight is 60%-80%; when the scenario is complex (complexity ≥ 7 points) or the proportion of poor-quality data is ≥ 30%, the remote control weight is 60%-90%. The prediction time domain of the model predictive control is set to 5-10 control cycles, the control time domain is set to 3-5 control cycles, and the action deviation correction response time is ≤ 10ms.

[0026] Preferably, the optimization module receives high-quality data and quality labeling results output by the evaluation module and control effect data fed back by the drive module, generates data labels through an automated labeling algorithm, and stores them in a structured training dataset for incremental training of the robot motion model. The incremental training adopts a small-batch training mode, triggering training once every 500 high-quality data points. During the training process, the bottom feature extraction layer of the model is frozen, and only the parameters of the top decision layer are fine-tuned. After training is completed, the model performance is verified (comparing the core indicators before and after training). If the verification is successful, the model parameters are updated; if it fails, the original parameters are retained and training data is supplemented.

[0027] The automated labeling algorithm has a built-in job type-action feature mapping rule library that covers typical jobs such as assembly, grasping, and placement. By matching action features such as joint angle sequences and peak interaction forces in sensor data with rule library entries, it automatically generates structured labels. Abnormal feature data triggers manual review and labeling.

[0028] The optimized model parameters obtained through training are fed back to the temporal coordination module in real time to optimize the adaptation rules and temporal synchronization thresholds, and then fed back to the driving module to update the control weights and model prediction control parameters. The reasons for the poor quality of the labeled data are analyzed in depth, and targeted suggestions for adjusting the collection parameters are generated in combination with the scene characteristics. These suggestions are then fed back to the data processing module and the temporal coordination module for parameter updates. Set up a closed-loop effect evaluation sub-unit, regularly collect statistics on core indicators such as job success rate, action accuracy, and system response speed, establish an evaluation model, and dynamically optimize the closed-loop iteration cycle and training parameters.

[0029] The label format of the automated annotation algorithm is "job type-action feature-quality level-collection scene". The structured training dataset is stored according to scene type. The dynamic optimization of the closed-loop iteration cycle is based on the core indicator compliance rate. When the job success rate is ≥95% and the action accuracy deviation is ≤0.3mm, the iteration cycle is extended to 1 hour / time. When the standard is not met, it is shortened to 10 minutes / time.

[0030] Preferably, the fault-tolerant adaptive module monitors the operating status of the aforementioned modules, including key indicators such as transmission link bandwidth, latency, sensor operating status, computing power consumption of each module, and robot motion execution accuracy, to construct a multi-dimensional and comprehensive status monitoring matrix; and adopts a fault prediction model based on machine learning to provide early warning of potential faults by analyzing historical operating data and real-time status data. The multi-dimensional status monitoring matrix comprises six core dimensions: transmission link, sensor status, module computing power, action execution, data quality, and environmental status. Data acquisition frequencies for each dimension range from 10Hz to 100Hz, categorized by importance. The transmission link dimension includes bandwidth, latency, and packet loss rate; the sensor status dimension includes operating voltage, data output frequency, and deviation value; the module computing power dimension includes CPU utilization, memory usage, and task processing time; the action execution dimension includes joint angle deviation, end-effector position accuracy, and action completion time; the data quality dimension includes integrity, temporal consistency, and signal-to-noise ratio; and the environmental status dimension includes temperature, humidity, and electromagnetic interference intensity.

[0031] The fault prediction model adopts the gradient boosting decision tree algorithm. The input features include the mean, variance, and number of mutations of sensor data, the moving average and fluctuation coefficient of link delay, the duration of module computing power occupation, the peak percentage, and other key features. The model prediction lead is 500ms-1s.

[0032] When a module failure or performance degradation is detected, a tiered fault tolerance strategy is automatically triggered. In the case of a minor failure, an adjustment command is sent to the data processing module to activate redundant sensors. In the case of a serious failure, a switching command is sent to the drive module to switch to a safety control mode, stop unnecessary operations, and provide timely feedback for manual intervention. Fault data and the fault tolerance adjustment process are recorded, and a standardized fault analysis report is generated.

[0033] The input feature dimensions of the fault prediction model include sensor data deviation rate (≥10% is abnormal), link delay fluctuation amplitude (≥20ms is abnormal), module computing power utilization rate (≥90% is abnormal), and action execution accuracy deviation (≥1mm is abnormal). The fault level classification criteria are: minor fault (single feature abnormality and no impact on operation) and serious fault (two or more feature abnormalities or have caused operation deviation ≥2mm). In the case of minor fault, the data processing module is instructed to reduce the frequency of non-critical data acquisition and to activate redundant sensors. In the case of serious fault, non-core operation processes are interrupted first.

[0034] The beneficial effects of this invention are as follows: 1. This invention establishes a three-dimensional timing synchronization mechanism for instructions, actions, and data acquisition through a timing coordination module. It utilizes time-sensitive networking technology to achieve dynamic coordination among the three, effectively reducing timing deviations. Simultaneously, it combines the spatiotemporal alignment algorithm of the data processing module to ensure the consistency of multimodal data in both time and space dimensions. This solves the problem of data and operational intent misalignment caused by the independent timing and lack of coordination among the three components in existing systems. It enables the acquired data to accurately match human operational logic and robot action states, improving the optimization efficiency of data closed-loop and successfully adapting to the needs of high-precision operation scenarios such as complex assembly and flexible material handling.

[0035] 2. This invention establishes a quantitative correlation system between data quality and control effectiveness based on the evaluation module. This system can filter out invalid data such as sensor malfunctions and ambiguous actions in real time, reducing the occupation of redundant storage and computing resources. The drive module, based on the data quality evaluation results, adjusts the weight ratio of remote operation and autonomous control in real time through a dynamic weight allocation model. In simple repetitive tasks, it strengthens autonomous control to reduce manual workload, and in complex or dynamically unknown scenarios, it improves remote control to ensure operational safety. At the same time, it further optimizes control accuracy through real-time deviation correction. This invention breaks through the limitations of existing technologies that rely on offline data verification and manual remote operation, realizing real-time linkage between data quality and control strategies. It accelerates the iterative process of robots from remote operation demonstration to autonomous operation and improves the success rate of operations in dynamic environments.

[0036] 3. This invention employs an edge computing architecture and lightweight transmission protocol in its transmission module to ensure low-latency data and command transmission. Combined with a bidirectional intelligent verification mechanism and redundant link switching, it effectively avoids command errors and data loss, improving the stability of the transmission link. The fault-tolerant adaptive module constructs a multi-dimensional status monitoring matrix, utilizing machine learning to provide early warnings of potential faults. Combined with a hierarchical fault-tolerant strategy, redundant sensors are activated for minor faults, and a safety control mode is switched for severe faults, ensuring that core functions are not affected. The optimization module uses high-quality data to perform incremental model training, feeding back optimization parameters to relevant modules in real time. It then adjusts the acquisition parameters and control strategies accordingly, and dynamically optimizes closed-loop operating parameters through core indicator evaluation, continuously improving system performance such as operational accuracy and response efficiency to meet diverse application needs. Attached Figure Description

[0037] Figure 1 This is a flowchart of the overall system of the present invention; Figure 2 This is a flowchart illustrating the data acquisition and timing coordination process of this invention. Figure 3 This is a flowchart of the data transmission and quality assessment process of this invention; Figure 4 This is a flowchart of the control optimization and fault-tolerant adaptive process of the present invention. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] like Figures 1 to 4As shown, this embodiment of the invention provides a remote-controlled data acquisition and closed-loop data system for an embodied intelligent humanoid robot, including a timing coordination module, a data processing module, a transmission module, an evaluation module, a driving module, an optimization module, and a fault-tolerant adaptive module. The specific implementation of each module is as follows: The timing coordination module receives multimodal commands from operators via various remote control terminals, such as VR devices, force feedback gloves, and motion capture systems. These multimodal commands include posture commands, force control commands, and voice commands. A multimodal command fusion algorithm based on a lightweight neural network is employed to filter command noise and perform semantic parsing, distinguishing between valid commands and interference signals, thus improving command recognition accuracy. Based on a real-time robot dynamics identification model, an event triggering mechanism is introduced to generate adaptive motion adaptation rules according to the operator's dynamic characteristics, such as the amplitude and speed of their movements. This enables precise conversion between human joint space movements and robot joint space movements. A standardized interface design supports plug-and-play functionality for different remote control terminals, reducing system deployment costs.

[0040] The standardized interface is designed based on the ROS (Robot Operating System) communication protocol and includes three types: data input interface, command output interface, and status feedback interface. The interface adopts a unified data format (JSON format) with clearly defined fields, including device ID, command type, data length, timestamp, and checksum. It supports plug-and-play adaptation to common remote control terminals such as VR devices and force feedback gloves.

[0041] The robot dynamics real-time identification model takes robot joint torque, angular velocity, and angular acceleration as input features, updates model parameters through an online recursive algorithm, adapts to changes in robot load and differences in joint friction characteristics, and supports real-time calibration of dynamic parameters for 6-24 degree-of-freedom humanoid robots.

[0042] A three-dimensional timing synchronization engine for commands, actions, and data acquisition is constructed. Time-Sensitive Networking (TSN) technology is used to achieve timing alignment of remote command transmission, robot action execution, and sensor data acquisition, with a timing deviation of no more than 5ms. The engine monitors the complexity of the operation scenario in real time and dynamically adjusts the timing synchronization threshold to adapt to different scenario requirements from routine operations to high-precision operations. At the same time, the timing synchronization signal is synchronized to the data processing module to ensure the coordination between data acquisition and command execution.

[0043] The data processing module receives timing synchronization signals from the timing coordination module, integrates robot body vision sensors (RGB-D camera), tactile sensors (finger-tip array sensors), inertial sensors (IMU), and environmental sensors (LiDAR) to construct a multi-source heterogeneous data acquisition network; and uses a spatiotemporal alignment algorithm to achieve real-time synchronous acquisition of multimodal data, ensuring the consistency of data from different sensors in time and space dimensions. By combining the dynamic complexity assessment results of the operation scenario output by the remote control command priority and timing coordination module, the acquisition frequency of each sensor is dynamically adjusted through reinforcement learning algorithm. For example, the acquisition frequency of tactile data is increased to 100Hz in fine operation scenarios and reduced to 50Hz in normal scenarios, so as to reduce energy consumption while ensuring data quality. A data quality pre-assessment subunit is set up to filter the acquired data in real time based on preset multi-dimensional indicators such as action consistency and environmental recognition, and remove invalid data such as sensor fault data and ambiguous action data to reduce redundant storage pressure. The core indicators for evaluating the dynamic complexity of the work scenario are consistent with those of the time-series collaboration module. The quantitative results are obtained synchronously and combined with the priority of remote control commands to form a dual decision-making basis for adjusting the sensor acquisition frequency, ensuring that the frequency adjustment is accurately matched with the scenario requirements. The effective data is standardized and its features are initially extracted to generate structured data blocks, which are then transmitted to the transmission module.

[0044] The motion deviation is obtained by calculating the Euclidean distance between the actual angle and the desired angle of each joint of the robot, and the spatial distance between the actual position of the end effector and the target position. The deviation calculation frequency is kept consistent with the sensor acquisition frequency to ensure the real-time nature of the deviation feedback.

[0045] The transmission module receives structured data blocks output by the data processing module and command transmission requirements from the timing coordination module. It adopts an edge computing architecture, deploys lightweight data transmission nodes locally on the robot, and pushes some data preprocessing tasks to the edge, shortening the data transmission path and realizing edge preprocessing and low-latency transmission of remote control commands and collected data. At the transmission layer, a lightweight transmission protocol is adopted, redundant transmission fields are eliminated, and a high-efficiency data compression algorithm is used to reduce bandwidth consumption, ensuring that the transmission delay of remote control commands does not exceed 50ms, thus meeting the requirements of real-time control. At the verification layer, a two-way intelligent verification mechanism is constructed to perform dual verification of the integrity and validity of the issued remote control commands. Hash verification technology is used to identify lost or erroneous commands to avoid robot malfunctions. The uploaded collected data is subjected to secondary quality verification, and the data credibility is comprehensively judged in combination with sensor working status data. A transmission status monitoring subunit is set up to monitor key indicators such as link bandwidth and latency in real time. When a transmission anomaly is detected, it automatically switches to the backup transmission link to form a transmission redundancy guarantee. The verified data and instructions are transmitted to the evaluation module and the driver module respectively to ensure the stability and reliability of the entire data transmission link.

[0046] The evaluation module receives the verified multi-source collected data transmitted by the transmission module, and uses a cross-modal attention mechanism to deeply mine the correlation features between different modal data, so as to achieve efficient fusion of multimodal data and generate a unified robot operation state description vector. This vector can accurately characterize the robot's action posture, environmental interaction force and surrounding environment features. The fused data is further denoised using a deep learning model to filter out noise caused by environmental interference, while extracting key features to enhance the data's representation capabilities. Establish a multi-dimensional data quality assessment model to quantitatively correlate data quality with the robot's actual control effect (such as motion accuracy and task success rate), output quality grading results of three levels: excellent, qualified, and poor, and accurately label the reasons for poor data, such as timing deviation, environmental interference, and non-standard actions. Transmit the excellent and qualified data and quality assessment results to the drive module and optimization module.

[0047] The driving module receives the operation status description vector and data quality evaluation results output by the evaluation module, and also receives the instruction conversion results from the timing coordination module to construct a dynamic weight allocation model, thereby realizing the real-time and accurate allocation of weights for remote control and robot autonomous control. A collaborative control model is constructed using deep reinforcement learning algorithms. The control strategy is optimized through extensive training in numerous scenarios. In simple, repetitive tasks, the autonomous control weight is increased to a maximum of 80%, significantly reducing the workload of manual operation. In complex or dynamically unknown scenarios, the remote control weight is increased to no less than 60%, ensuring the safety and accuracy of operations. At the same time, a model predictive control algorithm is introduced to optimize the robot joint control parameters based on the prediction results of the short-term future operation state, effectively suppressing motion jitter and ensuring smooth and accurate movements.

[0048] A real-time motion deviation correction subunit is set up. By frequently comparing the deviation between the actual working state and the expected state, an adaptive correction command is quickly generated and fed back to the execution end. The correction response time is no more than 10ms, which further improves the control accuracy. At the same time, the control effect data is fed back to the optimization module.

[0049] The optimization module receives high-quality data and quality labeling results output by the evaluation module and control effect data fed back by the driving module. It generates data labels through an automated labeling algorithm and stores them in a structured training dataset for incremental training of the robot motion model, thus avoiding the waste of resources caused by full training. The optimized model parameters obtained through training are fed back to the temporal coordination module in real time to optimize the adaptation rules and temporal synchronization threshold, improve the accuracy of action transition and coordination synchronization, and feed back to the driving module to update the control weights and model prediction control parameters, optimize the coordination control effect, and realize the dynamic iteration of the control strategy. The system performs in-depth analysis of the reasons for the poor quality of the labeled data, and generates targeted suggestions for adjusting the acquisition parameters based on the scene characteristics. For example, it adjusts the sensor acquisition frequency and optimizes the timing synchronization threshold. The suggestions are then fed back to the data processing module and the timing coordination module for parameter updates. Set up a closed-loop effect evaluation sub-unit, regularly collect statistics on core indicators such as operation success rate, action accuracy, and system response speed, establish an evaluation model, and dynamically optimize the closed-loop iteration cycle and training parameters to ensure the efficiency and pertinence of closed-loop optimization.

[0050] The fault-tolerant adaptive module monitors the operating status of the aforementioned modules, including key indicators such as transmission link bandwidth, latency, sensor operating status, computing power usage of each module, and robot motion execution accuracy. It constructs a multi-dimensional and comprehensive status monitoring matrix to achieve visualized perception of the system's operating status. It adopts a fault prediction model based on machine learning, and by analyzing historical operating data and real-time status data, it provides early warnings of potential faults such as sensor attenuation, link congestion, and insufficient computing power, providing maintenance personnel with sufficient time to handle the situation. When a module failure or performance degradation is detected, a tiered fault tolerance strategy is automatically triggered. In the case of a minor failure, an adjustment command is sent to the data processing module to reduce the frequency of non-critical data acquisition and activate redundant sensors to ensure the normal operation of core functions. In the case of a serious failure, a switching command is sent to the drive module to switch to a safety control mode, stop unnecessary operations, and promptly provide feedback for manual intervention to prevent the failure from escalating. The fault data and fault tolerance adjustment process are recorded in detail, and a standardized fault analysis report is generated.

[0051] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0052] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A remote-controlled data acquisition and data closed-loop system for an embodied intelligent humanoid robot, characterized in that, It includes a timing coordination module, a data processing module, a transmission module, an evaluation module, a driver module, an optimization module, and a fault-tolerant adaptive module; Timing Coordination Module: Employing multimodal command fusion and robot dynamics identification and adaptation technology, a three-dimensional timing synchronization mechanism for commands, actions, and data acquisition is established to achieve timing alignment of remote command conversion and transmission, action execution, and data acquisition, and to synchronize timing synchronization signals to the data processing module. Data processing module: Receives timing synchronization signals, constructs a multi-source heterogeneous data acquisition network based on multiple types of sensors, achieves synchronous data acquisition through spatiotemporal alignment and dynamic frequency adjustment, filters valid data after quality pre-assessment and standardizes the data, and outputs structured data to the transmission module; Transmission module: The transmission link is built based on the edge computing architecture. It verifies the validity of instructions and data through a two-way intelligent verification mechanism, monitors the link status in real time, and realizes redundancy switching. Evaluation module: Receives verified multi-source data, uses a cross-modal fusion mechanism to mine data correlation features to generate job status description information, optimizes data quality and establishes a quantitative correlation between data quality and control effectiveness, and outputs graded data and evaluation results; Drive module: Based on the operation status information and data quality assessment results, a dynamic weight allocation model is established. Reinforcement learning and predictive control algorithms are used to optimize the control strategy, realize the adaptation of remote operation and autonomous control, improve control accuracy through real-time deviation correction, and feed back control effect data to the optimization module. Optimization module: Completes incremental training of the model based on high-quality data, feeds back the optimization parameters to the time series collaboration module and the driving module, analyzes the causes of poor-quality data and outputs suggestions for adjusting the collected parameters, and dynamically optimizes the closed-loop operation parameters; Fault-tolerant adaptive module: It uses a fault prediction model to realize early warning of potential faults, triggers a graded fault tolerance strategy based on the fault level, records fault data and fault tolerance adjustment process and generates analysis reports.

2. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 1, characterized in that, The timing coordination module receives multimodal commands output by the operator through various types of remote control terminals, and uses a multimodal command fusion algorithm to complete command noise filtering and semantic parsing, distinguishing between valid commands and interference signals; it generates adaptive motion adaptation rules based on a real-time robot dynamics identification model to realize the conversion of human joint space movements to robot joint space movements, and supports plug-and-play functionality for different remote control terminals. The ternary timing synchronization mechanism adapts to the complexity of the operation scenario by dynamically adjusting the timing synchronization threshold, ensuring the timing alignment of remote control command transmission, robot action execution, and sensor data acquisition.

3. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 2, characterized in that, The data processing module integrates robot body vision sensors, tactile sensors, inertial sensors and environmental sensors, and realizes real-time synchronous acquisition of multimodal data through a spatiotemporal alignment algorithm; By combining the priority of remote control commands with the dynamic complexity assessment results of the operation scenario, the acquisition frequency of each sensor is dynamically adjusted; through the data quality pre-assessment subunit, the acquired data is filtered in real time based on preset multi-dimensional indicators including action consistency and environmental recognition, and invalid data is removed. The effective data is standardized and its features are initially extracted to generate structured data blocks.

4. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 3, characterized in that, The transmission module deploys lightweight data transmission nodes locally on the robot to achieve edge preprocessing and low-latency transmission of remote control commands and collected data. A lightweight transmission protocol combined with a data compression algorithm is adopted to reduce bandwidth consumption; a two-way intelligent verification mechanism is used to complete the dual verification of the integrity and validity of remote control commands, identify command loss or errors, perform secondary quality verification on the uploaded collected data, and comprehensively judge the data credibility by combining sensor working status data. The transmission status monitoring subunit monitors key indicators of the link in real time. In case of an anomaly, it automatically switches to the backup transmission link and transmits the verified data and instructions to the evaluation module and the driver module, respectively.

5. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 4, characterized in that, The evaluation module employs a cross-modal attention mechanism to deeply mine the correlation features between different modal data and generate a unified robot operation state description vector. The fused data is denoised and key features are extracted to enhance the data's representation capabilities. Establish a multi-dimensional data quality assessment model to quantitatively correlate data quality with the actual control effect of the robot, output quality grading results of three levels: excellent, qualified, and poor, and label the reasons for poor data.

6. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 5, characterized in that, The driving module constructs a dynamic weight allocation model based on the operation status description vector and data quality evaluation results output by the evaluation module and the instruction conversion results output by the timing coordination module, so as to realize the real-time allocation of weights for remote control and robot autonomous control. A cooperative control model is constructed using reinforcement learning algorithms, and the control weights are dynamically adjusted according to the complexity of the task scenario; a model predictive control algorithm is introduced to optimize the robot joint control parameters. By correcting the sub-unit in real time by measuring motion deviations, comparing the deviation between the actual working state and the expected state, and generating adaptive correction instructions to feed back to the execution end, the control accuracy is improved.

7. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 6, characterized in that, The optimization module completes incremental training of the robot motion model using high-quality data and feeds back the optimization parameters to the timing coordination module and the drive module. In-depth analysis of the causes of poor-quality data, combined with scene characteristics, generates suggestions for adjusting collection parameters, and feeds back to the corresponding modules; By periodically collecting core operational metrics through the closed-loop performance evaluation sub-unit, an evaluation model is established, and the closed-loop iteration cycle and training parameters are dynamically optimized.

8. The embodied intelligent humanoid robot remote control data acquisition and data closed-loop system according to claim 7, characterized in that, The fault-tolerant adaptive module constructs a multi-dimensional status monitoring matrix to monitor key indicators in real time, including link status, sensor working status, module computing power usage, and action execution accuracy; and uses a fault prediction model to achieve early warning of potential faults. A graded fault tolerance strategy is triggered based on the fault level. Redundant sensors are activated for minor faults, and the system switches to safety control mode for severe faults. Record fault data and fault tolerance adjustment processes to generate standardized fault analysis reports.

Images