Explosion-proof humanoid robot 5G cross-domain low-delay remote control system and method

Abstract

The invention belongs to the technical field of robot remote control, and particularly relates to an anti-explosion humanoid robot 5G cross-domain low-delay remote control system and method. The system comprises a master-end teleoperation intelligent cabin and a slave-end remote anti-explosion humanoid robot which perform information interaction through a 5G network; the system transmits a video stream through H.264 coding and a WebRTC protocol, sensor data are packaged through JSON, and human shape control and chassis control information are independently processed. According to the invention, accurate remote force feedback and control can be realized, the time delay of human shape control is within 100ms, the time delay of chassis control is within 150ms, the problems of stability and time delay of remote cross-domain control are effectively solved, and the personal safety of operators in a dangerous working environment is guaranteed.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote control of robots, and particularly relates to a 5G cross-domain low-latency remote control system and method for an explosion-proof humanoid robot. BACKGROUND

[0002] 5G technology, especially its ultra-reliable low-latency communication (URLLC) feature, is opening up a new path for cross-domain low-latency remote control. At present, this technology has been piloted and applied in core scenarios such as industrial automation (such as remote control of port gantry cranes and mine trucks), remote surgery, precise load control of power grids, and intelligent transportation vehicle-road cooperation. The core lies in using the millisecond-level (theoretically up to 1 ms) end-to-end latency, ultra-high reliability and wide-area coverage capability of the 5G network to efficiently connect control terminals (such as operation cabins) and execution terminals (such as mechanical arms, drones and vehicles) deployed in different regions (possibly across cities, provinces or even national boundaries), and to realize real-time and accurate human-machine interaction or automatic control of remote devices. This represents a major evolution of remote operation technology from traditional dedicated wired networks or locally limited wireless networks to wide-area flexible deployment based on public mobile communications.

[0003] The current level of robot intelligence is still limited and cannot match human intelligence. Therefore, in the field of dangerous operations (such as bomb disposal, aerospace and nuclear power), it is of great significance to explore how to integrate human intelligence into robots for remote control. This human-robot collaborative mode can effectively protect personnel safety and fully utilize human real-time decision-making capabilities, and has significant application value. Although the traditional wired teleoperation connection method is stable, the network cable restricts the flexibility of the device and deployment is relatively cumbersome. Using 5G technology to replace wired connection can significantly improve operation flexibility and deployment convenience.

[0004] The existing technology has the following problems: the traditional wired teleoperation method has poor flexibility, is cumbersome to deploy, and cannot meet the cross-domain remote control requirements; the existing wireless teleoperation system has high latency and cannot achieve real-time and accurate control; the network instability in cross-domain remote control causes serious latency jitter and packet loss problems; there is a lack of effective force feedback mechanism, and the operator cannot obtain real tactile feedback; the system stability and reliability need to be improved, and it is difficult to adapt to harsh communication environments. SUMMARY

[0005] The present application aims to solve the problems of high latency, poor stability and lack of effective force feedback in the prior art, and to achieve safe and reliable remote control in dangerous operating environments.

[0006] Technical solution: To achieve the above-mentioned purpose, the explosion-proof humanoid robot 5G cross-domain low-latency remote control system provided by the application comprises: The master end teleoperation intelligent cockpit is used for remote sending of control instructions and receiving of feedback information by the operator. The slave end remote explosion-proof humanoid robot comprises a mobile chassis and a humanoid body and is used for executing the control instructions sent by the master end teleoperation intelligent cockpit. The communication system based on the 5G network is used for realizing 5G communication between the master end teleoperation intelligent cockpit and the slave end remote explosion-proof humanoid robot. The humanoid body and the mobile chassis adopt a separated architecture in hardware, and the respective control instruction information is transmitted through independent 5G communication channels; the communication system adopts protocol optimization and a weak network resistance algorithm based on machine learning; the protocol optimization comprises using a UDP-based custom lightweight protocol to replace TCP, realizing a retransmission mechanism based on sequence numbers and selective acknowledgement for key data in the application layer, compressing the IP / UDP packet header through the ROHC algorithm, and embedding a CRC32 check code in the application layer data packet for end-to-end checking; the weak network resistance algorithm predicts network congestion and channel state through a machine learning algorithm, and dynamically adjusts the modulation mode, power and frequency band.

[0007] The explosion-proof humanoid robot 5G cross-domain low-latency remote control method based on the above-mentioned system comprises the following steps: S1: Establishing a master-slave end communication connection and control architecture: The teleoperation intelligent cockpit is built at the master end, and the master end intelligent terminal is set; the remote explosion-proof humanoid robot is built at the slave end, and the humanoid robot intelligent terminal and the mobile chassis intelligent terminal are set; The communication connection based on the 5G network is established, and a separated control architecture is adopted to separate the humanoid robot control and the mobile chassis control, wherein the master end intelligent terminal and the humanoid robot intelligent terminal communicate through a 5G router, and the master end intelligent terminal and the mobile chassis intelligent terminal communicate through a 5G module; S2: Executing chassis movement control: The master end intelligent terminal converts the operation data of the driving control device into linear speed and angular speed information in real time, and transmits the information to the mobile chassis intelligent terminal through a 5G communication channel, so that the mobile chassis intelligent terminal controls the movement and steering of the mobile chassis; S3: Executing humanoid robot teleoperation control: The master end intelligent terminal acquires the joint motor position, torque and gripper control information of the remote teleoperation device in real time, and transmits the information to the slave end humanoid robot through a 5G communication channel, so as to realize synchronous motion control of the double arms and the gripper; S4: Transmitting video and sensor data: The humanoid robot intelligent terminal collects video streams through multiple cameras and transmits them to the host intelligent terminal in real time for decoding and display; sensor data including odometer, IMU inertial measurement unit, and laser radar are packaged in JSON format and transmitted to the host intelligent terminal in real time through a 5G communication channel for environmental information analysis; S5: Implement force feedback processing: The humanoid robot intelligent terminal obtains humanoid joint motor position, torque, and six-axis force sensor information in real time, applies a low-pass filter to eliminate high-frequency noise, calculates force feedback results through a bilateral control algorithm, transmits them to the host intelligent terminal through a 5G communication channel, and provides force feedback to the operator through the remote teleoperation device, to realize master-slave force feedback closed-loop operation. S6: Execute protocol and resource scheduling optimization: Optimize the transmission protocol at the protocol layer, including using a custom lightweight protocol based on UDP to replace TCP, implementing a retransmission mechanism based on sequence numbers and selective acknowledgment at the application layer for key data, using the ROHC algorithm to compress the IP / UDP packet header, and embedding a CRC32 check code in the application layer data packet for end-to-end verification to reduce transmission overhead and latency; At the resource scheduling and management layer, introduce an anti-weak network algorithm based on machine learning to predict network congestion and channel state, dynamically adjust modulation mode, power, and frequency band to balance latency and energy efficiency.

[0008] Advantages: Compared with the prior art, the advantages of the present application are: the present application realizes true cross-domain low-latency remote control, with humanoid control latency within 100ms and chassis control latency within 150ms; provides a complete force feedback mechanism, allowing operators to obtain real tactile perception; uses a separate control architecture to improve system stability and reliability; through protocol optimization and intelligent resource scheduling, significantly reduces communication latency and bandwidth occupancy; suitable for dangerous operating environments, effectively ensuring personnel safety. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 The overall architecture diagram of the 5G cross-domain low-latency remote control system for the explosion-proof humanoid robot.

[0010] Figure 2 The functional software interface diagram of the remote cockpit, showing the keys and display area of the operation platform.

[0011] Figure 3 The 5G cross-domain low-latency remote control protocol flowchart for the explosion-proof humanoid robot, showing the transmission and processing process of data between the host and slave terminals.

[0012] Figure 1Remote explosion-proof humanoid robot 20, 5G base station 30. DETAILED DESCRIPTION

[0013] The technical solutions of the present application will be described in detail below with reference to the drawings, but the protection scope of the present application is not limited to the described embodiments.

[0014] Embodiment 1: As shown in the figure, the explosion-proof humanoid robot 5G cross-domain low-latency remote control system of the present application includes a master end remote operation intelligent cockpit 10, a slave end remote explosion-proof humanoid robot 20, and a 5G base station 30. Figure 1

[0015] The specific configuration of the master end remote operation intelligent cockpit 10 is as follows: Remote driving control cabin 101: integrates all control interfaces and human-computer interaction functions, has a seat with ergonomic design, and provides a comfortable operating environment for the operator; an operating table is integrated in front of the leg of the cockpit, the operating table is provided with a driving control device including an automobile level accelerator pedal, a brake pedal and a steering wheel; a display is erected on the operating table, which adopts high-resolution multi-screen display to display the slave end multi-channel camera picture in real time; Remote operation device 102: includes a multi-degree-of-freedom mechanical arm structure, which can accurately simulate human hand movements; Master end intelligent terminal 103: uses an industrial computer with a hardware code independent graphics card, which has powerful data processing capability; Master end 5G router 104: uses an industrial 5G router to ensure the stability and reliability of communication.

[0016] The specific configuration of the slave end remote explosion-proof humanoid robot is as follows: Mobile chassis 201: uses a four-wheel drive wheeled chassis with omnidirectional movement capability; ​The humanoid robot 202 features: a multi-DOF neck (3 degrees of freedom for flexible head rotation); a waist (2 degrees of freedom for forward / backward bending and left / right rotation); hips connecting the waist to the chassis for cushioning; humanoid arms (7 degrees of freedom per arm, mimicking human arm movement); an end effector for grasping objects of various shapes and sizes; a six-dimensional force sensor for sensing forces and torques in the X, Y, and Z directions; a 360-degree scanning lidar for environmental mapping and obstacle detection; an inertial measurement unit (IMU) providing attitude, acceleration, and angular velocity information; an odometry system based on a wheel encoder for precise position measurement; and multiple cameras, including head, chest, and hand cameras, providing a 360-degree field of view. The intelligent terminals include: the humanoid robot intelligent terminal 205 uses NVIDIA Jetson nano, and the mobile chassis intelligent terminal 203 uses Raspberry Pi 4B. The 5G communication module includes: a slave 5G router 206 for 5G communication of a humanoid robot, and a 5G module 204 for 5G communication of a mobile chassis.

[0017] Example 2: As Figure 2 As shown, the process of building the communication system for the remotely operated intelligent cockpit is as follows: Network connection establishment: The master industrial control computer establishes a communication connection with the slave NVIDIA Jetson nano through an industrial-grade 5G router; the chassis Raspberry Pi establishes an independent communication channel with the master through a 5G module.

[0018] QT Interface Development: Developing a remote cockpit visualization interface based on the QT framework, mainly including: Cockpit button panel: Displays various control buttons and status indicator lights; Functional interface area: Includes real-time display of sensor information and remote chassis fault clearing function; Video window area: Displays real-time footage from multiple cameras on the slave end.

[0019] Client program: Reads information from external sensors, including the status information of menu buttons, accelerator pedal, brake pedal and steering wheel.

[0020] Functional modules: Enable auxiliary functions such as video window refresh, device restart, and fault diagnosis.

[0021] Example 3: A 5G cross-domain low-latency remote control method for an explosion-proof humanoid robot based on the system provided in Example 1, comprising the following steps: Step one: build a master end teleoperation intelligent cockpit and a slave end remote explosion-proof humanoid robot based on 5G cross-domain low latency: The whole control system of the application is composed of a master end teleoperation intelligent cockpit and a slave end remote explosion-proof humanoid robot, as shown in Figure 1 The master end teleoperation intelligent cockpit is the control end, including a seat, a remote teleoperation device, a driving control device (throttle / brake / steering wheel / intelligent cockpit), a display, an operation table, a master end intelligent terminal and a master end 5G router and the like. The slave end remote explosion-proof humanoid robot is the controlled end, including a mobile chassis, a humanoid body (humanoid multi-degree-of-freedom neck, humanoid waist, humanoid hip), a humanoid arm, an end gripper, a six-dimensional force sensor, a laser radar, an IMU, an odometer, a plurality of cameras, a slave end intelligent terminal, a 5G module, a slave end 5G router and the like.

[0022] Step two: build a remote information interaction and communication control system of the teleoperation intelligent cockpit. The intelligent terminal of the teleoperation intelligent cockpit is an industrial computer with a hardware code independent graphics card, and the humanoid robot intelligent terminal is a Nvidia Jetson nano. Both of them communicate and interact with each other through an industrial-grade 5G router. The mobile chassis intelligent terminal is a Raspberry Pi, which communicates and interacts with the teleoperation intelligent cockpit through a 5G module.

[0023] The Jetson Nano is responsible for voice, video and sensor data transmission in the slave end humanoid robot, such as video streaming service, and does not involve upper body joint control. The Raspberry Pi is used for communication and video streaming with the remote cockpit, and there is no business overlap between the Jetson Nano and the Raspberry Pi. The master end industrial control computer is responsible for real-time calculation and data transmission of teleoperation and force feedback.

[0024] A remote driving cockpit visualization interface is built through QT, and its function interface diagram is shown in Figure 2 The main components include a cockpit button panel, a function interface area and a video window area. The button panel is based on the QT interface program, and the client reads external sensor information (menu buttons, throttle, steering wheel). The function interaction interface includes real-time status display of external sensor information, remote chassis fault clearing, video window refreshing and device restarting.

[0025] Step three: when the master operator controls the driving in the cockpit, the driving cockpit enters the remote control mode, real-time calculation of current speed conversion data is performed, and linear speed / angular velocity information is sent to the Raspberry Pi on the remote mobile chassis, which is forwarded to the chassis controller through the 485 protocol to control the mobile chassis for movement and steering control.

[0026] Step four: the remote explosion-proof humanoid robot will obtain multiple camera video stream information through h.264 encoding and Webrtc push stream protocol, and push the video stream to the remote operation intelligent cockpit. The web end automatically decodes the video stream in real time and displays it. Other sensor data information including odometer, IMU, and laser radar is packaged by json and transmitted in real time to the master terminal through the 5G module to analyze and judge the environment information.

[0027] Among them, the robot joint data and sensor data are preferentially pushed, and the robot body parameter data is preferentially pushed than the video stream. In the application, different data types are marked with different quality of service (QoS) marks. For example, the DSCP (differentiated service code point) field of the IP header is used; adaptive code rate adjustment is adopted, and bandwidth, packet loss rate, etc. are monitored in real time. If the network is poor, the video stream quality will be automatically reduced to ensure the real-time performance and fidelity of the robot body parameters; robot parameters and sensor data are transmitted by UDP, and video stream is based on TCP.

[0028] Step five: when the master operator ends the remote control of the robot chassis and starts to remotely operate the remote control device, the intelligent terminal obtains the position torque and gripper control information of each joint motor of the remote control device in real time, and transmits the data to the remote humanoid robot through the 5G router for control.

[0029] Step six: the humanoid robot intelligent terminal calculates the real-time position torque of each humanoid joint motor and six-axis force sensor information in real time, and transmits the force feedback result to the master operator in real time through the 5G module for feeling, analysis and judgment, and performs the next real-time control adjustment operation of the humanoid robot.

[0030] Master-slave force feedback mechanism: the operator manipulates the master (7-degree-of-freedom force feedback input device), and the slave (a 7-degree-of-freedom robot) completely reproduces the motion of the master at the remote end; at the same time, the force generated by the interaction between the slave and the environment is accurately fed back to the master, so that the operator has a "first-hand" tactile feeling.

[0031] As shown in Figure 3 , the operator moves the master device, and the high-precision sensor of the master measures the angles of the 7 joints and the pose of the end effector in real time; through high-bandwidth closed-loop control, the master motion command is mapped to the slave to reproduce the motion in real time, and the slave force sensor information is mapped back to the master to generate feedback force, forming a position-force coupled bilateral control; based on model inverse solution and impedance adjustment algorithm, the model difference between master and slave is processed; through state observer and feedforward dynamic compensation, transmission delay and dynamic disturbance are offset, ensuring the transparency and stability of the operation.

[0032] The application adopts a bilateral control architecture based on position-force, and is specifically implemented as: Forward channel: master end pose Directly as slave end desired pose ; Reverse channel: slave end force measurement Directly as master end feedback force ; Model inverse solution: adopt inverse kinematics (IK) algorithm to solve the pose instruction Real-time into slave end joint angle instruction ; Impedance adjustment: adjust the damping and stiffness parameters in the impedance model , so that the slave end presents different compliance characteristics, thereby processing the difference between the dynamic models of the master and slave ends.

[0033] A Luenberger observer is adopted, and its mathematical model is: Wherein, is the estimated state vector, is the derivative of the estimated state, A, B and C are system matrix, input matrix and output matrix respectively, is the delayed control input, is the delayed measurement output, and L is the observer gain matrix; the observer uses the delayed control instruction and sensor data , continuously corrects the estimated state through the gain matrix L, so as to estimate the real-time speed and contact force and other states of the slave end without delay.

[0034] Feedforward dynamic compensation: the compensation term is , Wherein, is the feedforward compensation torque, is the joint position vector, is the joint speed vector, , is the expected joint acceleration and speed, is the inertia matrix, is the Coriolis and centrifugal force matrix, is the gravity vector, and the calculated compensation term is used to compensate the inertia force , gravity , Coriolis force and centrifugal force of the humanoid robot itself.The inner dynamics effect is reduced, thereby significantly reducing the burden of feedback control and improving system transparency and stability.

[0035] Step seven: To solve the problem of remote cross-domain control stability, the upper body control of the humanoid robot in the control system of the application is based on a humanoid controller + Raspberry Pi mode, and the chassis is controlled by a chassis controller + Raspberry Pi mode, which is hardware isolated and does not affect the data interaction of each other; the control information of the humanoid robot and the mobile chassis is encoded and decoded separately, and the data interaction and control operation do not affect each other.

[0036] Step eight: To ensure the correctness of the force feedback result and not affect the normal operation of the master end teleoperation staff, the application uses low-pass filtering to determine the appropriate threshold to eliminate the influence of small changes, so as to improve the stability and effectiveness of the entire force feedback process.

[0037] The low-pass filtering in the force feedback control mainly includes the cutoff frequency and the filtering algorithm: the core of the joint module noise is the noise frequency. The cutoff frequency must be higher than the effective frequency of the expected force signal (usually from contact or human operation, generally lower than 20-30Hz), but much lower than the high-frequency electronic noise (usually >100Hz), to balance between fidelity (preserve the true force signal) and noise reduction, and the specific value is adjusted according to the actual working condition; the cutoff frequency of the dynamics bandwidth cannot exceed the effective force feedback bandwidth of the master / slave robot. The frequency of the filtered force signal is higher than the frequency that the robot can respond to, which will cause the system to be unstable (such as generating oscillation).

[0038] Step nine: To solve the problem of remote cross-domain time delay, the application optimizes the protocol and transmission layer, trims the TCP / IP redundant fields, adopts a lightweight protocol (a custom reliable transmission protocol based on UDP), reduces the data packet encapsulation and decapsulation overhead, and compresses the IP / UDP packet.

[0039] Specifically, the application has made the following targeted optimization on the transmission protocol: (1) protocol selection: adopt a lightweight custom protocol based on UDP to replace TCP to eliminate the delay caused by handshake, retransmission and congestion control; (2) custom reliability: implement a lightweight retransmission mechanism based on sequence number and selective acknowledgement (SACK) for critical data (such as control instructions) in the application layer; (3) packet header compression: use the ROHC algorithm to compress the 40-byte IP / UDP packet header in a continuous stream to typically 1-4 bytes, greatly reducing the transmission overhead; (4) 5G network adaptation: adapt to the high bandwidth and jitter characteristics of 5G network through forward error correction (FEC) and delay-based congestion control algorithm.

[0040] By the above technical means, the redundant data transmission is reduced, the protocol stack is simplified, and the time delay is reduced. The standard IP / UDP header is 28 bytes, and after ROHC compression, the typical value is 3-4 bytes. Taking the conservative value 4 bytes as an example: the overhead reduction value is 28-4=24 bytes; the overhead reduction ratio is (24 / 28) x 100% approximately 85.7%; considering the connection establishment, non-ideal compression package and other factors, the redundant data transmission is reduced, the overhead is reduced by 60%, the protocol stack is simplified, and the time delay is reduced.

[0041] Directly removing the TCP checksum will completely lose the bit error detection capability of the transport layer. In complex networks such as 5G / WAN, although the underlying link reliability is already very high, there is still a very low probability of bit flipping. Undetected errors can cause robots to receive incorrect instructions or state data, which may cause safety and stability risks.

[0042] The compensation method is to embed an application layer end-to-end check in the header of all key data packets (such as motion instructions and force feedback values).

[0043] RC32 and other efficient check codes; selective retransmission: when the application layer check fails, the receiver will request the sender to retransmit the specific key data packet; data priority differentiation: for very high frequency and low priority sensor data, on the premise that the underlying link quality is very high, a small amount of reliability can be sacrificed to achieve very low delay, but core control instructions must always have strong verification.

[0044] Step ten: to solve the problem of remote cross-domain time delay, the resource scheduling and management layer is optimized, the traditional weak network resistance algorithm is optimized, and a machine learning algorithm is introduced to dynamically adjust and balance the time delay and energy efficiency. Machine learning is used to predict network congestion and channel state, dynamically adjust modulation mode, power and frequency band, and balance time delay and energy efficiency.

[0045] The machine learning algorithm used in the application is a method combining online supervised learning and deep reinforcement learning (DRL).

[0046] Online supervised learning: used to build a real-time network state prediction model, which takes historical network parameter sequences as input and outputs predictions of future short-term network states (such as time delay and bandwidth).

[0047] Deep reinforcement learning (DRL): the core algorithm is modeled as a Markov decision process (MDP); Agent: resource scheduler; State: real-time and predicted network state (such as time delay, packet loss rate, bandwidth, signal strength) Action: Communication parameters to be adjusted (e.g., switching frequency bands, adjusting transmit power, changing Modulation and Coding Scheme, MCS) Reward: The goal is to minimize latency, maintain reliability (reduce packet loss rate), and penalize high energy consumption operations. The algorithm learns through constant trial and error in specific network conditions.

[0048] The robot system adopts a hybrid mode of "pre-training + online fine-tuning": Pre-trained model: Offline pre-training of the model in a dataset and simulation environment, resulting in a basic model.

[0049] Online learning and fine-tuning: The model continuously fine-tunes based on real-time network data.

[0050] Fine-tuning adjusts the internal parameters of the control algorithm (such as the weights of the neural network), allowing the output actions to adapt to real dynamics (such as different friction, inertia) and real sensor noise (such as image blur, force sensation delay). This is usually done quickly through a small amount of online demonstration or real-time reward signals.

[0051] Dynamic parameter tuning: Adjusting the parameters of the underlying controller, such as PID gain, stiffness and damping in impedance control. The goal is to make the robot's "feel" more suitable for the current task, such as increasing sensitivity for delicate operations or increasing stability for heavy assembly.

[0052] Online trajectory correction: Dynamically adjusting the predetermined motion trajectory based on real-time sensor feedback (especially force sensation). For example, feeling a jam when inserting, fine-tune the end position and attitude to align the hole; feeling a larger contact force, automatically reducing the motion amplitude to avoid damaging the object.

[0053] The key parameters input to the real-time prediction model mainly include: Latency: Including one-way delay and round-trip time (RTT), is the core prediction target; Packet Loss Rate: A key indicator of channel reliability; Bandwidth: Estimate of available uplink and downlink bandwidth; Jitter: The rate of change of latency, affecting the stability of transmission; Signal Strength (RSRP / RSRQ): In 5G / 4G networks, this is a basic physical layer parameter that determines connection quality.

[0054] These parameters are combined into a time series as input to the prediction model and reinforcement learning state space.

[0055] Based on the output of Markov decision, the robot system dynamically adjusts the following communication parameters, and the mechanism is as follows: Modulation mechanism: When the predicted network quality is good (high signal-to-noise ratio), high-order modulation (such as 1024-QAM) and efficient coding are selected to improve spectral efficiency and transmit more data in the same time, thereby reducing latency. When the predicted network quality is poor, it is automatically downgraded to low-order modulation (such as QPSK) and redundant coding, sacrificing speed for reliability to avoid greater latency caused by retransmission.

[0056] Power mechanism: Power optimization is performed under the premise of ensuring connection reliability. When the link quality is expected to be good, the transmission power is intelligently reduced to directly save device energy consumption; when the signal attenuation or interference is predicted to increase, the power is moderately increased in advance to resist quality deterioration and avoid the huge latency overhead caused by connection interruption.

[0057] Network band mechanism: The machine learning algorithm learns the historical performance of each frequency band (such as 5G C-Band vs. millimeter wave) in different scenarios. For example, when it is predicted that the user moves to the millimeter wave coverage area, the millimeter wave frequency band with high speed is preferentially selected to extremely reduce the latency; when it is predicted that it moves out of the coverage area, it is switched to a low-frequency band with wider coverage in advance to prevent connection interruption and ensure the continuity of the control loop.

[0058] Step eleven: The invention of the 5G cross-domain low-latency remote control system and method of the explosion-proof humanoid robot can adapt to scenarios such as short-term network instability causing latency jitter, network packet loss, etc.

[0059] In the actual 5G / 4G network and Internet environment, multiple typical application scenarios are selected to cover indoor and outdoor environments, including network switching and signal fluctuations; timestamps are stamped between the master control instruction input point and the slave execution mechanism response point (or delay measurement tools are used on closed-loop video streams), and the total delay from the "brain" to the "limb" is directly measured. The test results show that the humanoid control and force feedback latency is within 100ms, and the chassis control and camera sensor latency is within 150ms. Under this latency, the master operator almost feels no control delay, which can maximize the safety and stability of remote driving and remote operation.

[0060] The system realizes remote master-slave force feedback teleoperation control. The core mechanism is that the operator controls the slave end (including a mobile chassis and a humanoid robot) through the master end (including a teleoperation intelligent cockpit). The master controls the slave: the motion instructions (such as position and attitude) of the master end component are collected and sent to the slave end controller in real time, and the slave end mechanical arm is driven to realize synchronous motion. Slave end force feedback: when performing a task, the torque / force information generated by the contact between the slave end humanoid robot and the environment is perceived and fed back to the master end force feedback device. Operator tactile perception: the feedback torque / force acts on the operator's hand, allowing the operator to truly perceive the interaction force between the slave end and the environment. Precise closed-loop operation: this force feedback helps the operator make more accurate adjustments, and through continued operation of the master end control arm, the slave end completes the task with fine control.

[0061] An operation table is arranged in the front leg area of the cockpit. This operation table integrates key control interfaces: a mobile chassis control steering wheel: directly connected to the operation table, the operator can control the steering of the mobile chassis through this steering wheel. Master-slave control throttle and brake pedals: arranged on the operation table surface. The operator can control the acceleration and deceleration of the remote humanoid robot chassis through the throttle pedal to complete large-scale rapid movement. The operator can stop the chassis motion by stepping on the brake pedal. During system control, the operator can naturally adjust the sitting posture or operation posture (for example: stretch the legs, change the arm position) to prepare for subsequent teleoperation. This design makes human-machine interaction more flexible: the operator can directly control the chassis without getting up or leaving the seat, and can quickly and conveniently disconnect the master-slave cooperation to adjust the posture when needed.

[0062] There is a display screen in front of the cockpit to real-time return the environment information of the slave end to the master end display screen device for display, such as the information of multiple cameras and various sensors of the remote humanoid robot. Real-time acquisition of remote real environment images and humanoid robot information facilitates precise operation by the operator. The mobile chassis is a wheeled mobile chassis, which greatly reduces the energy consumption problem of the slave end device during movement and realizes rapid movement, improving the efficiency of remote operation of the device.

[0063] 5G mobile communication technology as today's mainstream advanced communication standards, is a comprehensive iteration and major leap of 4G technology. It aims to build a new generation of communication network with ultra-high speed, extreme response, green efficiency, super large capacity and can support massive device interconnection, mainly with a large number of core advantages. Ultra-high transmission rate. 5G can provide several times or even tens of times the peak rate of 4G, and the theoretical download speed can reach several Gbps level, so that the super high definition video stream and the large file transmission are close to instantaneous completion, which greatly improves the user experience. Ultra-low network delay: the air interface delay can be reduced to milliseconds (ms) level, which significantly improves the real-time interaction, and provides key support for remote surgery, industrial automation control, automatic driving, cloud gaming and other application scenarios with extremely high response speed. Greatly optimize the energy efficiency. Compared with the previous generation technology, 5G can significantly reduce device power consumption and prolong terminal battery life through more intelligent signaling mechanism, network slicing technology and optimized design of base station end, while providing higher performance, which embodies stronger environmental sustainability. Ultra-large network bandwidth. By using higher frequency spectrum resources (such as millimeter wave) and advanced modulation technology, 5G has a bandwidth capacity far exceeding 4G, which can simultaneously support more high data flow applications and meet the high bandwidth demand of user-intensive areas. Massive device concurrent access. One of the design goals of 5G network is to realize Internet of Things (IoT), which can support a large number of terminal devices connected to a single cell, with a connection density of up to million / km², laying a solid connection foundation for building large-scale smart cities, smart homes, sensor networks and other applications.

[0064] The weak network resistance algorithm is a communication optimization technology system designed for unstable network (high delay, packet loss, jitter) scenarios, and its core goal is to maintain the continuity, stability and low delay of communication through a dynamic adaptation mechanism. The essence is to build reliability on an unreliable channel, and the remote cross-domain low delay control has higher requirements for the weak network resistance algorithm. The application introduces a machine learning algorithm to predict the network state, intelligently and real-time adjust the key communication parameters (modulation mode, power, frequency band), so as to optimize the balance between delay and energy efficiency in harsh environment. Through data collection and environment perception, machine learning prediction modeling, intelligent resource optimization decision and other steps, the optimization of the whole weak network resistance algorithm is completed.

[0065] The application simplifies the protocol stack by cutting the TCP / IP redundant field. The bandwidth occupation can be reduced, and the bandwidth cost in the mobile network can be greatly saved. Removing the redundant data such as TCP checksum optional field can improve the payload ratio, reduce transmission delay, optimize device resource consumption, enhance weak network stability, effectively reduce the delay of cross-domain communication, and greatly improve the communication efficiency.

[0066] While the application has been described and illustrated with reference to specific preferred embodiments, it is not intended that it be limited to these particulars. Various changes in form and detail can be made without departing from the spirit and scope of the application as defined by the appended claims.

Claims

1. An explosion-proof humanoid robot 5G cross-domain low-latency remote control system, characterized in that, Comprise: A master end teleoperation intelligent cockpit for an operator to remotely send control instructions and receive feedback information; A slave end remote explosion-proof humanoid robot comprising a mobile chassis and a humanoid robot for executing the control instructions sent by the master end teleoperation intelligent cockpit and feeding back information; A communication system based on a 5G network for establishing a 5G communication channel between the master end teleoperation intelligent cockpit and the slave end remote explosion-proof humanoid robot to transmit control instructions and feedback information; Wherein, the humanoid robot and the mobile chassis adopt a separate architecture in hardware, and their respective control instructions and feedback information are transmitted through independent 5G communication channels; the communication system adopts protocol optimization and a weak network resistance algorithm based on machine learning, the protocol optimization includes using a UDP-based custom lightweight protocol to replace TCP, implementing a retransmission mechanism based on sequence numbers and selective acknowledgement at the application layer for key data, using an ROHC algorithm to compress IP / UDP packet headers, and embedding CRC32 check codes in application layer data packets for end-to-end checking; the weak network resistance algorithm predicts network congestion and channel state through a machine learning algorithm, and dynamically adjusts modulation mode, power and frequency band.

2. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 1, wherein The master end teleoperation intelligent cockpit comprises a seat for an operator to sit and operate, a remote teleoperation device for controlling the upper limb movements of the slave end remote explosion-proof humanoid robot, a driving control device for controlling the movement and steering of the mobile chassis, a display for displaying environmental information and robot status transmitted by the slave end remote explosion-proof humanoid robot in real time, a master end intelligent terminal for data processing, and a master end 5G router for 5G communication with the slave end intelligent terminal; The slave end remote explosion-proof humanoid robot comprises a four-wheel drive wheeled mobile chassis with omnidirectional movement capability, a humanoid robot with a humanoid body, humanoid arms, and end grippers installed at the ends of the arms, a sensor assembly comprising six-axis force sensors, laser radars, IMU inertial measurement units, and odometers based on wheel encoders installed at the end grippers, multiple cameras installed at the humanoid body and humanoid arms, a slave end intelligent terminal comprising a humanoid robot intelligent terminal and a mobile chassis intelligent terminal, and a 5G communication module comprising a slave end 5G router and a 5G module, wherein the slave end 5G router is used for 5G communication between the humanoid robot intelligent terminal and the master end intelligent terminal, and the 5G module is used for 5G communication between the mobile chassis intelligent terminal and the master end intelligent terminal.

3. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 2, wherein The master intelligent terminal converts the control instructions into linear speed and angular speed information in response to the operation of the driving control device, and transmits the information to the mobile chassis intelligent terminal through a 5G communication channel to control the movement and steering of the mobile chassis; at the same time, the master intelligent terminal obtains the joint motor position torque and gripper control information of the remote teleoperation device, and transmits them to the humanoid robot intelligent terminal through the 5G communication channel to drive the humanoid arms and end gripper to realize synchronous motion. The humanoid robot intelligent terminal obtains the force and torque information collected by the six-dimensional force sensor installed at the end gripper in real time, processes the information through low-pass filtering, offsets the transmission delay and dynamic interference through a bilateral control algorithm combined with a state observer and a feedforward dynamic compensation, transmits the force feedback result to the master intelligent terminal through the 5G communication channel, and provides the force feedback to the operator through the remote teleoperation device to realize master-slave force feedback closed-loop operation.

4. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 3, characterized in that, The bilateral control algorithm includes: taking the master position as the slave expected position through the forward channel, taking the measured force of the slave as the feedback force of the master through the reverse channel, and using the inverse kinematics algorithm for model inverse solution; The impedance model adopted by the inverse kinematics algorithm is: , wherein, is the desired force, , , are the desired inertia matrix, damping matrix, stiffness matrix, respectively, , , are the position error, velocity error, acceleration error, respectively; by adjusting the damping matrix and the stiffness matrix in the impedance model, the slave humanoid robot presents different compliant characteristics to handle the difference in the dynamic model between master and slave. The state observer uses the Luenberger observer, and its mathematical model is: ， wherein is an estimated state vector, is a derivative of the estimated state, A, B, C are system matrix, input matrix, output matrix, respectively, is a delayed control input, is a delayed measurement output, L is an observer gain matrix; The compensation term of the feedforward dynamic compensation is calculated by wherein, is the feedforward compensation torque, is the joint position vector, is the joint velocity vector, , is the desired joint acceleration and velocity, is the inertia matrix, is the Coriolis and centrifugal force matrix, is the gravity vector, the calculated compensation term is used to compensate for the dynamic effects including the inertial forces of the humanoid robot itself , the gravity , the Coriolis forces and the centrifugal forces .

5. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 3, characterized in that, In the separated control architecture, the humanoid robot intelligent terminal uses the Nvidia Jetson Nano processor to control the humanoid robot, the mobile chassis intelligent terminal uses the Raspberry Pi processor to control the mobile chassis, and the two terminals communicate with the master intelligent terminal through the slave 5G router and the 5G module respectively.

6. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 5, characterized in that, The communication system is used to realize cross-domain information interaction between the master remote operation intelligent cockpit and the slave remote explosion-proof humanoid robot, including video stream transmission, sensor data feedback, control instruction transmission and force feedback. Different data types are set with different quality of service marks, wherein the robot joint motor position torque data and sensor data are transmitted through UDP and have priority over video stream transmission based on TCP.

7. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 6, characterized in that, The humanoid robot intelligent terminal transmits multiple camera video streams through H.264 encoding and WebRTC protocol, and transmits sensor data such as odometer, IMU inertial measurement unit and laser radar through JSON packaging in real time. 8.The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 1, characterized in that, The custom lightweight protocol based on UDP eliminates the delay caused by TCP handshake, retransmission and congestion control, realizes a retransmission mechanism based on sequence number and selective acknowledgement at the application layer, wherein when the receiver detects data packet loss, it only requests retransmission of the specific missing data packet; end-to-end verification is performed by embedding CRC32 check code at the application layer, wherein if the receiver fails to verify the key data packet, selective retransmission is triggered.

9. The explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 8, characterized in that, The anti-weak network algorithm combines online supervised learning with deep reinforcement learning, models resource scheduling as a Markov decision process, takes real-time and predicted network state as state input, takes communication parameter adjustment as action output, and takes minimizing delay and maintaining reliability as reward target; the state input includes delay, packet loss rate, bandwidth and signal strength, and the action output includes modulation scheme adjustment, power control and frequency band switching.

10. The method of controlling an explosion-proof humanoid robot 5G cross-domain low-latency remote control system according to claim 2, characterized in that, Comprising the following steps: S1: Establishing master-slave communication connection and control architecture: Building a remote operation intelligent cockpit at the master end and setting up a master intelligent terminal, building a remote explosion-proof humanoid robot at the slave end and setting up a humanoid robot intelligent terminal and a mobile chassis intelligent terminal; Establishing a communication connection based on a 5G network, adopting a separated control architecture to separate the humanoid robot control and the mobile chassis control, wherein the master intelligent terminal and the humanoid robot intelligent terminal communicate through a 5G router, and the master intelligent terminal and the mobile chassis intelligent terminal communicate through a 5G module; S2: Executing chassis movement control: The master intelligent terminal converts the operation data of the driving control device into linear speed and angular speed information in real time, transmits the information to the mobile chassis intelligent terminal through a 5G communication channel, and controls the movement and steering of the mobile chassis by the mobile chassis intelligent terminal; S3: Executing remote operation control of the humanoid robot: The master intelligent terminal acquires the joint motor position, torque and gripper control information of the remote operation device in real time, and transmits the information to the slave humanoid robot through a 5G communication channel to realize synchronous motion control of the double arms and the gripper; S4: Transmitting video and sensor data: The humanoid robot intelligent terminal collects video streams through multiple cameras and transmits them to the master intelligent terminal in real time for decoding and display; sensor data including odometer, IMU inertial measurement unit and laser radar are packaged in JSON format and transmitted to the master intelligent terminal in real time through a 5G communication channel for environment information analysis; S5: Implementing force feedback processing: The humanoid robot intelligent terminal acquires the humanoid joint motor position, torque and six-axis force sensor information in real time, applies a low-pass filter to eliminate high-frequency noise, calculates the force feedback result through a bilateral control algorithm, transmits the result to the master intelligent terminal through a 5G communication channel, and provides force feedback to the operator through the remote operation device to realize master-slave force feedback closed-loop operation; S6: Executing protocol and resource scheduling optimization: Optimizing the transmission protocol at the protocol layer, including adopting a custom lightweight protocol based on UDP to replace TCP, implementing a retransmission mechanism based on sequence number and selective acknowledgement for key data at the application layer, adopting an ROHC algorithm to compress the IP / UDP packet header, and embedding a CRC32 check code in the application layer data packet for end-to-end verification to reduce transmission overhead and delay; At the resource scheduling and management layer, an anti-weak network algorithm based on machine learning is introduced to predict network congestion and channel state, dynamically adjust the modulation mode, power and frequency band to balance the delay and energy efficiency.

Images