Human-machine hybrid control method and system for weak communication environment

Abstract

This application discloses a human-machine hybrid control method and system for weak communication environments, relating to the field of remote human-machine control technology. The method includes: a back-end remote control center sending a first operation command input by an operator to a remote unmanned platform; the remote unmanned platform receiving the first operation command, generating a second operation command based on the current task and perception data, calculating the communication quality with the back-end remote control center, and adjusting its own motion state based on a human-machine hybrid strategy. Specifically: when the communication quality is above an upper threshold, the platform adjusts its own motion state according to the first operation command; when the communication quality is below a lower threshold, the platform adjusts its own motion state according to the second operation command; when the communication quality is between the upper and lower thresholds, the platform merges the first and second operation commands and adjusts its own motion state according to the merged operation command. This application improves the reliability of human-machine systems in weak communication environments.

Description

Technical Field

[0001] This application relates to the field of remote human-machine control technology, and in particular to a human-machine hybrid control method and system for weak communication environments. Background Technology

[0002] With the increasingly widespread application of remote unmanned platforms in complex environments such as earthquake relief and military reconnaissance, reliable control under weak communication conditions has become a critical issue that urgently needs to be addressed. Traditional remote unmanned operation modes can be divided into two types: remote operation and fully autonomous operation.

[0003] Teleoperation control refers to a human-machine interface mode in which an operator controls a remote unmanned platform in real-time or near real-time via a remote communication link. In this mode, the operator generates control commands based on sensor data (such as video, radar, and environmental conditions) transmitted from the remote unmanned platform, combined with their own decision-making abilities. These commands are then sent to the front-end equipment via a communication network to drive the platform to perform corresponding actions. Its core characteristic is the "human-in-the-loop" principle, relying on a stable communication link to achieve precise remote control. It is suitable for scenarios with complex tasks or uncertain environments (such as bomb disposal and sophisticated search and rescue).

[0004] Full Autonomous Control (FEC) refers to an intelligent operating mode in which a remote unmanned platform independently completes environmental understanding, task planning, and action control without real-time human intervention, relying on its onboard perception, decision-making, and execution systems. Its core characteristic is "decentralized decision-making," which uses multimodal sensors (such as LiDAR, visual cameras, and inertial navigation) to collect environmental data in real time, combines this data with pre-set algorithms (such as SLAM, deep learning, and path planning) to achieve dynamic task processing, and ensures robust execution based on a closed-loop feedback system.

[0005] The advantage of teleoperation mode lies in its ability to accurately transmit operator commands to a remote unmanned platform, enabling the platform to execute tasks precisely. However, in complex real-world scenarios, traditional teleoperation mode, relying on real-time data transmission, faces risks such as command delays and interruptions, severely impacting the operator's precise control over the remote unmanned platform. Fully autonomous mode, on the other hand, eliminates the need to consider communication conditions in the actual scenario; the remote unmanned platform can execute tasks through its own perception, planning, and control modules. However, fully autonomous mode may suffer from delayed decision-making or decreased task adaptability due to the inability to obtain remote control commands in a timely manner. Summary of the Invention

[0006] The purpose of this application is to provide a human-machine hybrid control method and system for weak communication environments. It can automatically switch control modes according to communication quality, and achieve a smooth transition between the two modes through the fusion of operation commands, thereby improving the reliability of the human-machine system in weak communication environments.

[0007] The technical solution provided in this application enables the human-machine system to dynamically adjust human-machine control weights based on real-time communication quality. When the communication link is in good condition, the operator remote control mode is prioritized to leverage human decision-making advantages; when communication quality deteriorates, it smoothly transitions to autonomous control mode, maintaining basic task execution capabilities through a local intelligent agent (i.e., a remote unmanned platform). The task execution capability of the human-machine system can be abstracted into the following mathematical model: ; in, (Autonomy) represents the autonomous capabilities of remotely operated unmanned platforms. (Communication) represents the transmission capacity of a communication link. The value range is [0,1]; (Humanity) represents the capacity for human intervention. (Task) represents comprehensive task capabilities.

[0008] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a human-machine hybrid control method for weak communication environments, including: The back-end remote control center receives and displays the sensing data sent by the remote unmanned platform, and receives the first operation command input by the operator and sends the first operation command to the remote unmanned platform; the first operation command is used to adjust the motion state of the remote unmanned platform; the sensing data includes environmental data collected by the remote unmanned platform and the motion state of the remote unmanned platform itself; the remote unmanned platform and the back-end remote control center exchange data or commands wirelessly. The remote unmanned platform receives a first operation command sent by the back-end remote control center, and generates a second operation command based on the current task and perception data; the second operation command is used to adjust the motion state of the remote unmanned platform itself. The remote unmanned platform adjusts its motion state according to a first operation command and a second operation command, using a human-machine hybrid strategy. The human-machine hybrid strategy involves: calculating the communication quality with the backend remote control center; comparing the communication quality with an upper threshold and a lower threshold; adjusting the motion state according to the first operation command when the communication quality is above the upper threshold; adjusting the motion state according to the second operation command when the communication quality is below the lower threshold; determining the weights of the first and second operation commands based on the communication quality, calculating the weighted sum of the first and second operation commands to obtain a fused operation command; and adjusting the motion state according to the fused operation command.

[0009] Secondly, this application provides another human-machine hybrid control method for weak communication environments, including: Acquire sensing data and send it to the backend remote control center; the sensing data includes environmental data and the motion status of the remote unmanned platform itself; Receives a first operation command sent by the backend remote control center; the first operation command is used to adjust the motion state of the remote unmanned platform; the first operation command is input by the operator to the backend remote control center through the human-machine interface of the backend remote control center. A second operation command is generated based on the current task and perception data; the second operation command is used to adjust the motion state of the remote unmanned platform itself. Based on the first and second operation instructions, the system adjusts its own motion state using a human-machine hybrid strategy. The human-machine hybrid strategy involves: calculating the communication quality with the backend remote control center; comparing the communication quality with an upper and lower threshold; adjusting the system's motion state based on the first operation instruction when the communication quality is above the upper threshold; adjusting the system's motion state based on the second operation instruction when the communication quality is below the lower threshold; and adjusting the system's motion state based on the second operation instruction when the communication quality is between the upper and lower thresholds.

[0010] Thirdly, this application provides a human-machine hybrid control system for weak communication environments, comprising: a back-end remote control center and a remote unmanned platform; the back-end remote control center includes a back-end signal receiving module, a back-end signal processing module, and a back-end signal transmitting module; the remote unmanned platform includes a front-end signal processing module, an instruction fusion module, and a fully autonomous module; wherein... The back-end signal receiving module is used to: receive sensing data sent by the remote unmanned platform, and output the sensing data to the back-end signal processing module.

[0011] The back-end signal processing module is used to: receive and display the sensing data sent by the back-end signal receiving module; receive the first operation command input by the operator and output the first operation command to the back-end signal sending module; the first operation command is used to adjust the motion state of the remote unmanned platform.

[0012] The backend signal transmission module is used to: receive the first operation instruction sent by the backend signal processing module, and send the first operation instruction to the remote unmanned platform.

[0013] The front-end signal processing module is used to: receive a first operation command sent by the back-end remote control center and output the first operation command to the command fusion module; receive sensing data sent by the fully autonomous module and send the sensing data to the back-end remote control center; calculate the communication quality with the back-end remote control center and output the communication quality to the command fusion module.

[0014] The fully autonomous module is used to: acquire sensing data and output the sensing data to the front-end signal processing module; generate a second operation command based on the current task and sensing data, and output the second operation command to the command fusion module; the second operation command is used to adjust the motion state of the remote unmanned platform; receive the operation command output by the command fusion module, and adjust the motion state of the remote unmanned platform itself according to the operation command output by the command fusion module.

[0015] The instruction fusion module is used to: receive a first operation instruction and communication quality output from the front-end signal processing module and a second operation instruction output from the fully autonomous module; compare the communication quality with an upper threshold and a lower threshold; when the communication quality is above the upper threshold, output the first operation instruction to the fully autonomous module; when the communication quality is below the lower threshold, determine the weights of the first and second operation instructions based on the communication quality, and calculate the weighted sum of the first and second operation instructions to obtain the fused operation instruction; and output the fused operation instruction to the fully autonomous module.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a human-machine hybrid control method and system for weak communication environments. A back-end remote control center receives and displays sensing data sent by a remote unmanned platform, and receives a first operation command input by an operator, which is then sent to the remote unmanned platform. The remote unmanned platform receives the first operation command from the back-end remote control center, generates a second operation command based on the current task and sensing data, and then adjusts its own motion state according to the first and second operation commands using a human-machine hybrid strategy. The human-machine hybrid strategy includes: calculating the communication quality with the back-end remote control center; comparing the communication quality with an upper and lower threshold; adjusting its own motion state according to the first operation command when the communication quality is above the upper threshold; adjusting its own motion state according to the second operation command when the communication quality is below the lower threshold; and adjusting its own motion state according to the second operation command when the communication quality is between the upper and lower thresholds. Finally, it determines the weights of the first and second operation commands based on the communication quality, calculates the weighted sum of the first and second operation commands to obtain a fused operation command, and adjusts its own motion state according to the fused operation command. Based on the above scheme, this application realizes that the remote unmanned platform can automatically switch between a remote operation mode based on remote control commands from the back-end remote control center and a fully autonomous mode based on autonomous commands from the remote unmanned platform according to the communication quality. Furthermore, a hybrid mode (fusion mode) is created. In the hybrid mode, the two types of operation commands generated by the remote unmanned platform and the back-end remote control center can be merged, thereby realizing a smooth transition between the two modes and improving the reliability of the human-machine system in weak communication environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a human-machine hybrid control method for weak communication environments provided in Embodiment 1 of this application; Figure 2 This is a flowchart illustrating a human-machine hybrid control method for weak communication environments provided in Embodiment 2 of this application. Figure 3 This is a schematic diagram of a human-machine hybrid control system for weak communication environments provided in Embodiment 3 of this application; Figure 4 This is a schematic diagram of a human-machine hybrid control system architecture for weak communication environments provided in Embodiment 3 of this application; Figure 5This is a schematic diagram of the operation mechanism of a human-machine hybrid control system for weak communication environments provided in Embodiment 3 of this application. Detailed Implementation

[0019] Currently, teleoperation control and fully autonomous control have become the two mainstream operating frameworks for remote unmanned platforms and are widely used in various application scenarios. However, these two modes still have some limitations in weak communication environments. For example, in complex real-world scenarios, traditional teleoperation modes face risks such as instruction delays and interruptions due to their reliance on real-time data transmission, severely affecting the operator's precise control of the remote unmanned platform; while fully autonomous modes may suffer from delayed decision-making or decreased task adaptability due to the inability to obtain remote operation instructions (also known as control instructions) in a timely manner. To address the limitations of existing technologies in weak communication environments, this application proposes a human-machine hybrid control method and system for weak communication environments, providing reliable technical support for the application of unmanned systems in weak communication environments. In this paper, the weak communication environment is also referred to as the weak communication environment, and the remote unmanned platform is also referred to as the front-end unmanned platform.

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] Example 1 In this embodiment, as Figure 1 As shown, a human-machine hybrid control method for weak communication environments is provided, including the following steps 101 to 103. Wherein: Step 101: The back-end remote control center receives and displays the sensing data sent by the remote unmanned platform, and receives the first operation command input by the operator and sends the first operation command to the remote unmanned platform; the first operation command is used to adjust the motion state of the remote unmanned platform; the sensing data includes environmental data collected by the remote unmanned platform and the motion state of the remote unmanned platform itself; the remote unmanned platform and the back-end remote control center exchange data or commands wirelessly.

[0023] Step 102: The remote unmanned platform receives the first operation command sent by the back-end remote control center, and generates a second operation command based on the current task and perception data; the second operation command is used to adjust the motion state of the remote unmanned platform.

[0024] Step 103: The remote unmanned platform adjusts its motion state according to the first and second operation instructions using a human-machine hybrid strategy. The human-machine hybrid strategy is as follows: calculate the communication quality with the back-end remote control center; compare the communication quality with an upper threshold and a lower threshold; when the comparison result is that the communication quality is above the upper threshold, adjust its motion state according to the first operation instruction; when the comparison result is that the communication quality is below the lower threshold, adjust its motion state according to the second operation instruction; when the comparison result is that the communication quality is between the upper and lower thresholds, determine the weights of the first and second operation instructions based on the communication quality, and calculate the weighted sum of the first and second operation instructions to obtain the fused operation instruction; adjust its motion state according to the fused operation instruction.

[0025] As an alternative implementation method, the front-end unmanned platform acquires perception data based on its onboard multimodal AI perception system.

[0026] As an alternative implementation, the back-end remote control center can construct a global digital twin scene based on perception data for operators to view.

[0027] Example 2 In this embodiment, as Figure 2 As shown, a human-machine hybrid control method for weak communication environments is provided. The main execution subject of this method is a remote unmanned platform (hereinafter referred to as the unmanned platform), and the method includes the following steps 201 to 204. Among them: Step 201: Acquire sensing data and send the sensing data to the backend remote control center; the sensing data includes environmental data and the motion status of the remote unmanned platform itself.

[0028] Step 202: Receive the first operation instruction sent by the backend remote control center; the first operation instruction is used to adjust the motion state of the remote unmanned platform; the first operation instruction is input by the operator to the backend remote control center through the human-machine interface of the backend remote control center.

[0029] Step 203: Generate a second operation command based on the current task and perception data; the second operation command is used to adjust the motion state of the remote unmanned platform itself.

[0030] Step 204: Based on the first and second operation instructions, adjust the user's motion state using a human-machine hybrid strategy. The human-machine hybrid strategy is as follows: calculate the communication quality with the backend remote control center; compare the communication quality with an upper threshold and a lower threshold; when the comparison result is that the communication quality is above the upper threshold, adjust the user's motion state according to the first operation instruction; when the comparison result is that the communication quality is below the lower threshold, adjust the user's motion state according to the second operation instruction; when the comparison result is that the communication quality is between the upper and lower thresholds, determine the weights of the first and second operation instructions based on the communication quality, and calculate the weighted sum of the first and second operation instructions to obtain the fused operation instruction; adjust the user's motion state according to the fused operation instruction.

[0031] As an optional implementation, calculating the weighted sum of the first operation instruction and the second operation instruction specifically includes: Step 204-11: Preprocess the first operation instruction and the second operation instruction to obtain the preprocessed first operation instruction and the second operation instruction; the preprocessing includes spatiotemporal alignment operation and format standardization operation.

[0032] Step 204-12: Using a weighted fusion model, calculate the weighted sum of the preprocessed first and second operation instructions to obtain the fused operation instructions; the mathematical expression of the weighted fusion model is: ; in, These are the merged operation instructions; and These are the first and second operation instructions after preprocessing, respectively. for The weight, ; for The weights; the weights The weights are determined based on communication quality. It is directly proportional to the quality of communication.

[0033] As an optional implementation, the communication quality with the back-end remote control center is calculated, specifically including: Steps 204-21: Calculate the bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio of the communication channel with the back-end remote control center.

[0034] Steps 204-22: Normalize the bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio respectively to obtain normalized bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio.

[0035] Steps 204-23 involve performing a weighted summation on the normalized bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio to obtain the weighted summation result.

[0036] Steps 204-24 determine the weighted summation result as the communication quality between the remote unmanned platform and the back-end remote control center.

[0037] In this embodiment, a comprehensive evaluation value is calculated based on the above four indicators, denoted as . Q This is used as a quantitative representation of communication quality. For example, during flight, a drone (remote unmanned platform) establishes a communication channel with the back-end remote control center to transmit operational commands and telemetry data. The following real-time indicators (assuming measurements at a certain moment) are obtained through the monitoring system: Bandwidth availability ( B : (This represents the proportion of available bandwidth to total bandwidth; the higher the better).

[0038] Packet loss rate ( L : (Indicates the percentage of data packets lost; the lower the better).

[0039] latency jitter J : (Indicates the amount of delay change, in milliseconds; the lower the better).

[0040] Signal-to-noise ratio (SNR) S : (Indicates the ratio of signal to noise, in decibels, the higher the better).

[0041] Calculate the comprehensive evaluation value based on the above indicators. Q (The range is typically 0 to 1, where 1 represents the best quality and 0 represents the worst quality), and Q This was determined to be related to communication quality.

[0042] Overall assessment value Q The calculation formula is: ; in: Q B It is the normalized value of bandwidth availability (used directly). B ,because B It is itself a decimal in the range of 0-1.

[0043] Q L This is the normalized value of the packet loss rate (since a high packet loss rate indicates poor quality, it is used...). (Converted to quality score).

[0044] Q J It is the normalized value of latency jitter (assuming the maximum acceptable jitter).J max = 50 ms; using Converted to quality rating, when J =0 equals 1. J = J max (When it is 0).

[0045] Q S It is the normalized value of the signal-to-noise ratio (assuming an SNR range of 0-40 dB; using...). S / S max Converted to quality score, where S max = 40dB, when S = 1 at 40dB S = 0dB (is 0).

[0046] Weight: = 0.25, = 0.25, = 0.25, = 0.25 (meaning this example is an equal-weighted example).

[0047] In another exemplary embodiment, a human-machine hybrid control method for weak communication environments further includes: Step 205: When the communication quality is detected to recover from below the upper limit threshold to above the upper limit threshold, the third instruction buffer queue is compared with the first instruction buffer queue one by one to determine whether the third instruction buffer queue and the first instruction buffer queue are completely matched; the third instruction buffer queue includes a preset number of locally executed operation instructions; the first instruction buffer queue includes a preset number of first operation instructions sent by the back-end remote control center.

[0048] Step 206: When the third instruction buffer queue and the first instruction buffer queue are not completely matched, an instruction error compensation operation is performed to generate N predictive compensation instructions, and the motion state of the remote unmanned platform is adjusted according to the N predictive compensation instructions.

[0049] As an optional implementation, step 206, "when the third instruction buffer queue and the first instruction buffer queue are not completely matched, perform instruction error compensation operation to generate N predictive compensation instructions," specifically includes: Step 206-1: When the third instruction buffer queue and the first instruction buffer queue are not completely matched, the first instruction buffer queue is truncated with the position of the first operation instruction that failed to match for the first time in the first instruction buffer queue as the starting position and the position of the first operation instruction at the end of the first instruction buffer queue as the ending position, so as to obtain the truncated first instruction buffer queue.

[0050] A successful comparison indicates that the locally executed operation instruction is the first operation instruction; a failed comparison indicates that the locally executed operation instruction is not the first operation instruction but the second operation instruction.

[0051] Step 206-2: Determine the expected motion state of the remote unmanned platform based on the intercepted first instruction buffer queue.

[0052] Step 206-3: Determine the current motion state of the remote unmanned platform.

[0053] Step 206-4: Calculate the difference between the expected motion state of the remote unmanned platform and the current motion state of the remote unmanned platform.

[0054] Step 206-5: Based on the difference and the dynamic model of the remote unmanned platform, reverse calculation is performed using state estimation and extrapolation algorithms to obtain N predictive compensation commands.

[0055] The generation process of the predictive compensation instruction is further explained below: First, the third instruction buffer queue (local execution record) and the first instruction buffer queue (backend transmission record) are compared frame by frame to identify the position of the first forked instruction and all subsequent inconsistent instruction sequences. Based on the difference between the real-time state of the unmanned platform detected by the current motion sensor (such as attitude, velocity, and position) and the theoretical state expected to be reached by the remote instruction sequence, combined with the dynamic model of the controlled object (such as the UAV's motion equation), state estimation and extrapolation algorithms (such as Kalman filtering or least squares fitting) are used to inversely calculate one or more predictive compensation instructions. This instruction is not a simple replay of the missing instruction, but rather converts the state error into an equivalent control quantity increment (such as generating a compensation angular velocity or acceleration instruction in attitude control). Its goal is to enable the controlled object to quickly converge to the expected trajectory point that the first instruction buffer queue should have reached in the next control cycle, while satisfying the system dynamic constraints and the preset safety range.

[0056] Example 3 Based on the same inventive concept, this application also provides a human-machine hybrid control system for weak communication environments. The solution provided by this system is similar to the solution described in the above method; therefore, the specific limitations of one or more embodiments of the human-machine hybrid control system for weak communication environments provided below can be found in the limitations of the human-machine hybrid control method for weak communication environments described above.

[0057] In this embodiment, as Figure 3 As shown, a human-machine hybrid control system for weak communication environments is provided, including a back-end remote control center and a remote unmanned platform; the back-end remote control center includes a back-end signal receiving module, a back-end signal processing module, and a back-end signal transmitting module; the remote unmanned platform includes a front-end signal processing module, a command fusion module, and a fully autonomous module; wherein: The back-end signal receiving module is used to: receive sensing data sent by the remote unmanned platform, and output the sensing data to the back-end signal processing module.

[0058] The back-end signal processing module is used to: receive and display the sensing data sent by the back-end signal receiving module; receive the first operation command input by the operator and output the first operation command to the back-end signal sending module; the first operation command is used to adjust the motion state of the remote unmanned platform.

[0059] The backend signal transmission module is used to: receive the first operation instruction sent by the backend signal processing module, and send the first operation instruction to the remote unmanned platform.

[0060] The front-end signal processing module is used to: receive a first operation command sent by the back-end remote control center and output the first operation command to the command fusion module; receive sensing data sent by the fully autonomous module and send the sensing data to the back-end remote control center; calculate the communication quality with the back-end remote control center and output the communication quality to the command fusion module.

[0061] The fully autonomous module is used to: acquire sensing data and output the sensing data to the front-end signal processing module; generate a second operation command based on the current task and sensing data, and output the second operation command to the command fusion module; the second operation command is used to adjust the motion state of the remote unmanned platform; receive the operation command output by the command fusion module, and adjust the motion state of the remote unmanned platform itself according to the operation command output by the command fusion module.

[0062] The instruction fusion module is used to: receive a first operation instruction and communication quality from the front-end signal processing module, and a second operation instruction from the fully autonomous module; compare the communication quality with an upper threshold and a lower threshold; when the communication quality is above the upper threshold, output the first operation instruction to the fully autonomous module; when the communication quality is below the lower threshold, output the second operation instruction to the fully autonomous module; when the communication quality is between the upper threshold and the lower threshold, determine the weights of the first operation instruction and the second operation instruction based on the communication quality, and calculate the weighted sum of the first operation instruction and the second operation instruction to obtain the fused operation instruction; and output the fused operation instruction to the fully autonomous module.

[0063] As an optional implementation method, such as Figure 4 As shown, the instruction fusion module includes a multi-source instruction receiving unit, a dynamic fusion calculation unit, and a fusion instruction sending unit; wherein, The multi-source instruction receiving unit is used to: receive the first operation instruction and communication quality output by the front-end signal processing module and the second operation instruction output by the fully autonomous module; and output the first operation instruction, the communication quality and the second operation instruction to the dynamic fusion computing unit.

[0064] The dynamic fusion computing unit is used for: receiving a first operation instruction, communication quality, and a second operation instruction output by the multi-source instruction receiving unit; comparing the communication quality with an upper threshold and a lower threshold; when the communication quality is above the upper threshold, outputting the first operation instruction to the fusion instruction sending unit; when the communication quality is below the lower threshold, outputting the second operation instruction to the fusion instruction sending unit; when the communication quality is between the upper threshold and the lower threshold, determining the weights of the first operation instruction and the second operation instruction based on the communication quality, and calculating the weighted sum of the first operation instruction and the second operation instruction to obtain the fused operation instruction; and outputting the fused operation instruction to the fusion instruction sending unit.

[0065] The fusion instruction sending unit is used to: receive the operation instructions output by the dynamic fusion computing unit; and output the operation instructions output by the dynamic fusion computing unit to the fully autonomous module; wherein the operation instructions output by the dynamic fusion computing unit are a first operation instruction, a second operation instruction, or a fused operation instruction.

[0066] As an optional implementation method, such as Figure 4 As shown, the fully autonomous module includes a perception unit, a planning and decision-making unit, and a motion control unit; wherein, The sensing unit is used to: acquire sensing data and output the sensing data to the planning and decision-making unit and the front-end signal processing module.

[0067] The planning and decision-making unit is used to: receive sensing data output by the sensing unit; generate an autonomous control strategy based on the current task and the sensing data, and output the autonomous control strategy to the motion control unit; the autonomous control strategy includes a motion path.

[0068] The motion control unit is used to: receive the autonomous control strategy output by the planning and decision-making unit; generate a second operation command according to the autonomous control strategy and send the second operation command to the command fusion module; receive the operation command output by the command fusion module and adjust the motion state of the remote unmanned platform itself according to the operation command output by the command fusion module.

[0069] As an optional implementation method, such as Figure 4 As shown, the front-end signal processing module includes a front-end signal receiving unit, a front-end signal processing unit, and a front-end signal transmitting unit. The front-end signal processing unit can perform signal analysis and processing based on the received signal to calculate the communication quality with the back-end remote control center, and then output the result to the command fusion module.

[0070] In this embodiment, the remote unmanned platform collects surrounding environmental information through a sensing unit and sends the collected information to a front-end signal transmission unit, which then transmits it to the back-end remote control center. The back-end operator (i.e., the operator) receives the data sent by the remote unmanned platform through a back-end signal receiving module, analyzes the front-end data through a back-end signal processing module, generates a first operation command, and then sends it back to the remote unmanned platform via the back-end signal transmission module. The command fusion module integrates multi-source commands and generates autonomous decisions (i.e., fused operation commands) through dynamic fusion calculations, which are then executed by the remote unmanned platform. The entire process achieves bidirectional information interaction in a weak communication environment, ensuring efficient collaboration and autonomous control between the front-end and back-end. This system innovatively integrates remote operation mode and fully autonomous mode, achieving continuous and reliable control across the entire time domain under complex channel conditions such as limited communication bandwidth and unstable latency by constructing a multimodal interaction interface, a dynamic weight allocation mechanism, and an adaptive communication scheduling algorithm.

[0071] The system can dynamically adjust the human-machine control weights based on real-time communication quality. When the communication link is in good condition, the operator remote operation mode is prioritized to leverage the advantages of human decision-making. When the communication quality deteriorates, it smoothly transitions to autonomous control mode, maintaining basic task execution capabilities through a local intelligent agent (i.e., a remote unmanned platform). This can be expressed by the following formula: ; in, (Autonomy) represents the autonomous capabilities of remotely operated unmanned platforms. (Communication) represents the transmission capacity of a communication link. The value range is [0,1]; (Humanity) represents the capacity for human intervention. (Task) represents comprehensive task capabilities.

[0072] The task execution capability of an unmanned system, namely its comprehensive task capability, is closely related to the autonomous capability of the unmanned platform, the information transmission capability of the communication link, and the human participation control capability. The task execution capability of an unmanned platform not only depends on the trainer (H), but the platform's capability (A) is also changing dynamically and is also affected by the communication link (C), making it a complex organic community.

[0073] The dual-mode seamless switching mechanism can solve key problems such as control interruption and command loss in traditional remote control systems in weak communication environments, and significantly improve the mission reliability and environmental adaptability of unmanned platforms in complex scenarios such as emergency rescue and disaster relief and remote area operations.

[0074] Example 1 of the application scenario of the system provided in this embodiment: In earthquake relief scenarios, the backend remote control center comprises a cloud server cluster for the backend command center. These servers receive real-time data from the frontend remote unmanned vehicles (remote unmanned platforms), including location information, environmental perception data, and task status, and perform global path planning, traffic scheduling, and anomaly monitoring. When manual intervention is required, operators can send commands to the unmanned vehicles through the backend remote control center, such as adjusting the vehicle's current movement state, adjusting task priority, or changing the vehicle's route. The frontend signal processing module is deployed in the onboard computing unit of each unmanned vehicle, such as an industrial control computer or embedded AI computing device. The fully autonomous module is responsible for processing data collected by local sensors (such as LiDAR, cameras, and ultrasonic radar), performing real-time environmental perception, obstacle detection, and local or global path planning. The processed data is sent to the backend remote control center, while simultaneously receiving commands from the backend remote control center to ensure the safe operation of the unmanned vehicles in complex environments. The command fusion module runs on an onboard high-performance computing platform, such as NVIDIA Jetson AGX Orin or similar AI computing devices. Its role is to integrate commands from different sources, including commands sent from the backend remote control center and output commands from the local fully autonomous module. Through dynamic fusion computing, the optimal control strategy is obtained, such as switching to autonomous mode when communication is delayed. The fully autonomous module is directly embedded in the underlying control hardware of the unmanned platform, including an environmental perception module (sensor array), a planning and decision-making module (real-time path planning algorithm), and a motion control module (drive motor, steering, and braking controller). When communication is interrupted or a rapid response is required, the system can operate completely independently, relying on local sensors and algorithms to complete operations such as obstacle avoidance, stopping, and emergency braking, ensuring that the vehicle can still safely perform tasks in environments without network connectivity.

[0075] In some embodiments, the instruction fusion module can also fuse cooperative information from other vehicles to obtain an adjusted driving strategy when multiple vehicles are cooperating.

[0076] The workflow of the system in this embodiment (hereinafter referred to as the system) is further explained below.

[0077] 1. Quantitative Analysis of Communication-Restricted Characteristics The system adopts a two-way communication architecture, where both operator commands and feedback information from the unmanned platform are transmitted via the communication link. In actual operation, the communication channel dynamically exhibits two typical operating conditions: when channel conditions are good, the system uses a standard transmission mode and a full data transmission mode; when channel quality deteriorates, the system can use the front-end signal processing module to perform multi-dimensional feature extraction and quantification analysis on the transmitted data, reducing the amount of data transmitted and ensuring reliable transmission.

[0078] (1-1) Communication system connectivity judgment During mission execution, the unmanned platform first performs real-time diagnostics and assessments of the current communication link status. By detecting key indicators such as signal strength, transmission latency, and data packet loss rate, it dynamically determines the connectivity and stability of the communication connection. This assessment process runs continuously at millisecond-level frequencies, providing real-time communication status data for subsequent autonomous decision-making.

[0079] (1-2) Signal reception The back-end remote control center sends operation instructions (i.e., the first operation instructions) containing status parameters such as speed and direction to the unmanned platform. At the same time, the unmanned platform feeds back its real-time status parameters and environmental information to the back-end remote control center, completing the control closed loop.

[0080] (1-3) Signal Analysis and Processing The system adopts an intelligent communication processing mechanism to evaluate the channel quality in real time after receiving the signal and determine the communication status through quantitative indicators (e.g., bandwidth availability β≥80%, packet loss rate ρ≤5%, latency jitter σ≤50ms, signal-to-noise ratio SNR≥20dB).

[0081] The bandwidth availability β can be expressed by the following formula: ; in, Represents the currently available bandwidth (Mbps). This represents the theoretical maximum bandwidth of the channel (Mbps).

[0082] Packet loss rate ρ It can be expressed by the following formula: ; in, Represents the number of lost data packets. This represents the total number of data packets sent.

[0083] latency jitter σ It can be expressed by the following formula: ; in, Representing the Data packet latency (ms) Represents average latency. This represents the size of the sampling window.

[0084] During operation, the unmanned platform first monitors and diagnoses the communication link status in real time. By comprehensively evaluating key indicators such as signal strength, transmission latency, and data packet loss rate, the platform can accurately determine whether the current communication system is connected. When an uninterrupted communication link is detected, the platform immediately activates the signal receiving mechanism to receive and parse commands sent by the backend remote control center in real time.

[0085] In other words, the unmanned platform performs multi-dimensional quantitative analysis of the received communication signals, including parameters such as channel bandwidth utilization, signal-to-noise ratio, and transmission stability. Based on this real-time analysis data, the platform can dynamically assess the quality of current communication conditions, providing crucial information for subsequent command processing and execution strategies. This process operates continuously at millisecond-level frequencies, ensuring that the unmanned platform can respond optimally under any communication conditions.

[0086] Among them, (1-1) first judges the communication status. If the signal is interrupted after (1-1), the fully autonomous mode is started directly. If the signal is connected after (1-1), then (1-2) is performed to receive the signal and (1-3) is performed to analyze and process the status of the communication signal.

[0087] 2. Autonomous control of unmanned platforms The unmanned platform has complete autonomous mission execution capabilities, and its fully autonomous module includes a perception unit, a planning and decision-making unit, and a motion control unit.

[0088] The sensing unit is responsible for collecting environmental information and the unmanned platform's own motion status in real time.

[0089] The planning and decision-making unit generates action strategies (i.e. autonomous control strategies) based on task objectives and environmental information.

[0090] The motion control unit generates action commands (i.e., second operation commands) according to the action strategy and executes each action command precisely.

[0091] The three units work together through an efficient data interaction mechanism to form a complete autonomous decision-making closed loop, enabling the unmanned platform to complete its current task independently and make rapid adaptive adjustments when the environment changes.

[0092] (2-1) Sensing unit The perception unit adopts a collaborative architecture of multiple sub-units, mainly including a sensing sub-unit, a positioning sub-unit, and a communication sub-unit, which together realize complete situational awareness capabilities.

[0093] The sensing subunit integrates multiple high-precision sensors, such as: lidar for acquiring 3D point cloud data, a vision camera for acquiring RGB image data and depth information, millimeter-wave radar for acquiring target velocity and azimuth data, and ultrasonic sensors for acquiring distance data to nearby obstacles. The sensing subunit constructs a high-precision 3D environment model through real-time data acquisition and fusion processing, and employs the real-time 3D reconstruction algorithm TSDF. This model provides an accurate environmental representation for subsequent path planning, dynamic obstacle avoidance, and task decision-making, ensuring the autonomous navigation and task execution capabilities of the unmanned platform in complex scenarios.

[0094] The positioning subunit employs advanced SLAM (Simultaneous Localization and Mapping) technology. By fusing data from LiDAR, visual cameras, and inertial measurement units (IMUs), it achieves real-time positioning of the platform's position (X / Y / Z coordinates) and attitude (pitch / roll / yaw angle) in three-dimensional space (three-dimensional environment model) with centimeter-level accuracy. At the same time, it builds and updates environmental feature maps to support navigation decisions.

[0095] The communication subunit is deeply integrated with the unmanned platform, sending the environmental feature map to the front-end signal transmission unit, which then sends it to the back-end remote control center. Through two-way data interaction, it assists the operator in obtaining remote environmental information and realizes collaborative perception of "humans on the loop".

[0096] (2-2) Planning and Decision-Making Unit The planning and decision-making unit of the unmanned platform adopts a hierarchical architecture to achieve autonomous decision-making throughout the entire process, from global path planning to local motion control.

[0097] The planning and decision-making unit includes a task layer and an execution layer.

[0098] The task layer generates a globally optimal path based on the current task objective and the environment map using the RRT* (Rapid Expanding Random Tree Star) algorithm. The specific process is as follows: First, nodes are randomly sampled in the feasible space of the environment map. Effective nodes are connected through incremental tree expansion. Then, the path is locally optimized and rewired. Finally, iterative convergence is performed to a feasible path that satisfies the optimality condition. This path satisfies both obstacle avoidance constraints and task efficiency requirements.

[0099] The execution layer utilizes real-time planning techniques such as the dynamic window method, combined with sensor data collected by the perception module, to perform online optimization and obstacle avoidance of local trajectories. The platform employs the Dynamic Window Method (DWA) for local trajectory optimization and real-time obstacle avoidance. Its execution process is as follows: First, a dynamic velocity window is generated based on the current motion state, sampling multiple feasible velocity combinations within the velocity space. Then, combining real-time obstacle information acquired by LiDAR and visual sensors, collision detection and trajectory simulation are performed on each velocity combination. Finally, a multi-objective cost function (including a weighted index of path length, energy efficiency, and safety margin) is used to evaluate all feasible trajectories, selecting the optimal solution as the current optimal local path, achieving safe and efficient local path tracking.

[0100] (2-3) Motion control unit The unmanned platform transmits the processed multidimensional information to the motion control unit through the planning and decision unit. The specific data transmitted includes: a) a three-dimensional point cloud map and feature information of the environment; b) the generated global optimal path coordinate sequence and the optimal local path coordinate sequence; c) the predicted trajectory and collision risk level of dynamic obstacles; d) the current pose state of the unmanned platform (including position, velocity, acceleration and other data, i.e. motion state).

[0101] The motion control unit employs a model predictive control (MPC) framework for real-time calculation: First, an optimization problem is constructed, incorporating dynamic constraints, obstacle avoidance constraints, and path tracking errors. Then, a QP (quadratic programming) solver calculates the optimal control variables (including linear velocity, angular velocity, and their rates of change), i.e., the second operation command, within a 10ms period and sends it to the command fusion unit. Finally, the operation command output by the command fusion unit (the second operation command in fully autonomous mode) is sent to the drive motor and steering mechanism via the CAN bus. During execution, a closed-loop feedback system consisting of an encoder and an inertial measurement unit (IMU) corrects control deviations in real time, ensuring a trajectory tracking accuracy of ±2cm.

[0102] The unmanned platform, through its sensing unit, planning and decision-making unit, and motion control unit, can achieve fully autonomous mission exploration even when communication is interrupted. If the communication system is not interrupted, but the channel quality is weak, the unmanned platform will still output commands through the fully autonomous system and fuse them with commands from the remote operator to obtain the final command output. During fusion, the motion control unit sends the calculated second operation command to the command fusion module, which then fuses the two commands and returns them to the motion control unit.

[0103] 3. Real-time integration of human and machine commands (3-1) Instruction Fusion As the core of the unmanned platform, the instruction fusion module adopts a multi-level fusion architecture to intelligently integrate two types of input instructions: First, it performs spatiotemporal alignment and format standardization processing on the local instructions (second operation instructions) generated autonomously by the unmanned platform and the remote operation instructions (first operation instructions) transmitted from the communication system; spatiotemporal alignment is used to ensure that the two types of operation instructions are aligned in time and space; then, based on the current communication quality, the fusion of instructions is achieved through a dynamic weight allocation algorithm, as detailed in the previous embodiment.

[0104] The system supports three modes: 1) Remote operation mode primarily using remote commands when communication is good; 2) A fully autonomous mode that relies primarily on local autonomous decision-making when communication is limited; 3) Hybrid weighted mode under normal conditions.

[0105] The merged instructions are sent to the merged instruction sending unit for instruction transmission.

[0106] (3-1) Instruction Output The command fusion module transmits the fused hybrid commands to the motion control unit through a highly reliable communication interface.

[0107] In some embodiments, the human-machine hybrid control system for weak communication environments further includes an error compensation module; the front-end signal processing module is also used to output the communication quality to the error compensation module.

[0108] The error compensation module is used for: The system receives the communication quality output from the front-end signal processing module. When the communication quality recovers from below the upper limit threshold to above the upper limit threshold, the system compares the third instruction buffer queue with the first instruction buffer queue one by one to determine whether the third instruction buffer queue and the first instruction buffer queue are completely matched. The third instruction buffer queue includes a preset number of locally executed operation instructions. The first instruction buffer queue includes a preset number of first operation instructions sent by the back-end remote control center.

[0109] When the third instruction buffer queue and the first instruction buffer queue are not completely matched, an instruction error compensation operation is performed to generate N predictive compensation instructions. These N predictive compensation instructions are then sent to the fully autonomous module, enabling the fully autonomous module to adjust the motion state of the remote unmanned platform according to the N predictive compensation instructions; N≥1. The specific process of the instruction error compensation operation is described above.

[0110] In this embodiment, the unmanned platform employs a dual-buffering mechanism to cache instruction data: the receiving thread parses the header information of the first operation instruction packet in real time, verifies data integrity, and caches it; the execution thread processes the action instruction queue (i.e., the actual operation instructions to be executed) based on a priority scheduling algorithm to ensure real-time response to critical instructions and caches them. The system uses timestamp synchronization and instruction prediction compensation technology to dynamically generate predicted instructions by combining historical instruction data, current system status, and communication latency estimates to compensate for control errors caused by transmission delays. This achieves seamless integration of local autonomous decision-making and remote control instructions.

[0111] The command fusion system achieves deep coupling between remote operation commands and autonomous decision-making commands through a multimodal collaborative control mechanism. The system employs a dynamic weight allocation algorithm based on communication quality assessment to maintain control effectiveness in weak communication environments through the following mechanisms: First operation instruction buffer queue construction: adopts a circular buffer structure, stores the most recent 10 groups of first operation instructions (including control variables and corresponding timestamps) according to timestamps, and automatically activates the queue when the communication delay exceeds the threshold.

[0112] When communication is interrupted, the system first retrieves the latest instruction from the first operation instruction buffer queue, while the prediction model updates the motion state estimate (second operation instruction) every 50ms. The two are weighted and fused to generate a compensation control quantity (fused instruction) to maintain the stable operation of the unmanned platform when communication is interrupted or delayed.

[0113] After communication is restored, differential instruction synchronization is achieved by comparing the CRC checksums of the local execution records with those of the cloud-based instructions (first operation instruction buffer queue) using a sliding window algorithm. The local execution records are the sequence of operation instructions actually executed by the unmanned platform during the communication interruption (including timestamps, control variables, and sensor feedback data). These records are stored in the vehicle's memory for subsequent comparison with cloud-based instructions. The CRC checksum is used to verify the integrity and consistency of instruction transmission. For example, when A sends data to B, A first performs a CRC operation on the transmitted data to obtain a CRC checksum, and then sends the transmitted data and the CRC checksum together to B. After receiving the transmitted data, B also performs a CRC operation on the transmitted data to obtain a CRC checksum and compares it with the received CRC checksum. If the two are completely identical, it is determined that the transmitted data is the original transmitted data. After communication is restored, the system compares the CRC checksums of the locally executed instructions with those of the original instructions sent from the cloud to detect whether control deviations are caused by packet loss or delay. If the comparison fails (e.g., CRC checksum mismatch), it indicates that the locally executed instructions do not match the cloud's expectations, and an error compensation mechanism needs to be triggered. Specifically, a sliding window algorithm can be used to select the instruction sequence of the most recent period (such as a window of the past 200ms), and compare the differences between local execution and cloud instructions frame by frame. Only the parts with deviations are synchronized (compensated). Its core purpose is to quickly eliminate the accumulated error between autonomous operation and remote control after communication is restored, and to control the synchronization error within ±1.5cm to ensure the integrity of the task and the safety of the movement.

[0114] See Figure 5 This application enables real-time fusion of remote operation mode and fully autonomous mode under weak communication link conditions, achieving the concept of all-time fused control. First, the unmanned platform analyzes the connectivity of the communication system. If the communication system is disconnected, the unmanned platform initiates fully autonomous mode to explore the mission. If the communication system is weak, the fully autonomous exploration commands and remote operation commands are fused through a command fusion system to enable unmanned platform exploration under weak communication conditions. If the communication system is good, the unmanned platform receives remote operation commands and conducts mission exploration.

[0115] This application has the following advantages: The human-machine hybrid intelligent control system proposed in this application innovatively solves the control interruption problem of traditional time-sharing systems in weak communication environments by deeply integrating remote operation and autonomous control modes. This architecture employs core technologies such as dynamic weight allocation and predictive compensation control, enabling it to automatically adjust the human-machine control ratio based on real-time communication quality, achieving seamless integration from high-level semantic commands to low-level motion control.

[0116] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0117] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A human-machine hybrid control method for weak communication environments, characterized in that, include: The back-end remote control center receives and displays the sensing data sent by the remote unmanned platform, and receives the first operation command input by the operator and sends the first operation command to the remote unmanned platform; the first operation command is used to adjust the motion state of the remote unmanned platform; the sensing data includes environmental data collected by the remote unmanned platform and the motion state of the remote unmanned platform itself; the remote unmanned platform and the back-end remote control center exchange data or commands wirelessly. The remote unmanned platform receives a first operation command sent by the back-end remote control center, and generates a second operation command based on the current task and perception data; the second operation command is used to adjust the motion state of the remote unmanned platform itself. The remote unmanned platform adjusts its motion state according to a first operation command and a second operation command, using a human-machine hybrid strategy. The human-machine hybrid strategy involves: calculating the communication quality with the backend remote control center; comparing the communication quality with an upper threshold and a lower threshold; adjusting the motion state according to the first operation command when the communication quality is above the upper threshold; adjusting the motion state according to the second operation command when the communication quality is below the lower threshold; and determining the weights of the first and second operation commands based on the communication quality, and calculating a weighted sum of the first and second operation commands to obtain the fused operation command. Adjust its own motion state according to the merged operation instructions.

2. A human-machine hybrid control method for weak communication environments, characterized in that, include: Acquire sensing data and send it to the backend remote control center; the sensing data includes environmental data and the motion status of the remote unmanned platform itself; Receives a first operation command sent by the backend remote control center; the first operation command is used to adjust the motion state of the remote unmanned platform; the first operation command is input by the operator to the backend remote control center through the human-machine interface of the backend remote control center. A second operation command is generated based on the current task and perception data; the second operation command is used to adjust the motion state of the remote unmanned platform itself. Based on the first and second operation instructions, the system adjusts its own motion state using a human-machine hybrid strategy. The human-machine hybrid strategy involves: calculating the communication quality with the backend remote control center; comparing the communication quality with an upper and lower threshold; adjusting the system's motion state according to the first operation instruction when the communication quality is above the upper threshold; adjusting the system's motion state according to the second operation instruction when the communication quality is below the lower threshold; and determining the weights of the first and second operation instructions based on the communication quality, and calculating the weighted sum of the first and second operation instructions to obtain the fused operation instructions. Adjust its own motion state according to the merged operation instructions.

3. The human-machine hybrid control method for weak communication environments according to claim 2, characterized in that, Calculate the weighted sum of the first and second operation instructions, specifically including: The first operation instruction and the second operation instruction are preprocessed to obtain the preprocessed first operation instruction and the second operation instruction; the preprocessing includes spatiotemporal alignment operation and format normalization operation. Using a weighted fusion model, the weighted sum of the preprocessed first and second operation instructions is calculated to obtain the fused operation instructions; the mathematical expression of the weighted fusion model is: ； in, These are the merged operation instructions; and These are the first and second operation instructions after preprocessing, respectively. for The weight, ; for The weights; the weights The weights are determined based on communication quality. It is directly proportional to the quality of communication.

4. The human-machine hybrid control method for weak communication environments according to claim 2, characterized in that, The communication quality between the computer and the back-end remote control center is calculated, specifically including: Calculate the bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio of the communication channel with the back-end remote control center; The bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio are normalized respectively to obtain normalized bandwidth availability, packet loss rate, latency jitter, and signal-to-noise ratio; The normalized bandwidth availability, packet loss rate, latency jitter and signal-to-noise ratio are weighted and summed to obtain the weighted summation result; The weighted summation result is used to determine the communication quality between the remote unmanned platform and the back-end remote control center.

5. The human-machine hybrid control method for weak communication environments according to claim 2, characterized in that, Also includes: When the communication quality is detected to recover from below the upper limit threshold to above the upper limit threshold, the third instruction buffer queue is compared with the first instruction buffer queue one by one to determine whether the third instruction buffer queue and the first instruction buffer queue are completely matched; the third instruction buffer queue includes a preset number of locally executed operation instructions; the first instruction buffer queue includes a preset number of first operation instructions sent by the back-end remote control center. When the third instruction buffer queue and the first instruction buffer queue are not completely matched, an instruction error compensation operation is performed to generate N predictive compensation instructions, and the motion state of the remote unmanned platform is adjusted according to the N predictive compensation instructions; N≥1.

6. The human-machine hybrid control method for weak communication environments according to claim 5, characterized in that, When the third instruction buffer queue and the first instruction buffer queue are not completely matched, an instruction error compensation operation is performed to generate N predictive compensation instructions, specifically including: When the third instruction buffer queue and the first instruction buffer queue do not match perfectly, the first instruction buffer queue is truncated, starting from the position of the first operation instruction that failed to match for the first time in the first instruction buffer queue and ending from the position of the first operation instruction at the end of the first instruction buffer queue. Based on the intercepted first instruction buffer queue, determine the expected motion state of the remote unmanned platform; Determine the current motion status of the remote unmanned platform; Calculate the difference between the expected motion state of the remote unmanned platform and the current motion state of the remote unmanned platform; Based on the difference and the dynamic model of the remote unmanned platform, N predictive compensation commands are obtained by reverse calculation using state estimation and extrapolation algorithms.

7. A human-machine hybrid control system for weak communication environments, characterized in that, include: The system comprises a back-end remote control center and a remote unmanned platform; the back-end remote control center includes a back-end signal receiving module, a back-end signal processing module, and a back-end signal transmitting module; the remote unmanned platform includes a front-end signal processing module, a command fusion module, and a fully autonomous module; wherein... The back-end signal receiving module is used to: receive sensing data sent by the remote unmanned platform, and output the sensing data to the back-end signal processing module; The back-end signal processing module is used to: receive and display the sensing data sent by the back-end signal receiving module; receive the first operation command input by the operator and output the first operation command to the back-end signal sending module; the first operation command is used to adjust the motion state of the remote unmanned platform. The backend signal transmission module is used to: receive the first operation instruction sent by the backend signal processing module, and send the first operation instruction to the remote unmanned platform; The front-end signal processing module is used to: receive a first operation command sent by the back-end remote control center and output the first operation command to the command fusion module; receive sensing data sent by the fully autonomous module and send the sensing data to the back-end remote control center; calculate the communication quality with the back-end remote control center and output the communication quality to the command fusion module; The fully autonomous module is used to: acquire sensing data and output the sensing data to the front-end signal processing module; generate a second operation command based on the current task and sensing data, and output the second operation command to the command fusion module; the second operation command is used to adjust the motion state of the remote unmanned platform; receive the operation command output by the command fusion module, and adjust the motion state of the remote unmanned platform itself according to the operation command output by the command fusion module; The instruction fusion module is used to: receive a first operation instruction and communication quality output from the front-end signal processing module, and a second operation instruction output from the fully autonomous module; compare the communication quality with an upper threshold and a lower threshold; when the communication quality is above the upper threshold, output the first operation instruction to the fully autonomous module; when the communication quality is below the lower threshold, output the second operation instruction to the fully autonomous module; when the communication quality is between the upper threshold and the lower threshold, determine the weights of the first operation instruction and the second operation instruction based on the communication quality, and calculate the weighted sum of the first operation instruction and the second operation instruction to obtain the fused operation instruction. The integrated operation commands are output to the fully autonomous module.

8. The human-machine hybrid control system for weak communication environments according to claim 7, characterized in that, The fully autonomous module includes a perception unit, a planning and decision-making unit, and a motion control unit; wherein... The sensing unit is used to: acquire sensing data and output the sensing data to the planning and decision-making unit and the front-end signal processing module; The planning and decision-making unit is used to: receive sensing data output by the sensing unit; generate an autonomous control strategy based on the current task and the sensing data, and output the autonomous control strategy to the motion control unit; the autonomous control strategy includes a motion path; The motion control unit is used to: receive the autonomous control strategy output by the planning and decision-making unit; generate a second operation command according to the autonomous control strategy and send the second operation command to the command fusion module; receive the operation command output by the command fusion module and adjust the motion state of the remote unmanned platform itself according to the operation command output by the command fusion module.

9. The human-machine hybrid control system for weak communication environments according to claim 7, characterized in that, In calculating the weighted sum of the first and second operation instructions, the specific operation performed by the instruction fusion module is as follows: The first operation instruction and the second operation instruction are preprocessed to obtain the preprocessed first operation instruction and the second operation instruction; the preprocessing includes spatiotemporal alignment operation and format normalization operation. Using a weighted fusion model, the weighted sum of the preprocessed first and second operation instructions is calculated to obtain the fused operation instructions; the mathematical expression of the weighted fusion model is: ； in, These are the merged operation instructions; and These are the first and second operation instructions after preprocessing, respectively. for The weight, ; for The weights; the weights The weights are determined based on communication quality. It is directly proportional to the quality of communication.

10. The human-machine hybrid control system for weak communication environments according to claim 7, characterized in that, It also includes an error compensation module; the front-end signal processing module is further used to output the communication quality to the error compensation module; The error compensation module is used for: The system receives the communication quality output from the front-end signal processing module. When the communication quality recovers from below the upper limit threshold to above the upper limit threshold, the system compares the third instruction buffer queue with the first instruction buffer queue one by one to determine whether the third instruction buffer queue and the first instruction buffer queue are completely matched. The third instruction buffer queue includes a preset number of locally executed operation instructions. The first instruction buffer queue includes a preset number of first operation instructions sent by the back-end remote control center. When the third instruction buffer queue and the first instruction buffer queue are not completely matched, an instruction error compensation operation is performed to generate N predictive compensation instructions, and the N predictive compensation instructions are sent to the fully autonomous module so that the fully autonomous module adjusts the motion state of the remote unmanned platform itself according to the N predictive compensation instructions; N≥1.

Images