Deep learning-based remote control system for mechanical equipment in closed space

Abstract

The invention discloses a deep learning-based remote control system for mechanical equipment in a closed space, and the system comprises a data collection module which collects and completes the alignment of an image, an environment distance and motion state data; the data fusion module is used for extracting space-time fusion features through image enhancement and an image convolutional network; the cooperative compensation module is used for realizing delay compensation and prediction state output by utilizing a Smith predictor and a multi-layer long and short time memory network coupling residual structure; the hybrid control network module is used for generating control parameters through adaptive fusion based on model prediction control and H-infinity robust control; the instruction issuing and feedback module is responsible for driving instruction issuing and feedback acquisition; and the on-line self-adaptive updating module is used for carrying out on-line fine adjustment on the compensation network based on the feedback error. According to the invention, the precision, the real-time performance and the robustness of remote control in the closed space are improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and remote control technology of mechanical equipment, and in particular to a remote control system for mechanical equipment in a confined space based on deep learning. Background Technology

[0002] Currently, remote control of mechanical equipment in confined spaces is widely used in special operating environments such as nuclear power plant maintenance, tunnel construction, and underground utility tunnel operation and maintenance. Traditional remote control systems typically rely on single video feeds or a small amount of sensor data, transmitting control commands and monitoring data to a remote control terminal via wired or wireless communication links. However, due to limitations such as low-light environments, complex structural obstructions, and network communication latency, traditional systems suffer from significant shortcomings in data acquisition accuracy, real-time performance, and control stability.

[0003] On the one hand, the image quality captured by cameras under low-light conditions is poor, resulting in limited visual information for operators and affecting mission safety and efficiency. On the other hand, existing systems mostly adopt simple data fusion and direct feedback control methods, failing to effectively combine multi-source sensing data to achieve environmental understanding and dynamic prediction, making it difficult to cope with delays, disturbances, and sudden anomalies during equipment movement.

[0004] Furthermore, current mainstream control methods lack adaptability to network latency and the reliability of visual information. Common methods such as PID control and traditional model predictive control are prone to system oscillations or command failures when communication fluctuations or signal degradation occur, failing to guarantee control robustness and safety in complex operating scenarios. To address these issues, while some research has attempted to introduce deep learning and state prediction mechanisms, these are mostly implemented using single networks or isolated modules, failing to form a multi-layered, end-to-end integrated and adaptive optimization system.

[0005] Therefore, how to provide a remote control system for mechanical equipment in enclosed spaces based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a remote control system for enclosed space mechanical equipment based on deep learning. Through low-light environment image enhancement, multi-source data spatiotemporal fusion, collaborative delay prediction with compensation, and adaptive hybrid control, this system comprehensively improves the remote control accuracy, real-time performance, and robustness of enclosed space mechanical equipment. The system utilizes deep neural networks to process multimodal perception information under low light, noise, and complex scenarios, enabling dynamic prediction and precise control of equipment status, significantly enhancing operational safety and stability in complex environments.

[0007] A remote control system for enclosed space mechanical equipment based on deep learning according to an embodiment of the present invention includes the following steps:

[0008] The data acquisition module is used to simultaneously acquire images, environmental distance and motion status data through a low-light camera, sonar sensor and inertial measurement unit, and complete timestamp alignment;

[0009] The data fusion module is used to perform low-light enhancement on image data. It inputs the enhanced image, distance data, and motion state into the graph convolutional network and outputs spatiotemporal fusion features.

[0010] The collaborative compensation module receives spatiotemporal fusion features, uses the Smith predictor, multi-layer long short-term memory network and residual correction structure to perform delay compensation, and outputs the predicted state.

[0011] The hybrid control network module includes a model predictive control branch and an H∞ robust control branch, which independently calculate candidate control parameters based on the predicted state, network latency, and visual credibility index, respectively. The fusion layer adaptively adjusts the weights based on network latency and visual credibility, and weights and fuses the control parameters output by the two branches to generate the final control parameters.

[0012] The instruction issuance and feedback module is used to map the final control parameters into drive instructions for each execution unit of the mechanical equipment, issue and collect the actual actions and feedback data of the equipment, and send them back to the main control terminal.

[0013] The online adaptive update module is used to calculate the error based on the feedback data and the predicted state, and to fine-tune the parameters of the collaborative compensation network online when the error exceeds the threshold.

[0014] Optionally, modules can be integrated using the following methods:

[0015] S1. The low-light camera collects image data, the sonar sensor collects environmental distance data, and the inertial measurement unit collects attitude and acceleration data.

[0016] S2. Image data is input into the residual dense generative adversarial network to generate an enhanced image. The distance data between the enhanced image and the environment, as well as the attitude and acceleration data, are input into the graph convolutional network to form a spatiotemporal fusion feature set.

[0017] S3. The spatiotemporal fusion feature input collaborative compensation network couples the Smith predictor delay compensation structure with a multi-layer long short-term memory network and residual units, collaboratively updates the internal state and uses the prediction error to drive weight adjustment, and outputs the delay-compensated prediction state.

[0018] S4. The delay compensation prediction state, network delay and visual credibility index are input into the hybrid control network. The hybrid control network adaptively fuses the network delay and visual credibility between the model branch and the H∞ branch, and outputs control parameters.

[0019] S5. Control parameters are mapped to preliminary execution instructions and sent to mechanical equipment. The mechanical equipment executes the instructions and returns real-time feedback.

[0020] S6. Calculate the error between the real-time feedback and the delay compensation prediction status. When the error exceeds the preset threshold, fine-tune the long short-term memory network and residual branch in the collaborative compensation network online and update the delay compensation prediction status.

[0021] S7. Calculate the final execution command based on the updated delay compensation prediction state and control parameters, and send it to the mechanical equipment.

[0022] This invention integrates a Smith predictor, a multi-layer long short-term memory network, and a residual correction module into a collaborative compensation network to achieve delay state prediction and dynamic correction for low-light image enhancement and multimodal spatiotemporal features. Furthermore, by combining model predictive control and H∞ robust control branches in a hybrid control network, control parameters are adaptively fused based on real-time communication latency and visual reliability to output refined execution instructions. The system can trigger an online fine-tuning mechanism when the feedback error exceeds the threshold, enabling model self-learning and updating. This method comprehensively improves the response stability, control accuracy, and adaptive capability of remote control systems in confined spaces under conditions of low light, high latency, and dynamic disturbances.

[0023] Optionally, step S1 includes the following specific steps:

[0024] S11. The low-light camera acquires image frames according to a fixed sampling period. The image frames have a uniform resolution and channel structure. Continuous image frames form an image data sequence in chronological order. The time interval between frames is not greater than the preset image refresh interval.

[0025] S12. The sonar sensor array synchronously transmits detection signals in each cycle and measures the propagation delay of echoes in each direction. Based on the preset speed, it calculates the environmental distance values ​​in multiple directions and generates the environmental distance data vector at the current moment.

[0026] S13. The inertial measurement unit periodically records attitude and acceleration information. The attitude is represented by Euler angles about the X, Y, and Z axes, and the acceleration is composed of linear accelerations along the X, Y, and Z axes. The six physical quantities together form the motion state vector at the current moment.

[0027] S14. Image data, environmental distance data, and attitude acceleration data are labeled with timestamps and then aligned. Image frames, environmental distance vectors, and motion state vectors acquired at the same time point are extracted and combined into ternary acquisition samples. Each ternary sampling result constitutes a time-series acquisition dataset, which serves as the input for subsequent processing steps.

[0028] This invention acquires low-light images, environmental distance, and three-axis attitude acceleration information simultaneously within a fixed sampling period, and uses a unified timestamp to accurately align the three types of heterogeneous data, constructing a structurally consistent ternary sampling dataset. This not only ensures the temporal consistency and spatial correspondence of multi-source sensing data, but also effectively solves the problems of asynchronous multi-sensor data, information loss, and large matching errors in traditional systems, significantly improving the perception integrity and data fusion efficiency of remote control systems in enclosed spaces.

[0029] Optionally, step S2 includes the following specific steps:

[0030] S21. Input each frame of image data from the time-series acquisition dataset into the residual dense generative adversarial network sequentially to perform pixel-level enhancement processing on the low-light images, and output the enhanced image frame sequence. Each enhanced image frame maintains the same resolution and number of channels as the original. The enhancement process includes multi-scale feature extraction, dense residual connections, and adversarial optimization of the discriminant network.

[0031] S22, at every moment Extract enhanced image frames The three types of data—environmental distance data vector, motion state vector, and so on—are used together as the node feature input to the graph convolutional network.

[0032] S23. In the graph convolutional network, the enhanced image frame is used as the visual node, each component of the environmental distance data is used as the spatial perception node, and the motion state vector is used as the pose node. An undirected weighted graph structure is constructed based on the preset spatial adjacency relationship, and the actual physical or spatial connection between nodes is reflected by the edge weight.

[0033] S24. Perform multi-layer graph convolution operations on all node features in the graph structure to achieve deep fusion of enhanced image features, spatial distance features, and motion state features. After fusion, the node features at each time step... Aggregates into spatiotemporal fusion feature vectors ;

[0034] S25. Arrange all spatiotemporal fusion feature vectors in chronological order. .

[0035] This invention enhances low-light image frames at the pixel level by inputting residual dense generative adversarial networks, and constructs heterogeneous graph convolutional structures by combining enhanced images, environmental distance data, and pose information, which are mapped to visual nodes, spatial perception nodes, and pose nodes, respectively. Then, multi-layer graph convolution is used to achieve deep fusion of multimodal features, which not only significantly improves the usability of image information in low-light environments, but also breaks down the structural correlation between visual and non-visual data. The resulting temporal fusion feature set has high-dimensional, strongly correlated, and dynamically continuous representation capabilities.

[0036] Optionally, step S3 includes the following specific steps:

[0037] S31. Input the feature vectors of each time step in the temporal fusion feature set into the collaborative compensation network in chronological order, and initialize the network's state buffer and memory units.

[0038] S32. The collaborative compensation network consists of a Smith predictor module, a multi-layer long short-term memory network structure, and a residual correction module. The Smith predictor module is used to extract historical features with a fixed delay length, the memory network is used to extract cross-temporal state evolution trends, and the residual correction module is used to construct and correct the output state.

[0039] S33, at every moment The Smith predictor module is invoked to read the time points from the stored feature sequences. eigenvectors ,in To preset the system delay steps, the current feature vector and Concatenate to form the input matrix: ;

[0040] S34, Input matrix The input is fed into a multi-layer long short-term memory network structure, and each layer processes the input layer by layer in the following manner:

[0041] The first layer of the Long Short-Term Memory network receives... The current state is updated based on the hidden state and the cell state of the previous time step;

[0042] Each layer of the Long Short-Term Memory (LSTM) network executes a gating mechanism, including the activation operations of the input gate, forget gate, and output gate, and updates the hidden state vector at the current time step. With the unit state vector ,in Indicates the first layer;

[0043] The last layer outputs the hidden state vector. , as the main channel output of the predicted state;

[0044] S35, will With delay characteristics Perform difference calculations to obtain the residual vector: The residual vector is input into the residual correction module, which is a residual network containing multiple layers of one-dimensional convolutions and nonlinear activation structures. This module performs feature transformation on the residual vector and... Element-wise addition generates a preliminary delay-compensated output vector. ;

[0045] S36, will The original fused feature vector at the current time By performing dimension-by-dimensional interpolation, the prediction error vector is obtained: If the error norm Exceeding the set threshold Then, the online weight update mechanism is invoked to perform a small-batch reverse adjustment operation on the forget gate weight matrix and the convolution kernel parameters in the residual module of the long short-term memory network. The update step size is calculated according to the set dynamic learning rate adjustment strategy.

[0046] S37. Output the updated delay compensation prediction state vector. And cache it in the predicted state sequence.

[0047] This invention constructs a collaborative compensation network integrating a Smith predictor, a multi-layer LSTM structure, and a residual correction module. For the temporal fusion features of the input at each time step, it extracts historical states within a delay window to construct the input matrix. A multi-layer gating mechanism dynamically models the state evolution trend, and then a residual network is introduced to perform fine-grained correction on the prediction results, thereby generating a smooth and highly robust delay compensation output. When the prediction error exceeds a threshold, the system automatically triggers an online fine-tuning and update mechanism for the LSTM forget gate and the convolutional kernel of the residual module, achieving adaptive model evolution. This mechanism effectively alleviates the prediction bias caused by communication delays and environmental disturbances, significantly improving the system's accurate prediction capability for future states and the stability of real-time control response in complex, enclosed spaces.

[0048] Optionally, the Smith predictor module includes the following steps:

[0049] Initialize the state buffer and set a fixed system delay step. Spatiotemporal fusion feature vectors of historical time steps are recorded using a FIFO structure. The current buffer length is not less than ;

[0050] At the current time step Extract the time index from the buffer as Historical feature vector , as a delayed reference input;

[0051] Extraction time step to All feature vectors between these points constitute the prediction window sequence: ;

[0052] The prediction window sequence is input into a set of feedforward modeling networks with fixed weight parameters. The modeling networks include: a time difference coding layer, a linear prediction layer, and a state correction layer.

[0053] The time difference coding layer calculates the set of state evolution vectors based on the feature differences between adjacent time steps: All Concatenation serves as the input for state evolution;

[0054] The linear prediction layer feeds the state evolution input into a multi-channel convolutional structure, extracts the temporal transformation pattern, and outputs the predicted state for the current step. and with Jointly constitute the prediction vector: ;

[0055] State correction layer Channel normalization and structural constraint processing are performed, and the corrected predicted state vector is output as the delay reference input of the collaborative compensation network.

[0056] At the current moment and The data is cached together, the state buffer is updated, and the process waits for the next moment to enter the Smith predictor module for loop processing.

[0057] This invention designs a Smith predictor module based on a fixed delay step, which records historical spatiotemporal fusion features in a state buffer. Through a three-layer feedforward structure of time difference encoding, linear convolutional prediction, and normalization correction, it models the temporal evolution trend of the feature sequence, enabling advance estimation of the future state at the current moment. This module not only effectively compensates for control lag caused by system communication delays but also ensures the consistency of the prediction results in channel structure and dynamic characteristics through a state correction layer, thereby improving the responsiveness and control stability of remote control in confined spaces.

[0058] Optionally, step S4 includes the following specific steps:

[0059] S41. Obtain the delay compensation prediction state vector This serves as the system state input at the current moment;

[0060] S42. Measure the network communication delay between the main control terminal's command issuance and feedback signal round-trip in real time using the timestamp mechanism within the device's communication protocol. ;

[0061] S43. Based on the enhanced image sequence from the low-light camera, the visual credibility index is calculated by aggregating spatial gradient variance and noise intensity. ;

[0062] S44, will , , The inputs are respectively into a hybrid control network, which includes a parallel model predictive control branch and an H∞ robust control branch. The two branches independently calculate the control parameters.

[0063] S45. The specific process of the model predictive control branch is as follows:

[0064] S451. Initialize the state space parameter matrix and All are obtained by device identification, and the prediction step size is set. and sampling period ;

[0065] S452, Set the state and input constraint range, respectively , ,in For the physical limit, For the safe zone of the drive, ;

[0066] S453, with Using the initial state, construct a model for prediction: and with the target state For reference, the objective function is optimized as follows:

[0067] ;

[0068] in, This is the weight matrix;

[0069] S454. Solving for the optimal input sequence online using quadratic programming. Actual implementation As the output of the current step model, the remaining input is used for repeated optimization in the next cycle's sliding window.

[0070] The specific process of S46 and H∞ branches is as follows:

[0071] S461, with Define the augmenting system for the current state: ,in Assign a matrix to the perturbation. For environmental disturbance input;

[0072] S462, Assume controlled output: ,in Select a matrix for the controlled variable;

[0073] S463 specifies the H∞ robustness performance index:

[0074] ;

[0075] in, The maximum energy amplification factor set for the design. Controlled output;

[0076] S464. Solve for symmetric positive definite matrices using the linear matrix inequality method. With robust gain , so that:

[0077] ;

[0078] get Then, calculate the H∞ control input in real time: ;

[0079] S47, The fusion layer is based on real-time... and Respectively with threshold Compare and calculate the branch weights of the model. :

[0080] ;

[0081] S48, the final output control parameters are: and will As an execution instruction issued to mechanical equipment.

[0082] This invention introduces a hybrid control network based on a joint input of state, credibility, and delay, constructing a control framework with two parallel branches: Model Predictive Control (MPC) and H∞ Robust Control. These branches dynamically calculate control parameters for model optimization and disturbance-resistant robust scenarios, respectively. In the MPC branch, the system constructs an objective function and physical constraints based on the predicted state, and solves the control sequence through quadratic programming to achieve high-precision tracking. In the H∞ branch, through augmented system modeling and robust performance index setting, it calculates control inputs insensitive to environmental disturbances in real time. Simultaneously, the fusion layer adaptively adjusts the control weights of the two branches based on communication delay and visual credibility indices, achieving dynamic control fusion under multiple operating conditions. This strategy significantly enhances the control stability and system robustness of remote control under conditions of delay fluctuations and perceptual uncertainty, ensuring the precision and continuity of mechanical equipment operation in complex, enclosed environments.

[0083] Optionally, step S5 includes the following specific steps:

[0084] S51, merging the control parameter vector output The mechanical equipment control protocol is used to convert each component into drive commands, forming a drive command sequence. ,in For the first The target command of each execution unit at the current moment. Total number of execution units;

[0085] S52, transmit the drive instruction sequence through the main control terminal communication interface. The data is simultaneously sent to the mechanical equipment, and each execution unit of the mechanical equipment operates within the sampling period. Internal basis received Each action is executed according to a predetermined procedure, including position adjustment, attitude change, or torque output, and all actions are strictly implemented according to the control protocol parameters.

[0086] S53. The mechanical equipment collects the actual operating status of each execution unit through local sensors and records the current position of each execution unit in sequence. Speed ​​of movement acceleration Output force and task completion marker ;

[0087] S54. Mark all collected feedback status data, timestamps and corresponding instruction numbers, package them into real-time feedback data packets, and transmit them back to the main control terminal via the device's uplink communication link.

[0088] This invention maps the control parameter vector output by the hybrid control network component by component into drive commands conforming to the control protocol, and synchronously sends them to each execution unit, ensuring that mechanical equipment completes high-precision position, attitude, and torque actions under different operating conditions. Simultaneously, local sensors on the equipment collect real-time operating status data from each execution unit, adding timestamps and command numbers to form structured feedback packets, which are then transmitted back to the main control unit for closed-loop response. This mechanism not only achieves standardized execution of control commands and high-frequency closed-loop status feedback, but also significantly improves the reliability of command transmission and the completeness of status perception in remote control systems in complex, enclosed spaces.

[0089] Optionally, step S6 includes the following specific steps:

[0090] S61. The main control unit receives status data from the mechanical equipment in each control cycle. And synchronously read the predicted state vector of the previous cycle with delay compensation. ,in The predicted state is the output of the Smith predictor and the collaborative compensation network, with dimensions and Consistent;

[0091] S62. Based on the feedback status data With delay compensation prediction state Calculate the prediction error vector based on the corresponding components. The physical meaning of each component is consistent with the feedback from the equipment;

[0092] S63. Set the prediction error threshold. For the error vector norm and When comparing, If the network enters the adaptive update process, it will proceed; otherwise, the current network parameters will remain unchanged.

[0093] S64. When the prediction error exceeds the threshold, the multi-layer long short-term memory network and residual branch in the collaborative compensation network are invoked. The feedback state data, error vector and historical state of the current period are input into the network. The mini-batch backpropagation algorithm is used to fine-tune the weight matrix of the long short-term memory network and the weight of the residual branch. The update objective is to minimize the prediction error of the next period.

[0094] S65. After completing the online update of network weights, recalculate the delay-compensated prediction state using the latest network parameters, and obtain... This serves as the predictive input for the next control cycle, and the closed-loop process described above continues.

[0095] This invention constructs an error-driven online closed-loop update mechanism. In each control cycle, it compares the feedback state of the mechanical equipment with the delay-compensated prediction state of the previous cycle, calculates a high-dimensional prediction error vector, and determines whether to initiate the adaptive update process of the collaborative compensation network based on whether the error norm exceeds a set threshold. If the error is too large, the system jointly inputs the current feedback state, error information, and historical state into a multi-layer LSTM and residual branch, and fine-tunes its weights online through mini-batch backpropagation, thereby achieving dynamic reconstruction of the prediction model parameters. After the update, the system regenerates the delay-compensated prediction state for the next cycle based on the latest parameters, achieving continuous self-correction and performance evolution. This mechanism effectively improves the prediction stability and control response accuracy of the system under complex environmental disturbances and task changes, ensuring high adaptability and robustness during remote control.

[0096] Optionally, step S7 includes the following specific steps:

[0097] S71. The master control unit acquires the updated delay compensation prediction status and the fused control parameters in each control cycle.

[0098] S72. Generate instructions by combining the delay compensation prediction state and control parameters, and map them component by component into a sequence of drive instructions for each execution unit according to the equipment control protocol, wherein each drive instruction corresponds to a specific target position, velocity, acceleration or output force parameter.

[0099] S73. Verify each drive command parameter by combining the delay compensation prediction state, compare the target parameter with the physical boundary of the device, truncate the parameters that exceed the allowable range according to the upper and lower limits, and reorganize all the corrected parameters into the final execution command sequence.

[0100] S74. The final execution instruction sequence is sent to the mechanical equipment through the main control terminal communication interface. Each execution unit completes the target action in sequence. The action content includes spatial position movement, attitude change and torque output. All actions strictly correspond to their respective instruction parameters.

[0101] S75. After the mechanical equipment completes the command action, the local sensors collect the actual position, speed, acceleration, output force and task completion status of each execution unit, and package the feedback status data with timestamp and command number, and transmit it back to the main control terminal through the equipment communication link.

[0102] This invention acquires updated delay-compensated predicted states and fused control parameters within each control cycle, and uses a control protocol to map these parameters component-by-component into a refined sequence of drive commands. Furthermore, it performs real-time verification and truncation correction of each target parameter based on the physical boundaries of the equipment, ensuring that the generated final command sequence is controllable and executable within a safe range. This command sequence is synchronously distributed to each execution unit, achieving precise position, attitude, and torque manipulation. After task execution, the mechanical equipment collects feedback data from each action through local sensors, adds timestamps and numbers, packages it, and sends it back to the main control unit to form a closed loop. This mechanism effectively guarantees the executability and safety of predictive control results on the actual equipment, improving the accuracy, stability, and complete data traceability of remote control tasks.

[0103] The beneficial effects of this invention are:

[0104] This invention effectively solves the problems of unclear imaging, isolated sensor data, and inaccurate status recognition in traditional remote control systems for mechanical equipment in enclosed spaces under low-light conditions by employing low-light image enhancement, multi-source data spatiotemporal fusion, and collaborative compensation prediction mechanisms. By using a residual dense generative adversarial network to improve image quality and deeply fusing image, distance, and motion state data in a graph convolutional network, it significantly enhances environmental perception capabilities and the accuracy of equipment status recognition.

[0105] The invention introduces a Smith predictor, a multi-layer long short-term memory network, and a residual correction structure to achieve efficient delay compensation and state prediction, addressing network latency and motion dynamics. This mechanism effectively compensates for the real-time performance and stability deficiencies of traditional feedback control under high latency and signal disturbance conditions, ensuring continuous and stable operation of the system under complex operating conditions.

[0106] The hybrid control network employs an adaptive fusion approach combining model predictive control and H∞ robust control, dynamically adjusting control parameter weights by integrating network latency and visual reliability metrics. Compared to traditional single control strategies, this scheme exhibits stronger environmental adaptability and robustness, effectively suppressing system oscillations, improving control accuracy, and significantly enhancing the safety and reliability of remote equipment operation. Attached Figure Description

[0107] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0108] Figure 1 This is a flowchart of a remote control system for enclosed space mechanical equipment based on deep learning, as proposed in this invention.

[0109] Figure 2 This is a block diagram illustrating the principle of the collaborative compensation network delay compensation and prediction mechanism proposed in this invention.

[0110] Figure 3 This is a flowchart of the adaptive fusion of model predictive control and H∞ robust control in the hybrid control network proposed in this invention. Detailed Implementation

[0111] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0112] refer to Figures 1-3 A remote control system for mechanical equipment in enclosed spaces based on deep learning includes the following steps:

[0113] The data acquisition module is used to simultaneously acquire images, environmental distance and motion status data through a low-light camera, sonar sensor and inertial measurement unit, and complete timestamp alignment;

[0114] The data fusion module is used to perform low-light enhancement on image data. It inputs the enhanced image, distance data, and motion state into the graph convolutional network and outputs spatiotemporal fusion features.

[0115] The collaborative compensation module receives spatiotemporal fusion features, uses the Smith predictor, multi-layer long short-term memory network and residual correction structure to perform delay compensation, and outputs the predicted state.

[0116] The hybrid control network module includes a model predictive control branch and an H∞ robust control branch, which independently calculate candidate control parameters based on the predicted state, network latency, and visual credibility index, respectively. The fusion layer adaptively adjusts the weights based on network latency and visual credibility, and weights and fuses the control parameters output by the two branches to generate the final control parameters.

[0117] The instruction issuance and feedback module is used to map the final control parameters into drive instructions for each execution unit of the mechanical equipment, issue and collect the actual actions and feedback data of the equipment, and send them back to the main control terminal.

[0118] The online adaptive update module is used to calculate the error based on the feedback data and the predicted state, and to fine-tune the parameters of the collaborative compensation network online when the error exceeds the threshold.

[0119] In this embodiment, the modules are interconnected using the following method:

[0120] S1. The low-light camera collects image data, the sonar sensor collects environmental distance data, and the inertial measurement unit collects attitude and acceleration data.

[0121] S2. Image data is input into the residual dense generative adversarial network to generate an enhanced image. The distance data between the enhanced image and the environment, as well as the attitude and acceleration data, are input into the graph convolutional network to form a spatiotemporal fusion feature set.

[0122] S3. The spatiotemporal fusion feature input collaborative compensation network couples the Smith predictor delay compensation structure with a multi-layer long short-term memory network and residual units, collaboratively updates the internal state and uses the prediction error to drive weight adjustment, and outputs the delay-compensated prediction state.

[0123] S4. The delay compensation prediction state, network delay and visual credibility index are input into the hybrid control network. The hybrid control network adaptively fuses the network delay and visual credibility between the model branch and the H∞ branch, and outputs control parameters.

[0124] S5. Control parameters are mapped to preliminary execution instructions and sent to mechanical equipment. The mechanical equipment executes the instructions and returns real-time feedback.

[0125] S6. Calculate the error between the real-time feedback and the delay compensation prediction status. When the error exceeds the preset threshold, fine-tune the long short-term memory network and residual branch in the collaborative compensation network online and update the delay compensation prediction status.

[0126] S7. Calculate the final execution command based on the updated delay compensation prediction state and control parameters, and send it to the mechanical equipment.

[0127] In this embodiment, step S1 includes the following specific steps:

[0128] S11. The low-light camera acquires image frames according to a fixed sampling period. The image frames have a uniform resolution and channel structure. Continuous image frames form an image data sequence in chronological order. The time interval between frames is not greater than the preset image refresh interval.

[0129] S12. The sonar sensor array synchronously transmits detection signals in each cycle and measures the propagation delay of echoes in each direction. Based on the preset speed, it calculates the environmental distance values ​​in multiple directions and generates the environmental distance data vector at the current moment.

[0130] S13. The inertial measurement unit periodically records attitude and acceleration information. The attitude is represented by Euler angles about the X, Y, and Z axes, and the acceleration is composed of linear accelerations along the X, Y, and Z axes. The six physical quantities together form the motion state vector at the current moment.

[0131] S14. Image data, environmental distance data, and attitude acceleration data are labeled with timestamps and then aligned. Image frames, environmental distance vectors, and motion state vectors acquired at the same time point are extracted and combined into ternary acquisition samples. Each ternary sampling result constitutes a time-series acquisition dataset, which serves as the input for subsequent processing steps.

[0132] In this embodiment, step S2 includes the following specific steps:

[0133] S21. Input each frame of image data from the time-series acquisition dataset into the residual dense generative adversarial network sequentially to perform pixel-level enhancement processing on the low-light images, and output the enhanced image frame sequence. Each enhanced image frame maintains the same resolution and number of channels as the original. The enhancement process includes multi-scale feature extraction, dense residual connections, and adversarial optimization of the discriminant network.

[0134] S22, at every moment Extract enhanced image frames The three types of data—environmental distance data vector, motion state vector, and so on—are used together as the node feature input to the graph convolutional network.

[0135] S23. In the graph convolutional network, the enhanced image frame is used as the visual node, each component of the environmental distance data is used as the spatial perception node, and the motion state vector is used as the pose node. An undirected weighted graph structure is constructed based on the preset spatial adjacency relationship, and the actual physical or spatial connection between nodes is reflected by the edge weight.

[0136] S24. Perform multi-layer graph convolution operations on all node features in the graph structure to achieve deep fusion of enhanced image features, spatial distance features, and motion state features. After fusion, the node features at each time step... Aggregates into spatiotemporal fusion feature vectors ;

[0137] S25. Arrange all spatiotemporal fusion feature vectors in chronological order. .

[0138] In this embodiment, step S3 includes the following specific steps:

[0139] S31. Input the feature vectors of each time step in the temporal fusion feature set into the collaborative compensation network in chronological order, and initialize the network's state buffer and memory units.

[0140] S32. The collaborative compensation network consists of a Smith predictor module, a multi-layer long short-term memory network structure, and a residual correction module. The Smith predictor module is used to extract historical features with a fixed delay length, the memory network is used to extract cross-temporal state evolution trends, and the residual correction module is used to construct and correct the output state.

[0141] S33, at every moment The Smith predictor module is invoked to read the time points from the stored feature sequences. eigenvectors ,in To preset the system delay steps, the current feature vector and Concatenate to form the input matrix: ;

[0142] S34, Input matrix The input is fed into a multi-layer long short-term memory network structure, and each layer processes the input layer by layer in the following manner:

[0143] The first layer of the Long Short-Term Memory network receives... The current state is updated based on the hidden state and the cell state of the previous time step;

[0144] Each layer of the Long Short-Term Memory (LSTM) network executes a gating mechanism, including the activation operations of the input gate, forget gate, and output gate, and updates the hidden state vector at the current time step. With the unit state vector ,in Indicates the first layer;

[0145] The last layer outputs the hidden state vector. , as the main channel output of the predicted state;

[0146] S35, will With delay characteristics Perform difference calculations to obtain the residual vector: The residual vector is input into the residual correction module, which is a residual network containing multiple layers of one-dimensional convolutions and nonlinear activation structures. This module performs feature transformation on the residual vector and... Element-wise addition generates a preliminary delay-compensated output vector. ;

[0147] S36, will The original fused feature vector at the current time By performing dimension-by-dimensional interpolation, the prediction error vector is obtained: If the error norm Exceeding the set threshold Then, the online weight update mechanism is invoked to perform a small-batch reverse adjustment operation on the forget gate weight matrix and the convolution kernel parameters in the residual module of the long short-term memory network. The update step size is calculated according to the set dynamic learning rate adjustment strategy.

[0148] S37. Output the updated delay compensation prediction state vector. And cache it in the predicted state sequence.

[0149] In this embodiment, the Smith predictor module includes the following steps:

[0150] Initialize the state buffer and set a fixed system delay step. Spatiotemporal fusion feature vectors of historical time steps are recorded using a FIFO structure. The current buffer length is not less than ;

[0151] At the current time step Extract the time index from the buffer as Historical feature vector , as a delayed reference input;

[0152] Extraction time step to All feature vectors between these points constitute the prediction window sequence: ;

[0153] The prediction window sequence is input into a set of feedforward modeling networks with fixed weight parameters. The modeling networks include: a time difference coding layer, a linear prediction layer, and a state correction layer.

[0154] The time difference coding layer calculates the set of state evolution vectors based on the feature differences between adjacent time steps: All Concatenation serves as the input for state evolution;

[0155] The linear prediction layer feeds the state evolution input into a multi-channel convolutional structure, extracts the temporal transformation pattern, and outputs the predicted state for the current step. and with Jointly constitute the prediction vector: ;

[0156] State correction layer Channel normalization and structural constraint processing are performed, and the corrected predicted state vector is output as the delay reference input of the collaborative compensation network.

[0157] At the current moment and The data is cached together, the state buffer is updated, and the process waits for the next moment to enter the Smith predictor module for loop processing.

[0158] In this embodiment, step S4 includes the following specific steps:

[0159] S41. Obtain the delay compensation prediction state vector This serves as the system state input at the current moment;

[0160] S42. Measure the network communication delay between the main control terminal's command issuance and feedback signal round-trip in real time using the timestamp mechanism within the device's communication protocol. ;

[0161] S43. Based on the enhanced image sequence from the low-light camera, the visual credibility index is calculated by aggregating spatial gradient variance and noise intensity. ;

[0162] S44, will , , The inputs are respectively into a hybrid control network, which includes a parallel model predictive control branch and an H∞ robust control branch. The two branches independently calculate the control parameters.

[0163] S45. The specific process of the model predictive control branch is as follows:

[0164] S451. Initialize the state space parameter matrix and All are obtained by device identification, and the prediction step size is set. and sampling period ;

[0165] S452, Set the state and input constraint range, respectively , ,in For the physical limit, For the safe zone of the drive, ;

[0166] S453, with Using the initial state, construct a model for prediction: and with the target state For reference, the objective function is optimized as follows:

[0167] ;

[0168] in, This is the weight matrix;

[0169] S454. Solving for the optimal input sequence online using quadratic programming. Actual implementation As the output of the current step model, the remaining input is used for repeated optimization in the next cycle's sliding window.

[0170] The specific process of S46 and H∞ branches is as follows:

[0171] S461, with Define the augmenting system for the current state: ,in Assign a matrix to the perturbation. For environmental disturbance input;

[0172] S462, Assume controlled output: ,in Select a matrix for the controlled variable;

[0173] S463 specifies the H∞ robustness performance index:

[0174] ;

[0175] in, The maximum energy amplification factor set for the design. Controlled output;

[0176] S464. Solve for symmetric positive definite matrices using the linear matrix inequality method. With robust gain , so that:

[0177] ;

[0178] get Then, calculate the H∞ control input in real time: ;

[0179] S47, The fusion layer is based on real-time... and Respectively with threshold Compare and calculate the branch weights of the model. :

[0180] ;

[0181] S48, the final output control parameters are: and will As an execution instruction issued to mechanical equipment.

[0182] In this embodiment, step S5 includes the following specific steps:

[0183] S51, merging the control parameter vector output The mechanical equipment control protocol is used to convert each component into drive commands, forming a drive command sequence. ,in For the first The target command of each execution unit at the current moment. Total number of execution units;

[0184] S52, transmit the drive instruction sequence through the main control terminal communication interface. The data is simultaneously sent to the mechanical equipment, and each execution unit of the mechanical equipment operates within the sampling period. Internal basis received Each action is executed according to a predetermined procedure, including position adjustment, attitude change, or torque output, and all actions are strictly implemented according to the control protocol parameters.

[0185] S53. The mechanical equipment collects the actual operating status of each execution unit through local sensors and records the current position of each execution unit in sequence. Speed ​​of movement acceleration Output force and task completion marker ;

[0186] S54. Mark all collected feedback status data, timestamps and corresponding instruction numbers, package them into real-time feedback data packets, and transmit them back to the main control terminal via the device's uplink communication link.

[0187] In this embodiment, step S6 includes the following specific steps:

[0188] S61. The main control unit receives status data from the mechanical equipment in each control cycle. And synchronously read the predicted state vector of the previous cycle with delay compensation. ,in The predicted state is the output of the Smith predictor and the collaborative compensation network, with dimensions and Consistent;

[0189] S62. Based on the feedback status data With delay compensation prediction state Calculate the prediction error vector based on the corresponding components. The physical meaning of each component is consistent with the feedback from the equipment;

[0190] S63. Set the prediction error threshold. For the error vector norm and When comparing, If the network enters the adaptive update process, it will proceed; otherwise, the current network parameters will remain unchanged.

[0191] S64. When the prediction error exceeds the threshold, the multi-layer long short-term memory network and residual branch in the collaborative compensation network are invoked. The feedback state data, error vector and historical state of the current period are input into the network. The mini-batch backpropagation algorithm is used to fine-tune the weight matrix of the long short-term memory network and the weight of the residual branch. The update objective is to minimize the prediction error of the next period.

[0192] S65. After completing the online update of network weights, recalculate the delay-compensated prediction state using the latest network parameters, and obtain... This serves as the predictive input for the next control cycle, and the closed-loop process described above continues.

[0193] In this embodiment, step S7 includes the following specific steps:

[0194] S71. The master control unit acquires the updated delay compensation prediction status and the fused control parameters in each control cycle.

[0195] S72. Generate instructions by combining the delay compensation prediction state and control parameters, and map them component by component into a sequence of drive instructions for each execution unit according to the equipment control protocol, wherein each drive instruction corresponds to a specific target position, velocity, acceleration or output force parameter.

[0196] S73. Verify each drive command parameter by combining the delay compensation prediction state, compare the target parameter with the physical boundary of the device, truncate the parameters that exceed the allowable range according to the upper and lower limits, and reorganize all the corrected parameters into the final execution command sequence.

[0197] S74. The final execution instruction sequence is sent to the mechanical equipment through the main control terminal communication interface. Each execution unit completes the target action in sequence. The action content includes spatial position movement, attitude change and torque output. All actions strictly correspond to their respective instruction parameters.

[0198] S75. After the mechanical equipment completes the command action, the local sensors collect the actual position, speed, acceleration, output force and task completion status of each execution unit, and package the feedback status data with timestamp and command number, and transmit it back to the main control terminal through the equipment communication link.

[0199] Example 1:

[0200] To further verify the applicability and reliability of this invention in complex, confined spaces, we selected a high-risk underground maintenance base as the experimental site. This base consists of multiple maintenance passages approximately 120 meters deep and five enclosed chambers. The interior is consistently in a low-light (<5 lx) environment, with walls primarily of metal and numerous pipelines and obstacles. Conventional wireless communication experiences jitter delays of 20ms–180ms during movement. Previously, this base used manual remote control combined with a monocular camera and a limited number of ultrasonic sensors for maintenance operations. Specifically, the images lacked detail and depth information; operators had to repeatedly adjust the robotic arm's posture when network latency increased to complete a single bolt removal or installation task, averaging about 55 minutes; if network latency spiked to over 150ms, the maximum positioning error at the robotic arm's end effector could reach 8mm, with a misoperation rate as high as 7%–9%; and fault confirmation was often delayed by more than 10 seconds, posing significant safety risks and maintenance costs.

[0201] After the system of this invention is put into operation, low-light cameras, sonar arrays, and inertial measurement units are first fixed on four mobile robots and four multi-axis robotic arms, respectively. The sampling frequency of all sensors is set to 30Hz and automatically aligned by timestamps. After the system starts, the residual dense generative adversarial network enhances the original low-light image in real time, usually restoring important details such as wall metal reflections and weld textures within 0.3s. The enhanced image, environmental distance, and attitude vector are input into the graph convolutional network, which completes multimodal topology construction and spatiotemporal fusion feature extraction at the 1ms level. The fused features are fed into the collaborative compensation network in real time, where the Smith predictor extracts a fixed delay window (about 50ms) from the historical feature sequence and, together with two layers of LSTM, predicts the state within the next 300ms. The residual correction module performs dimension-wise correction on the predicted value, keeping the output state error within 3%, and can still give smooth instructions even when the network latency fluctuates to 180ms. The hybrid control network dynamically adjusts the weights of the MPC and H∞ branches based on the current communication latency and visual reliability (normalized index range 0.65–0.93), achieving adaptive fusion of control parameters before finely distributing them at 10ms intervals in the local driver layer. After each robotic arm or robot executes the command, it sends real-time feedback data back to the master control unit. If the prediction error norm exceeds 5%, the system immediately triggers online micro-batch gradient updates, adjusting the LSTM forget gate weights and residual convolution kernels with small steps ranging from 0.001 to 0.005 to ensure the model continuously adapts to gradual environmental changes.

[0202] During a two-month (approximately 720 hours) continuous operation experiment, we selected eight sets of equipment (equipment 1-4 were all channel mobile robots, and equipment 5-8 were all enclosed compartment robotic arms) to complete the same set of bolt disassembly and sealing maintenance tasks. The experiment simultaneously recorded key performance indicators before system deployment (baseline) and after deployment (this invention), including image clarity, average operating error, task completion time, fault response time, and misoperation rate. All data are statistical averages under typical scenarios, possessing high realism and representativeness. Table 1 lists the detailed comparison results.

[0203] Table 1 Performance Comparison of Equipment Before and After Remote Control Experiment in Confined Space

[0204] Equipment Number Scenario Type Clarity (front→back) Mean Error (mm) (front→back) Task Time (min) (front→back) Fault Response (s) (front→back) Misoperation Rate (%) (front→back) 1 Passage (mobile robot) 0.30→0.86 5.0→3.2 54.2→44.6 19.8→8.5 8.2→1.5 2 Cabin (robot arm) 0.29→0.83 5.2→2.9 53.5→43.8 19.4→8.1 7.9→1.2 3 Passage (mobile robot) 0.31→0.88 4.8→3.0 55.0→45.2 19.6→7.9 8.5→1.3 4 Cabin (robot arm) 0.33→0.87 5.1→3.1 52.8→42.5 19.2→8.3 8.0→1.4 5 Passage (mobile robot) 0.28→0.80 5.3→3.4 55.5→46.0 20.1→8.7 9.1→1.7 6 Cabin (robot arm) 0.32→0.90 4.7→2.8 51.9→41.9 19.0→7.8 7.5→1.1 7 Passage (mobile robot) 0.30→0.84 5.0→3.3 54.0→45.0 19.7→8.4 8.3→1.6 8 Cabin (robot arm) 0.31→0.86 4.9→2.7 52.4→42.8 19.5→8.0 7.8→1.3

[0205] As can be clearly seen from Table 1, the system of the present invention shows significant improvement in the following aspects after deployment: Image clarity: The average value before deployment was 0.30-0.33, which was improved to 0.80-0.90 after deployment, with an average increase of about 185%. Metal details and weld edges can be clearly restored in low light environments, which greatly reduces the operator's operational errors caused by visual blur.

[0206] Average operational error: Before deployment, the error of each device was concentrated in the range of 4.7-5.3mm. After deployment, it decreased to 2.7-3.4mm, an average reduction of about 40%. This is mainly due to the collaborative compensation of the Smith-LSTM prediction and residual correction modules, which effectively alleviated the instruction latency caused by network jitter. Task completion time: Before deployment, the average time was between 52.8 and 55.5 minutes. After deployment, it was shortened to 41.9-46.0 minutes, an average reduction of about 15%-20%. Image enhancement and multimodal fusion made task execution smoother and reduced the number of repeated fine-tuning. Fault response time: Before deployment, it usually took 19-20 seconds to identify and respond to anomalies. After deployment, the control system calculated the prediction error in a timely manner and completed the update and intervention within 8-9 seconds, reducing the response time by an average of 42%. Error rate: Before deployment, the error rate of each device was as high as 7.9%-9.1%. After deployment, it decreased to 1.1%-1.7%, an average reduction of about 80%, improving operational safety and task success rate.

[0207] The comparative data above are derived from continuous experimental samples under the same tasks, environmental conditions, and operating procedures. The data analysis results fully demonstrate that the system of this invention can significantly improve the accuracy, efficiency, and robustness of remote control under conditions of low illumination, multiple obstructions, and unstable networks in enclosed spaces. Furthermore, it is of great significance in reducing the burden on maintenance personnel and mitigating safety risks. This invention not only performs exceptionally well in single task execution, but its online adaptive learning capability also enables continuous optimization of the system during long-term operation, further improving the system's intelligence and equipment lifespan.

[0208] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A remote control system for enclosed space mechanical equipment based on deep learning, characterized in that, include: The data acquisition module is used to simultaneously acquire images, environmental distance and motion status data through a low-light camera, sonar sensor and inertial measurement unit, and complete timestamp alignment; The data fusion module is used to perform low-light enhancement on image data. It inputs the enhanced image, distance data, and motion state into the graph convolutional network and outputs spatiotemporal fusion features. The collaborative compensation module receives spatiotemporal fusion features, uses the Smith predictor, multi-layer long short-term memory network and residual correction structure to perform delay compensation, and outputs the predicted state. The hybrid control network module includes a model predictive control branch and an H∞ robust control branch, which independently calculate candidate control parameters based on the predicted state, network latency, and visual credibility index, respectively. The fusion layer adaptively adjusts the weights based on network latency and visual credibility, and weights and fuses the control parameters output by the two branches to generate the final control parameters. The instruction issuance and feedback module is used to map the final control parameters into drive instructions for each execution unit of the mechanical equipment, issue and collect the actual actions and feedback data of the equipment, and send them back to the main control terminal. The online adaptive update module is used to calculate the error based on the feedback data and the predicted state, and to fine-tune the parameters of the collaborative compensation network online when the error exceeds the threshold.

2. A remote control system for enclosed space mechanical equipment based on deep learning, characterized in that, The modules are connected in the following way: S1. The low-light camera collects image data, the sonar sensor collects environmental distance data, and the inertial measurement unit collects attitude and acceleration data. S2. Image data is input into the residual dense generative adversarial network to generate an enhanced image. The distance data between the enhanced image and the environment, as well as the attitude and acceleration data, are input into the graph convolutional network to form a spatiotemporal fusion feature set. S3. The spatiotemporal fusion feature input collaborative compensation network couples the Smith predictor delay compensation structure with a multi-layer long short-term memory network and residual units, collaboratively updates the internal state and uses the prediction error to drive weight adjustment, and outputs the delay-compensated prediction state. S4. The delay compensation prediction state, network delay and visual credibility index are input into the hybrid control network. The hybrid control network adaptively fuses the network delay and visual credibility between the model branch and the H∞ branch, and outputs control parameters. S5. Control parameters are mapped to preliminary execution instructions and sent to mechanical equipment. The mechanical equipment executes the instructions and returns real-time feedback. S6. Calculate the error between the real-time feedback and the delay compensation prediction status. When the error exceeds the preset threshold, fine-tune the long short-term memory network and residual branch in the collaborative compensation network online and update the delay compensation prediction status. S7. Calculate the final execution command based on the updated delay compensation prediction state and control parameters, and send it to the mechanical equipment.

3. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S1 includes the following specific steps: S11. The low-light camera acquires image frames according to a fixed sampling period. The image frames have a uniform resolution and channel structure. Continuous image frames form an image data sequence in chronological order. The time interval between frames is not greater than the preset image refresh interval. S12. The sonar sensor array synchronously transmits detection signals in each cycle and measures the propagation delay of echoes in each direction. Based on the preset speed, it calculates the environmental distance values ​​in multiple directions and generates the environmental distance data vector at the current moment. S13. The inertial measurement unit periodically records attitude and acceleration information. The attitude is represented by Euler angles about the X, Y, and Z axes, and the acceleration is composed of linear accelerations along the X, Y, and Z axes. The six physical quantities together form the motion state vector at the current moment. S14. Image data, environmental distance data, and attitude acceleration data are labeled with timestamps and then aligned. Image frames, environmental distance vectors, and motion state vectors acquired at the same time point are extracted and combined into ternary acquisition samples. Each ternary sampling result constitutes a time-series acquisition dataset, which serves as the input for subsequent processing steps.

4. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S2 includes the following specific steps: S21. Input each frame of image data from the time-series acquisition dataset into the residual dense generative adversarial network sequentially to perform pixel-level enhancement processing on the low-light images, and output the enhanced image frame sequence. Each enhanced image frame maintains the same resolution and number of channels as the original. The enhancement process includes multi-scale feature extraction, dense residual connections, and adversarial optimization of the discriminant network. S22, at every moment Extract enhanced image frames The three types of data—environmental distance data vector, motion state vector, and so on—are used together as the node feature input to the graph convolutional network. S23. In the graph convolutional network, the enhanced image frame is used as the visual node, each component of the environmental distance data is used as the spatial perception node, and the motion state vector is used as the pose node. An undirected weighted graph structure is constructed based on the preset spatial adjacency relationship, and the actual physical or spatial connection between nodes is reflected by the edge weight. S24. Perform multi-layer graph convolution operations on all node features in the graph structure to achieve deep fusion of enhanced image features, spatial distance features, and motion state features. After fusion, the node features at each time step... Aggregates into spatiotemporal fusion feature vectors ; S25. Arrange all spatiotemporal fusion feature vectors in chronological order. This forms a temporal fusion feature set.

5. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S3 includes the following specific steps: S31. Input the feature vectors of each time step in the temporal fusion feature set into the collaborative compensation network in chronological order, and initialize the network's state buffer and memory units. S32. The collaborative compensation network consists of a Smith predictor module, a multi-layer long short-term memory network structure, and a residual correction module. The Smith predictor module is used to extract historical features with a fixed delay length, the memory network is used to extract cross-temporal state evolution trends, and the residual correction module is used to construct and correct the output state. S33, at every moment The Smith predictor module is invoked to read the time points from the stored feature sequences. eigenvectors ,in To preset the system delay steps, the current feature vector and Concatenate to form the input matrix: ; S34, Input matrix The input is fed into a multi-layer long short-term memory network structure, and each layer processes the input layer by layer in the following manner: The first layer of the Long Short-Term Memory network receives... The current state is updated based on the hidden state and the cell state of the previous time step; Each layer of the Long Short-Term Memory (LSTM) network executes a gating mechanism, including the activation operations of the input gate, forget gate, and output gate, and updates the hidden state vector at the current time step. With the unit state vector ,in Indicates the first layer; The last layer outputs the hidden state vector. , as the main channel output of the predicted state; S35, will With delay characteristics Perform difference calculations to obtain the residual vector: The residual vector is input into the residual correction module, which is a residual network containing multiple layers of one-dimensional convolutions and nonlinear activation structures. This module performs feature transformation on the residual vector and... Element-wise addition generates a preliminary delay-compensated output vector. ; S36, will The original fused feature vector at the current time By performing dimension-by-dimensional interpolation, the prediction error vector is obtained: If the error norm Exceeding the set threshold Then, the online weight update mechanism is invoked to perform a small-batch reverse adjustment operation on the forget gate weight matrix and the convolution kernel parameters in the residual module of the long short-term memory network. The update step size is calculated according to the set learning rate adjustment strategy. S37. Output the updated delay compensation prediction state vector. And cache it in the predicted state sequence.

6. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, The Smith predictor module includes the following steps: Initialize the state buffer and set a fixed system delay step. Spatiotemporal fusion feature vectors of historical time steps are recorded using a FIFO structure. The current buffer length is not less than ; At the current time step Extract the time index from the buffer as Historical feature vector , as a delayed reference input; Extraction time step to All feature vectors between these points constitute the prediction window sequence: ; The prediction window sequence is input into a set of feedforward modeling networks with fixed weight parameters. The modeling networks include: a time difference coding layer, a linear prediction layer, and a state correction layer. The time difference coding layer calculates the set of state evolution vectors based on the feature differences between adjacent time steps: All Concatenation serves as the input for state evolution; The linear prediction layer feeds the state evolution input into a multi-channel convolutional structure, extracts the temporal transformation pattern, and outputs the predicted state for the current step. and with Jointly constitute the prediction vector: ; State correction layer Channel normalization and structural constraint processing are performed, and the corrected predicted state vector is output as the delay reference input of the collaborative compensation network. At the current moment and The data is cached together, the state buffer is updated, and the process waits for the next moment to enter the Smith predictor module for loop processing.

7. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S4 includes the following specific steps: S41. Obtain the delay compensation prediction state vector This serves as the system state input at the current moment; S42. Measure the network communication delay between the main control terminal's command issuance and feedback signal round-trip in real time using the timestamp mechanism within the device's communication protocol. ; S43. Based on the enhanced image sequence from the low-light camera, the visual credibility index is calculated by aggregating spatial gradient variance and noise intensity. ; S44, will , , The inputs are respectively into a hybrid control network, which includes a parallel model predictive control branch and an H∞ robust control branch. The two branches independently calculate the control parameters. S45. The specific process of the model predictive control branch is as follows: S451. Initialize the state space parameter matrix and All are obtained by device identification, and the prediction step size is set. and sampling period ; S452, Set the state and input constraint range, respectively , ,in For the physical limit, For the safe zone of the drive, ; S453, with Using the initial state, construct a model for prediction: and with the target state For reference, optimize the following objective function: ； in, This is the weight matrix; S454. Solving for the optimal input sequence online using quadratic programming. Actual implementation As the output of the current step model, the remaining input is used for repeated optimization in the next cycle's sliding window. The specific process of S46 and H∞ branches is as follows: S461, with Define the augmenting system for the current state: ,in Assign a matrix to the perturbation. For environmental disturbance input; S462, Assume controlled output: ,in Select a matrix for the controlled variable; S463 specifies the H∞ robustness performance index: ； in, The maximum energy amplification factor set for the design. Controlled output; S464. Solve for symmetric positive definite matrices using the linear matrix inequality method. With robust gain , so that: ； get Then, the H∞ control input is calculated in real time: ; S47, The fusion layer is based on real-time... and Respectively with threshold Compare and calculate the branch weights of the model. : ； S48, the final output control parameters are: and will As an execution instruction issued to mechanical equipment.

8. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S5 includes the following specific steps: S51, merging the control parameter vector output The mechanical equipment control protocol is used to convert each component into drive commands, forming a drive command sequence. ,in For the first The target command of each execution unit at the current moment. Total number of execution units; S52, transmit the drive instruction sequence through the main control terminal communication interface. The data is simultaneously sent to the mechanical equipment, and each execution unit of the mechanical equipment operates within the sampling period. Internal basis received Each action is executed according to a predetermined procedure, including position adjustment, attitude change, or torque output, and all actions are strictly implemented according to the control protocol parameters. S53. The mechanical equipment collects the actual operating status of each execution unit through local sensors and records the current position of each execution unit in sequence. Speed ​​of movement acceleration Output force and task completion marker ; S54. Mark all collected feedback status data, timestamps and corresponding instruction numbers, package them into real-time feedback data packets, and transmit them back to the main control terminal via the device's uplink communication link.

9. The remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S6 includes the following specific steps: S61. The main control unit receives status data from the mechanical equipment in each control cycle. And synchronously read the predicted state vector of the previous cycle with delay compensation. ,in The predicted state is the output of the Smith predictor and the collaborative compensation network, with dimensions and Consistent; S62. Based on the feedback status data With delay compensation prediction state Calculate the prediction error vector based on the corresponding components. The physical meaning of each component is consistent with the feedback from the equipment; S63. Set the prediction error threshold. For the error vector norm and When comparing, If the network enters the adaptive update process, it will proceed; otherwise, the current network parameters will remain unchanged. S64. When the prediction error exceeds the threshold, the multi-layer long short-term memory network and residual branch in the collaborative compensation network are invoked. The feedback state data, error vector and historical state of the current period are input into the network. The mini-batch backpropagation algorithm is used to fine-tune the weight matrix of the long short-term memory network and the weight of the residual branch. The update objective is to minimize the prediction error of the next period. S65. After completing the online update of network weights, recalculate the delay-compensated prediction state using the latest network parameters, and obtain... This serves as the predictive input for the next control cycle, and the closed-loop process described above continues.

10. A remote control system for enclosed space mechanical equipment based on deep learning according to claim 2, characterized in that, S7 includes the following specific steps: S71. The master control unit acquires the updated delay compensation prediction status and the fused control parameters in each control cycle. S72. Generate instructions by combining the delay compensation prediction state and control parameters, and map them component by component into a sequence of drive instructions for each execution unit according to the equipment control protocol, wherein each drive instruction corresponds to a specific target position, velocity, acceleration or output force parameter. S73. Verify each drive command parameter by combining the delay compensation prediction state, compare the target parameter with the physical boundary of the device, truncate the parameters that exceed the allowable range according to the upper and lower limits, and reorganize all the corrected parameters into the final execution command sequence. S74. The final execution instruction sequence is sent to the mechanical equipment through the main control terminal communication interface. Each execution unit completes the target action in sequence. The action content includes spatial position movement, attitude change and torque output. All actions strictly correspond to their respective instruction parameters. S75. After the mechanical equipment completes the command action, the local sensors collect the actual position, speed, acceleration, output force and task completion status of each execution unit, and package the feedback status data with timestamp and command number, and transmit it back to the main control terminal through the equipment communication link.

Images