An underground shovel remote control system

Abstract

The application discloses a kind of underground shovel loader remote control systems, belong to underground engineering equipment remote control technical field, the system includes remote cockpit subsystem, vehicle-mounted subsystem, communication subsystem and cloud digital twin and collaborative decision-making platform;Vehicle-mounted subsystem is used to collect the vehicle state data and operation environment perception data of underground shovel loader and upload to cloud end;Cloud digital twin and collaborative decision-making platform is based on the data constructs and real-time updates the digital twin model synchronized with physical shovel loader, and in the model, the original control instruction input by remote cockpit is pre-played simulation, and the control instruction is corrected or optimized according to the pre-play result and issued for execution.The system combines multi-modal perception fusion, tactile and somatosensory feedback and local autonomous operation mechanism under communication anomaly, improves the safety and reliability of underground shovel loader remote control operation.

Description

Technical Field

[0001] This invention relates to the field of remote control technology for underground engineering equipment, and more specifically, to a remote control system for an underground loader. Background Technology

[0002] Underground loaders, commonly used loading and transport equipment in mines and underground engineering, typically operate in complex environments characterized by limited space, insufficient lighting, high dust concentrations, and challenging working conditions. To reduce the workload and safety risks for operators in these high-risk environments, remote control of underground loaders has been increasingly adopted in existing technologies, allowing operators to move from the underground work site to the surface or a relatively safe control area. However, most existing remote control systems for underground loaders still rely on the traditional "direct command issuance and immediate vehicle execution" control mode, which remains significantly inadequate in terms of control safety and operational reliability in complex underground environments.

[0003] On the one hand, existing remote control systems typically rely on onboard camera footage and basic status information to provide feedback to operators. Operators must then judge the vehicle's posture, surrounding environment, and potential risks based on limited visual information and delayed feedback. In narrow underground tunnels with small turning radii, severe obstructions, or pervasive dust, single or weakly fused sensor information is insufficient to accurately reflect the actual working environment. This can easily lead to operators misjudging obstacle distances, vehicle posture, and trajectory, thereby increasing the risk of vehicle collisions, jamming, or overturning. Although existing technologies incorporate multiple sensors, they often employ fixed weights or simple superposition methods for fusion, making it difficult to adapt to fluctuations in perception reliability caused by dynamic changes in underground environmental conditions.

[0004] On the other hand, due to limitations in underground communication conditions, remote control systems generally suffer from fluctuating communication latency and even short-term interruptions. Existing technologies often send remote control commands directly to the onboard actuators. When communication latency increases or the link becomes unstable, the vehicle status seen by the operator often lags behind the actual status, leading to a mismatch between the control actions and the vehicle's actual response, further amplifying operational risks. While some systems offer protection by reducing vehicle speed or setting fixed safety thresholds, they still lack proactive assessment of the execution results of control commands, failing to effectively mitigate potential risks before command execution.

[0005] Furthermore, most existing remote control systems for underground loaders still rely primarily on visual feedback for human-machine interaction, making it difficult for operators to intuitively perceive key information such as changes in vehicle load, uneven road surfaces, or loading resistance. During loading operations, operators cannot judge the shoveling resistance and vehicle stress state through "feel" and "body feel" as they would when driving on-site, which can easily lead to problems such as excessive thrust, abnormal bucket stress, or equipment impact. At the same time, when the communication link fails, some systems only take simple emergency stop or control disconnection measures, lacking a smooth and safe transition mechanism, affecting the continuity of operations and potentially causing new safety hazards.

[0006] Therefore, how to improve the reliability of remote control systems in perceiving the working environment and vehicle status in complex underground environments, how to improve the safety and predictability of control command execution under unstable communication conditions, and how to enhance operators' ability to perceive risks and load status through more natural human-machine interaction have become key technical issues that urgently need to be addressed in the field of remote control technology for underground loaders.

[0007] Therefore, there is an urgent need for a remote control system for underground loader to solve these problems. Summary of the Invention

[0008] The purpose of this invention is to solve the technical problems mentioned in the background section and to provide a remote control system for underground shovels.

[0009] The above-mentioned objective of the present invention is achieved through the following technical solution: A remote control system for an underground loader includes: The remote cockpit subsystem is used to receive control commands input by the operator and provide the operator with vehicle status and risk feedback information; The vehicle-mounted subsystem, deployed on the underground loader, is used to collect vehicle operating status data and operational environment perception data, and drive the underground loader to perform corresponding actions according to the received control commands; A communication subsystem is used to establish a two-way data communication link between the remote cockpit subsystem and the vehicle-mounted subsystem; And a cloud-based digital twin and collaborative decision-making platform, which is connected to the communication subsystem; The cloud-based digital twin and collaborative decision-making platform: based on vehicle status data and operational environment perception data from the vehicle-mounted subsystem, constructs and updates in real time a dynamic digital twin model synchronized with the physical underground loader and its operational environment; Furthermore, the cloud-based digital twin and collaborative decision-making platform further: receives raw control commands from the remote cockpit subsystem, performs pre-simulation of the execution results of the raw control commands in the dynamic digital twin model, constrains, corrects, or optimizes the raw control commands based on the pre-simulation results, generates final control commands that satisfy preset safety constraints and kinematic constraints, and sends them to the vehicle subsystem for execution through the communication subsystem.

[0010] As a preferred technical solution of the present invention, the vehicle subsystem includes a multimodal perception fusion device, which includes a visible light camera, an infrared thermal imaging device, a millimeter-wave radar, and a solid-state lidar. The multimodal perception fusion device adopts an adaptive weight fusion method, which dynamically adjusts the fusion weight of various sensor data according to the lighting conditions, dust concentration and sensor echo quality in the underground working environment, so as to generate fused three-dimensional environmental perception information and send it to the cloud digital twin and collaborative decision-making platform.

[0011] As a preferred embodiment of the present invention, when the cloud-based digital twin and collaborative decision-making platform executes the pre-simulation, it includes: Based on the dynamic digital twin model, the motion trajectory and attitude changes of the underground loader are predicted when the original control command is executed within a preset time period in the future; Determine whether the predicted motion trajectory and posture spatially interfere with the work environment model or exceed a preset safety threshold; When no interference occurs and the safety threshold is not exceeded, the original control command is smoothed and optimized to generate the final control command. When interference occurs or a safety threshold is exceeded, an intervention control command is generated, and the intervention control command is fused with the original control command to generate the final control command, and / or an early warning message is generated and sent to the remote cockpit subsystem.

[0012] As a preferred technical solution of the present invention, the remote cockpit subsystem includes a tactile and somatosensory feedback device; The warning information includes tactile feedback instructions; The tactile and somatosensory feedback device outputs physical feedback to the operator corresponding to the pre-simulation results according to the tactile feedback command, and the physical feedback includes at least one of the following: Reverse force, damping change, or self-centering torque acting on the steering wheel or control device; The driver's seat is driven to generate multi-degree-of-freedom movement to simulate changes in vehicle posture or uneven road conditions; It generates directional, array-like vibrations at specific points on the operator's body.

[0013] As a preferred technical solution of the present invention, the remote cockpit subsystem includes a force feedback operation device and a status feedback display device; The system also includes a latency management mechanism, wherein the latency management mechanism is as follows: Measure the forward communication delay and reverse communication delay of the communication subsystem; The received vehicle status data is time-compensated based on the reverse communication delay so that the information presented by the status feedback display device is consistent with the current state of the dynamic digital twin model. Based on the forward communication delay and the prediction results of the pre-simulation, the output force or damping characteristics of the force feedback operation device are dynamically adjusted to provide the operator with tactile cues that match the predicted next state of the vehicle.

[0014] As a preferred technical solution of the present invention, the vehicle subsystem further includes a local autonomous decision-making device; The local autonomous decision-making device automatically activates the local autonomous operation mode when it detects that the communication link with the remote cockpit subsystem is interrupted or the communication quality is lower than a preset threshold. In the local autonomous operation mode, based on data collected by local sensors and preset control logic, the underground loader is subjected to safe slow-down control until communication is restored or the vehicle comes to a safe stop.

[0015] As a preferred technical solution of the present invention, the safe slow-down control in the local autonomous operation mode includes at least one of the following strategies: Continue traveling the preset distance at a preset safe speed along the current direction of travel; It can follow the track along the lane wall or the boundary of a pre-set electronic fence; Control the underground loader to travel to the predefined nearest safe stopping position and perform a stopping operation.

[0016] The present invention also provides a method for remote control of an underground loader, the method comprising: Collect vehicle status data and multimodal perception data of the working environment of the underground loader, and transmit the data to the cloud-based digital twin and collaborative decision-making platform; Based on the data, a dynamic digital twin model is constructed and updated in real time, synchronized with the physical underground loader and its operating environment; Receive raw control commands from the remote cockpit subsystem; The execution results of the original control commands are simulated in the dynamic digital twin model. The final control commands that satisfy the preset safety constraints are generated based on the results of the pre-simulation. The final control command is sent to the vehicle-mounted subsystem to control the underground loader to perform its tasks.

[0017] As a preferred technical solution of the present invention, the process of performing pre-simulation on the original control command and generating the final control command includes: Predict the operating status of the underground loader within a preset time period after executing the original control command; Determine whether the operating status poses a security risk; When there is no safety risk, the original control command is optimized and used as the final control command. When a safety risk exists, a risk avoidance control command is generated, and the risk avoidance control command is merged with the original control command to become the final control command. At the same time, a warning message corresponding to the type of safety risk is generated and sent to the remote cockpit subsystem.

[0018] As a preferred embodiment of the present invention, the method further includes a communication interruption handling step: Real-time monitoring of the communication link status between the remote cockpit subsystem and the vehicle subsystem; When a communication link interruption is detected, the local autonomous operation mode of the vehicle subsystem is activated; In the local autonomous operation mode, the underground loader is controlled to operate in a safe and slow manner based on the last valid control command stored locally and / or the preset safety policy. Once communication is restored, control of the underground loader will be smoothly transferred back to the remote cockpit subsystem.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention, by introducing a cloud-based digital twin and collaborative decision-making platform, transforms the remote control of underground loaders from the traditional "direct command control" to a "pre-verification and execution" control mode. Before being sent to the vehicle, the original control commands input from the remote cockpit are pre-simulated in a digital twin model that is synchronously updated with the real vehicle and the working environment. The system can identify potential collision risks, abnormal postures, or working conditions exceeding safety thresholds in advance, and constrain, correct, or optimize the control commands accordingly, thereby effectively avoiding safety accidents caused by communication delays, operational errors, or environmental complexity. This method significantly improves the safety and reliability of remote control operations of underground loaders and reduces the dependence on the operator's reaction speed and experience level.

[0020] 2. This invention improves the stability and accuracy of environmental perception under complex underground conditions by designing a multimodal perception and adaptive fusion mechanism. The system comprehensively utilizes information from multiple sources of sensors, including visible light, infrared, millimeter-wave radar, and lidar, and dynamically adjusts the fusion weights of various data according to the lighting conditions, dust concentration, and sensor echo quality in the underground tunnel. This allows for the formation of reliable three-dimensional environmental perception results even under adverse conditions such as low illumination, high dust, or limited line of sight, providing a stable data foundation for digital twin models and pre-simulation. Compared to solutions that rely on a single sensor or a fixed fusion strategy, this invention has stronger environmental adaptability and higher operational continuity in complex underground environments.

[0021] 3. This invention significantly improves the human-machine interaction experience and system fault tolerance of remote operation by introducing tactile and somatosensory collaborative feedback and a local autonomous operation mechanism under communication anomalies. The system can transform the risk direction, risk intensity, and load changes of shoveling operations identified in the digital twin pre-simulation into physical feedback such as steering wheel force, pedal damping, seat posture, or vibration, allowing the operator to obtain a "feel" and "sense" close to real driving in a remote environment, thereby more intuitively understanding the operation status and making reasonable operations. At the same time, in the event of degraded or interrupted communication quality, the vehicle subsystem can automatically switch to a local autonomous operation mode to maintain vehicle operation or complete a safe stop in a safe and slow manner, avoiding the risk of loss of control due to communication anomalies, and improving the practicality and engineering application value of the system in actual mining conditions. Attached Figure Description

[0022] Fig. 1 This is a system block diagram of the present invention; Fig. 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with embodiments and appendices. Figs. 1-2 The present invention will be further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0024] Example 1: This example uses an underground loader with a rated load of 10 tons as an example to illustrate the specific application process of a remote control system for an underground loader based on digital twin and multimodal perception in a real mining operation scenario. The underground loader is deployed in an underground roadway of a metal mine. The roadway width is approximately 4.5 meters and the roadway height is approximately 4 meters. The working environment has typical underground working conditions such as large variations in dust concentration, insufficient lighting, and small roadway curvature radius.

[0025] In this embodiment, the vehicle-mounted subsystem is installed on the underground loader body and is equipped with a visible light camera, an infrared thermal imaging device, a millimeter-wave radar, and a solid-state lidar to collect information about the vehicle's surrounding environment from different physical dimensions. Specifically, the visible light camera acquires information about the tunnel structure and obstacle contours; the infrared thermal imaging device supplements target contour information under low-light and high-dust conditions; the millimeter-wave radar detects the distance and relative motion of targets in front and to the sides; and the solid-state lidar constructs a high-precision 3D point cloud environment model. After the data collected by various sensors is synchronized in time within the vehicle-mounted subsystem, it is transmitted in real-time to the cloud-based digital twin and collaborative decision-making platform via the communication subsystem.

[0026] After receiving data uploaded by the vehicle-mounted subsystem, the cloud-based digital twin and collaborative decision-making platform constructs a digital twin model synchronized with the physical underground loader and its operating environment. This digital twin model not only reflects the real-time position, attitude, speed, and bucket status of the underground loader, but also dynamically presents environmental information such as tunnel structure, ore pile distribution, and obstacle locations. During operation, the digital twin model continuously updates in sync with the physical loader, used for simulation verification of remote operation behaviors.

[0027] The remote cockpit subsystem is located in the ground control room. Operators control the underground loader via a force feedback steering wheel, force feedback pedals, and a display terminal. After the operator inputs driving, steering, and bucket movement commands in the cockpit, these raw control commands are first transmitted to a cloud-based digital twin and collaborative decision-making platform, rather than being directly sent to the underground loader for execution. The cloud platform performs a pre-simulation of the execution effects of the raw control commands in the digital twin model, predicting the underground loader's trajectory, attitude changes, and relative relationship with the surrounding environment over a future period.

[0028] When simulation results indicate a risk of collision between the underground loader and the tunnel wall or obstacles under the current control command, the cloud-based digital twin and collaborative decision-making platform will send tactile and somatosensory feedback information to the remote cockpit subsystem while generating the final control command. In this embodiment, when the system determines that there is a tendency for the underground loader to collide on its right side, the force feedback steering wheel will generate a leftward-pointing return resistance, and simultaneously, the vibration unit located on the right side of the driver's seat will activate, allowing the operator to perceive the direction of the risk through both tactile and somatosensory sensations during operation. After receiving the above feedback, the operator can adjust the steering operation in a timely manner without relying on visual visuals, thereby avoiding contact between the vehicle and the tunnel wall.

[0029] When an underground loader performs loading operations, the cloud-based digital twin and collaborative decision-making platform also predicts the workload during the contact process between the bucket and the ore pile. When the digital twin model predicts that the resistance gradually increases as the bucket cuts into the ore pile, the system simulates a noticeable "heavy" feel in the remote cockpit via a force feedback steering wheel, while simultaneously controlling the multi-degree-of-freedom motion platform to produce a slight backward tilt change, allowing the operator to intuitively perceive the changes in the loading load. Through this tactile and sensory feedback, the operator can reasonably adjust the advance speed and bucket movements to avoid equipment impact or decreased operational efficiency due to excessive advance speed or excessive load.

[0030] During operation, the communication subsystem continuously monitors the communication status between the remote cockpit subsystem and the vehicle-mounted subsystem. When a decline in communication quality or a link interruption is detected, the vehicle-mounted subsystem automatically enters a partially autonomous operation mode. In this mode, the underground loader continues to travel a preset distance at a safe speed or follows a track along the tunnel boundary based on environmental information collected by local sensors and the last valid control command. It also performs a safe stop operation when necessary, thereby avoiding the risk of loss of control due to communication anomalies.

[0031] Once the communication link is restored, the system gradually returns control to the remote cab subsystem through a smooth transition, avoiding abrupt changes in vehicle control status. The operator can continue to remotely control the underground loader from the cab, and the entire operation process is uninterrupted.

[0032] As can be seen from the above embodiments, the remote control system for underground loaders described in this invention can achieve safe, efficient and perceptible remote control of underground loaders in actual underground mining environments through digital twin pre-simulation, multimodal perception fusion and tactile and somatosensory collaborative feedback, significantly reducing operational risks and improving operational stability and user experience.

[0033] Example 2: During operation, the remote control system for the underground loader of the present invention continuously collects multimodal perception data of the underground working environment and the operating status data of the underground loader via the vehicle-mounted subsystem, and uploads it to the cloud-based digital twin and collaborative decision-making platform through the communication subsystem. The multimodal perception data includes image data acquired by a visible light camera, temperature distribution data acquired by an infrared thermal imaging device, target distance and speed data acquired by a millimeter-wave radar, and three-dimensional point cloud data acquired by a solid-state lidar. To adapt to the working environment of underground tunnels with poor lighting conditions, high dust concentration, and frequent changes, the system first quantitatively evaluates the reliability of various sensor data under the current working conditions, and then performs weighted fusion of multi-source data accordingly.

[0034] At discrete time The light intensity of the underground working environment is defined as follows: Dust concentration is , No. The data quality index of the sensor at this moment is Based on the above environmental parameters and data quality parameters, the first... Overall reliability of this type of sensor: ; in, Indicates the first Sensor-like devices at all times Overall reliability; This indicates the dust concentration in the underground working environment; Indicates ambient light intensity; Indicates the first Data quality evaluation values ​​for sensor types; Indicates the effect of dust on the first The attenuation coefficient of the influence of sensor-like components; , Indicates the effect of light on the first Mapping parameters affected by sensor-like devices; This represents an S-shaped function used to map the effects of illumination to a finite interval; This indicates the sensor category index.

[0035] After obtaining the reliability of various sensors, they are normalized to obtain the fusion weights: ; in, Indicates the first Sensor-like devices at all times The fusion weight; This indicates the total number of sensor categories participating in the fusion. This represents the sum of the reliability of all sensors. This normalization process ensures that the sum of the weights of all types of sensors is 1.

[0036] Based on the aforementioned fusion weights, different sensors will be used to measure spatial voxels. The occupancy probabilities are weighted and fused to construct a unified three-dimensional environment occupancy model: ; in, Voxel representation At any moment The probability of fusion being blocked by obstacles; Indicates the first Sensor-like sensors for voxels The occupancy probability estimation results; Represents the coordinates of a voxel unit in three-dimensional space.

[0037] Based on the preset occupancy probability threshold Extracting the obstacle set from the fused environment model: ; in, Indicates at time The set of voxels identified as obstacles; This indicates the threshold for obstacle detection.

[0038] At the same time, the onboard subsystem collects the operating status data of the underground loader and expresses it in a unified state vector form in the cloud-based digital twin platform: ; in, Indicates the time when the underground loader is The digital twin state; This indicates the spatial position of the vehicle in the lane coordinate system; Represents the vehicle's linear velocity vector; Represents the vehicle attitude angle vector; Represents the vehicle's angular velocity vector; It indicates operating parameters such as bucket position, tilting angle, and opening / closing status; It indicates operating and health status parameters such as hydraulic pressure, wheel speed, steering angle, and load.

[0039] The cloud-based digital twin platform updates the twin's state based on the vehicle's kinematics or dynamics model. ; in, Indicates the twin state at the next moment; Represents the state transition function of an underground loader; Indicates at time The control input vector applied to the underground loader.

[0040] When the remote cockpit subsystem is at time Output raw control commands Subsequently, the cloud-based digital twin platform uses the current twin state... Using these as initial conditions, a pre-simulation is performed to predict the execution results of the original control commands within a preset future time period: ; in, Indicates the future Twin prediction state at each prediction step size; Indicates the number of prediction steps; Equal to the current twin state .

[0041] Based on predicted state sequence and obstacle set Calculate the minimum distance between the underground loader and obstacles during the prediction process. ; in, Indicates the prediction of the first The minimum distance between the vehicle and the obstacle during the movement; This represents the geometric envelope of the vehicle in the predicted state. Represents any point on the envelope; Represents obstacle pixel points; This represents the Euclidean distance.

[0042] According to the safe distance threshold Construct collision risk indicators: ; in, Indicates the prediction of the first The intensity of the collision risk; This indicates the preset safe distance threshold.

[0043] Furthermore, collision risk, vehicle attitude risk, and speed risk are unified and represented as a comprehensive risk: ; in, Indicates the prediction of the first The overall risk value of the step; Indicates the predicted roll angle; Indicates the maximum allowable roll angle threshold; Represents the predicted velocity vector; Indicates the upper limit of safe speed; , , These represent the weighting coefficients for different risk items. The maximum risk within the forecast period is used as the basis for current control decisions. ; in, This represents the most unfavorable risk level for executing the original control command within the predicted future timeframe. Intervention weights are generated based on this maximum risk value. ; in, Indicates the weight of risk intervention; Indicates the risk trigger threshold; This represents the risk saturation value. When the risk is low, the original control commands are smoothly optimized. ; in, This indicates the control command after smoothing and optimization. This indicates the last control command issued at the previous moment; This indicates that, when risks exist, the smoothing weight coefficients are used to solve for the safety control instructions that satisfy the safety distance constraints. ; in, This indicates an intervention control command that meets the safety distance constraint.

[0044] The final control command is obtained by fusing risk weights: ; in, This indicates the final control command issued to the onboard subsystem for execution.

[0045] The maximum risk value is used to generate both tactile and somatosensory feedback signals: ; in, Indicates the force feedback torque of the steering wheel or control device; Indicates the force feedback gain; This indicates the component of the risk direction in the lateral direction of the vehicle.

[0046] Vibration feedback intensity is defined as: ; in, Indicates vibration intensity; This indicates the vibration feedback gain.

[0047] When the communication link quality degrades or is interrupted, the onboard subsystem implements slow autonomous control based on the last valid final control command: ; in, This indicates control commands in a partially autonomous mode; Indicates the last effective control time; Indicates the attenuation coefficient; Indicates the decay rate; Indicates the control cycle.

[0048] After communication is restored, control is gradually returned to the remote cockpit subsystem through a weighted smoothing method, thereby ensuring the continuity and safety of system operation.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A remote control system for an underground loader, characterized in that, include: The remote cockpit subsystem is used to receive control commands input by the operator and provide the operator with vehicle status and risk feedback information; The vehicle-mounted subsystem, deployed on the underground loader, is used to collect vehicle operating status data and operational environment perception data, and drive the underground loader to perform corresponding actions according to the received control commands; A communication subsystem is used to establish a two-way data communication link between the remote cockpit subsystem and the vehicle-mounted subsystem; And a cloud-based digital twin and collaborative decision-making platform, which is connected to the communication subsystem; The cloud-based digital twin and collaborative decision-making platform: based on vehicle status data and operational environment perception data from the vehicle-mounted subsystem, constructs and updates in real time a dynamic digital twin model synchronized with the physical underground loader and its operational environment; Furthermore, the cloud-based digital twin and collaborative decision-making platform is further used to: receive raw control commands from the remote cockpit subsystem, perform pre-simulation of the execution results of the raw control commands in the dynamic digital twin model, constrain, correct or optimize the raw control commands based on the results of the pre-simulation, generate final control commands that satisfy preset safety constraints and kinematic constraints, and send them to the vehicle subsystem for execution through the communication subsystem.

2. The remote control system for underground loader according to claim 1, characterized in that, The vehicle-mounted subsystem includes a multimodal perception fusion device, which includes a visible light camera, an infrared thermal imaging device, a millimeter-wave radar, and a solid-state lidar. The multimodal perception fusion device adopts an adaptive weight fusion method, which dynamically adjusts the fusion weight of various sensor data according to the lighting conditions, dust concentration and sensor echo quality in the underground working environment, so as to generate fused three-dimensional environmental perception information and send it to the cloud digital twin and collaborative decision-making platform.

3. The remote control system for underground loader according to claim 1, characterized in that, When the cloud-based digital twin and collaborative decision-making platform executes the pre-simulation, it includes: Based on the dynamic digital twin model, the motion trajectory and attitude changes of the underground loader are predicted when the original control command is executed within a preset time period in the future; Determine whether the predicted motion trajectory and posture spatially interfere with the work environment model or exceed a preset safety threshold; When no interference occurs and the safety threshold is not exceeded, the original control command is smoothed and optimized to generate the final control command. When interference occurs or a safety threshold is exceeded, an intervention control command is generated, and the intervention control command is fused with the original control command to generate the final control command, and / or an early warning message is generated and sent to the remote cockpit subsystem.

4. The remote control system for underground loader according to claim 3, characterized in that, The remote cockpit subsystem includes tactile and motion feedback devices; The warning information includes tactile feedback instructions; The tactile and somatosensory feedback device outputs physical feedback to the operator corresponding to the pre-simulation results according to the tactile feedback command, and the physical feedback includes at least one of the following: Reverse force, damping change, or self-centering torque acting on the steering wheel or control device; The driver's seat is driven to generate multi-degree-of-freedom movement to simulate changes in vehicle posture or uneven road conditions; It generates directional, array-like vibrations at specific points on the operator's body.

5. The remote control system for underground loader according to claim 1, characterized in that, The remote cockpit subsystem includes a force feedback operating device and a status feedback display device; The system also includes a latency management mechanism, which is as follows: Measure the forward communication delay and reverse communication delay of the communication subsystem; The received vehicle status data is time-compensated based on the reverse communication delay so that the information presented by the status feedback display device is consistent with the current state of the dynamic digital twin model. Based on the forward communication delay and the prediction results of the pre-simulation, the output force or damping characteristics of the force feedback operation device are dynamically adjusted to provide the operator with tactile cues that match the predicted next state of the vehicle.

6. The remote control system for underground loader according to claim 1, characterized in that, The vehicle-mounted subsystem also includes a local autonomous decision-making device; The local autonomous decision-making device automatically activates the local autonomous operation mode when it detects that the communication link with the remote cockpit subsystem is interrupted or the communication quality is lower than a preset threshold. In the local autonomous operation mode, based on data collected by local sensors and preset control logic, the underground loader is subjected to safe slow-down control until communication is restored or the vehicle comes to a safe stop.

7. The remote control system for underground loader according to claim 6, characterized in that, The safe slowdown control in the local autonomous operation mode includes at least one of the following strategies: Continue traveling the preset distance at a preset safe speed along the current direction of travel; It can follow the track along the lane wall or the boundary of a pre-set electronic fence; Control the underground loader to travel to the predefined nearest safe stopping position and perform a stopping operation.

8. A remote control method for underground loader based on digital twin and multimodal sensing, characterized in that, The method, applied to the remote control system for underground loader as described in any one of claims 1 to 7, comprises: Collect vehicle status data and multimodal perception data of the working environment of the underground loader, and transmit the data to the cloud-based digital twin and collaborative decision-making platform; Based on the data, a dynamic digital twin model is constructed and updated in real time, synchronized with the physical underground loader and its operating environment; Receive raw control commands from the remote cockpit subsystem; The execution results of the original control commands are simulated in the dynamic digital twin model. The final control commands that satisfy the preset safety constraints are generated based on the results of the pre-simulation. The final control command is sent to the vehicle-mounted subsystem to control the underground loader to perform its tasks.

9. The method according to claim 8, characterized in that, The process of performing pre-simulation on the original control commands and generating the final control commands includes: Predict the operating status of the underground loader within a preset time period after executing the original control command; Determine whether the operating status poses a security risk; When there is no safety risk, the original control command is optimized and used as the final control command. When a safety risk exists, a risk avoidance control command is generated, and the risk avoidance control command is merged with the original control command to become the final control command. At the same time, a warning message corresponding to the type of safety risk is generated and sent to the remote cockpit subsystem.

10. The method according to claim 8, characterized in that, The method also includes a communication interruption handling step: Real-time monitoring of the communication link status between the remote cockpit subsystem and the vehicle subsystem; When a communication link interruption is detected, the local autonomous operation mode of the vehicle subsystem is activated; In the local autonomous operation mode, the underground loader is controlled to operate in a safe and slow manner based on the last valid control command stored locally and / or the preset safety policy. Once communication is restored, control of the underground loader will be smoothly transferred back to the remote cockpit subsystem.

Images