Deep-sea mining digital twin system and method based on semi-physical simulation

Abstract

The invention provides a deep-sea mining digital twin system and method based on semi-physical simulation, and the deep-sea mining digital twin system based on semi-physical simulation not only integrates hardware control, semi-physical simulation, algorithm prediction and three-dimensional simulation, but also integrates hardware equipment such as a mining ship, a mining vehicle, a vertical pipe, a hose and a repeater. A four-layer architecture including a resource layer, an algorithm layer with a special selective simulation technology, a platform layer and an application layer is also included, a virtual-real mutual feedback mechanism is established between hardware equipment and a software system, digital twinning and state control of different dimensions can be achieved, and high-precision and light-load simulation of the equipment state is achieved.

Description

Technical Field

[0001] This invention relates to the field of deep-sea mining technology, and in particular to a digital twin system and method for deep-sea mining based on semi-physical simulation. Background Technology

[0002] In deep-sea mining operations, three major defects—data acquisition blind spots, delays in human intervention, and simulations deviating from reality—prevent the system from achieving real-time closed-loop management of equipment status and control. This is the core root cause of safety risks in deep-sea mining.

[0003] Specifically, existing technologies primarily rely on physical sensors to directly collect equipment status data and use offline simulation models to rehearse operational processes. This approach has the following functional limitations: (a) Control failures caused by blind spots in environmental monitoring: The deep-sea environment is fraught with unpredictable disturbances. For example, sudden undercurrents can instantly alter the stress state of equipment, but fixed sensors cannot cover the entire operating area, resulting in incomplete data collection of critical parameters such as current speed and direction. Complex seabed topography (such as hydrothermal vents and steep slopes) causes sensor signal attenuation or distortion, leading to incorrect position coordinates obtained by the mining vehicle's navigation system. Equipment obstructions (such as hoses entangled in the mining vehicle's robotic arm) result in missing data on localized environmental temperature and pressure. These monitoring blind spots prevent the control system from accurately assessing the actual operational risks of the equipment.

[0004] (ii) Response delays caused by human decision-making chains: The existing system requires operators to manually analyze sensor data and issue commands. When the mining vehicle encounters a strong current, the manual assessment of propeller thrust adjustment requires several seconds, causing the vehicle's attitude angles (yaw / pitch) to continuously deteriorate. After a hose tension over-limit alarm, the delay in manually deciding to release the repeater may lead to hose breakage. When multiple devices are operating collaboratively (e.g., the mining vessel's submersion and the mining vehicle's movement are synchronized), asynchronous manual coordination commands will cause equipment action conflicts.

[0005] (iii) The offline simulation model is disconnected from the real-time operating conditions: The offline simulation model used in the pre-operation simulation has fundamental limitations: the model loads preset environmental parameters (such as constant ocean current velocity) and cannot dynamically access the sensor data stream updated every second in actual operations. When the physical equipment status changes (such as a mining truck's robotic arm suddenly jamming), the offline model still runs according to the preset trajectory, causing the virtual scene to deviate significantly from reality. Equipment failure simulation relies on a historical case library and cannot map new abnormal working conditions in real time (such as abnormal vibration of riser joints), rendering the pre-operation simulation results worthless.

[0006] (iv) High hardware requirements for synchronization based on real-time sensor data If real-time simulations of the ship's hull and hose attitudes are built based on real-time physical device data, a physically accurate reflection and synchronization can be achieved. However, this requires extremely high hardware computing power. Therefore, long-term data acquisition operations place high demands on hardware stability. Reducing the hardware load is thus an important issue. Summary of the Invention

[0007] 1. Core technology of the invention First, it involves accumulating and predicting the attitude data of the hull, vehicle body, and hose, and creating the current prediction frame. This is then compared with the simulation frame formed by the data from the physical equipment to determine which type of frame to use, thus forming a selective simulation. Second, the calculation of the predicted simulation frame is performed at preset time intervals, and the comparison is then performed, which reduces the hardware load. Third, to ensure accuracy, once the frame type is determined, simulations are performed according to the formation mechanism of that type until the next comparison calculation.

[0008] To address the aforementioned problems, the present invention aims to provide a digital twin system and method for deep-sea mining based on semi-physical simulation.

[0009] This invention provides a digital twin system for deep-sea mining based on semi-physical simulation, the system comprising: The resource layer is used to acquire and store heterogeneous data from multiple sources. The algorithm layer is used to selectively simulate deep-sea mining equipment based on the prediction frames formed by historical data of the physical equipment and the simulation frames formed by real-time data of the physical equipment, so as to achieve real-time state synchronization. The platform layer is used to achieve frame-level synchronization between physical devices and virtual models, as well as command issuance. The application layer is used to realize 3D visualization, risk warning and intelligent closed-loop control; it loads digital models of mining ships and mining trucks, builds control modules for mining ships and mining trucks, and integrates force response calculation models and semi-physical simulation engines to simulate the dynamic evolution process of mining ships and mining trucks under external forces, respectively. The real-time status of mining ships is displayed through time-series simulation and visualization, and digital twins are realized with the selective simulation.

[0010] Optionally, the platform layer includes: An interactive server is used to convert virtual instructions into signals executable by physical devices. The algorithm server is used to calculate key mechanical indicators in real time and trigger abnormal state flags. A 3D rendering server is used to visualize the status of physical equipment and the marine environment.

[0011] Optionally, the deep-sea mining digital twin system connects to multiple physical devices via an interactive server at the platform layer to obtain multi-source heterogeneous data and then stores it at the resource layer; The physical equipment includes one or more of the following: mining vessel, mining truck, riser, hose, and repeater; The semi-physical simulation engine includes: a mining vessel motion simulation model, a hose simulation algorithm model, a mining vehicle simulation algorithm model, and an ocean wave and current simulation model, used to synchronously generate the hose morphology, mining vessel state, mining vehicle state, and marine environment state. That is, this describes the attitude of the mining vessel, mining vehicle, and hose, which is a motion simulation under the action of external forces.

[0012] Optionally, the deep-sea mining digital twin system communicates with the mining vehicle in real time based on the Modbus protocol, realizing the synchronous mapping of the system's remote control of the mining vehicle and its status feedback; The semi-physical simulation engine utilizes predictive models of mining vehicles and vessels, combined with environmental information, to dynamically extrapolate their future states. This supports intelligent decision-making and operational optimization. It sends control commands to the mining vehicles and vessels for forward / backward movement, turning, surfacing, and diving, as well as manipulation commands to the robotic arms and manipulators, adjusting different speed values ​​to achieve virtual-to-real control. In other words, this describes the prediction of changes in the attitude of the mining vehicles and vessels, thus combining this with the aforementioned motion simulation to simulate attitude changes; this falls under the category of motion simulation under control.

[0013] Optionally, the selective simulation of the prediction frame formed based on historical data of the physical device and the simulation frame formed based on real-time data of the physical device specifically includes the following steps: S1: For each type of physical device, acquire historical data of that type of physical device within a preset time period, establish an LSTM model, divide the historical data into training set and validation set, and divide the training set and validation set according to time sequence, with each time point corresponding to a training subset and a validation subset. S2: The training subset is used to train the LSTM model for each time node unit, and the accuracy is verified using the validation subset to obtain the trained LSTM model at each time node. S3: Input the real-time data of the physical device into the current time node of the trained LSTM model to obtain the prediction data of the next time node, and use the semi-physical simulation engine to form a prediction frame, obtain the real-time data of the physical device corresponding to the next time node in the LSTM model, and use the semi-physical simulation engine to form a simulation frame. S4: Use a pre-trained convolutional neural network to identify the physical device region of the simulation frame and the predicted frame at the next time node, and calculate the difference between the two regions. If the number of pixels in the difference exceeds the threshold, select the simulation frame as the frame for the next time node and all subsequent time nodes; otherwise, select the predicted frame as the frame for the next time node and all subsequent time nodes. It is understandable that the smaller the pixel difference between the two, the smaller the positional offset between them; if the difference is small, it means they overlap.

[0014] S5: When the preset time period ends, repeat steps S3-S4 and this step. If the time period does not end, continue to form frames according to the selected simulation method.

[0015] Thus, there is a certain probability that a prediction frame will be selected, thereby eliminating the need for real-time data from physical devices for simulation within the preset time period. Only the current data is required to predict the data at all time points within that preset time period, thus reducing the hardware load. Due to the threshold setting, when the threshold is exceeded, the more accurate simulation frame is prioritized; otherwise, it indicates that the prediction frame has a small difference from the simulation frame and also possesses a certain degree of accuracy, so selecting the prediction frame also reduces the hardware load.

[0016] Optionally, the preset time period is 5s-30s.

[0017] Optionally, the algorithm layer is also used to establish a three-dimensional digital twin model based on the hose's geometric parameters, and to realize real-time prediction and visualization rendering of the hose's spatial configuration, effective tension, curvature, and stress by integrating a real-time prediction algorithm for the hose's dynamic response. Furthermore, it provides a hazard warning through the application layer when the hose's tension, curvature, and stress exceed the threshold.

[0018] This invention also provides a deep-sea mining operation control method applied to a deep-sea mining digital twin system based on semi-physical simulation, the method comprising: Acquire multi-source heterogeneous data of the current frame, the multi-source heterogeneous data including marine environmental parameters and status information of multiple physical devices; The multi-source heterogeneous data is mapped in real time to the corresponding variables of physical equipment and environment in the deep-sea mining digital twin system, and the motion trajectory of the physical equipment is reproduced in real time. Based on the ocean wave and current simulation model, mining vessel motion simulation model, hose simulation algorithm model and mining vehicle simulation algorithm model, hose shape, mining vessel status and mining vehicle status are generated and synchronized to the deep-sea mining digital twin system. The system synchronizes simulation information with the interactive server. The interactive server converts the incoming information according to the signal required by the physical device and sends the signal to the corresponding physical device. The physical device responds to the incoming signal and synchronizes its status according to the signal content. At the same time, it feeds back its own status and the collected environmental information.

[0019] Optionally, the status information of the physical equipment includes the status information of the mining vehicle, the status information of the hose, and the status information of the mining vessel; Marine environmental parameters include temperature, salinity, ocean current, and density information at various locations.

[0020] Optionally, the method further includes: the deep-sea mining digital twin system performs spatiotemporal alignment of the physical equipment measured data and virtual simulation data according to the processing stage identifier based on the feedback information, generates a twin data association set, and maps the measured features and simulation features to a unified dimension through feature space unification processing to form a verification feature correspondence set; The twin device is adjusted based on the dynamic deviation correction algorithm, and the content is rendered to the graphics card according to different categories. The system also judges whether the current state is normal based on the feedback and prepares the data for the next frame. The algorithm server predicts the environmental parameters for the next frame based on historical environmental parameters and environmental models. It also calculates the forces on the hose, mining vessel, and mining vehicle based on hose morphology prediction algorithm, mining vessel motion prediction algorithm, mining vehicle motion prediction algorithm model, and mining vehicle optimal path planning algorithm, and predicts the hose morphology and the state of the mining vessel and mining vehicle.

[0021] Optionally, the method further includes: the deep-sea mining digital twin system outputs the predicted driving speed and direction of the mining vehicle, which is synchronized to the physical equipment of the mining vehicle through an interactive server for dynamic deviation adjustment; at the same time, the mining vehicle can be controlled in real time through the digital twin system; Based on real-time control information, environmental information and mining vehicle status are updated in real time in the deep-sea mining digital twin system.

[0022] The deep-sea mining digital twin system and method based on semi-physical simulation of the present invention have the following beneficial effects: 1. Application of Semi-Physical Simulation Technology: This invention innovatively combines physical sensor data and deep learning algorithms to achieve high-precision simulation of equipment status. This method not only considers physical factors in the actual environment but also simulates possible changes and abnormal situations through deep learning algorithms.

[0023] 2. Real-time monitoring and feedback: By monitoring equipment and environmental data in real time, the system can promptly detect and report potential risks and anomalies, thereby improving operational safety.

[0024] 3. Data-driven decision support: By leveraging big data analytics and deep learning technologies, the system can provide data-driven decision support to help optimize mining operations and improve production efficiency.

[0025] 4. Comprehensive equipment and environment simulation: The system can fully restore and simulate the state of equipment such as mining vessels, mining cars, risers, hoses, and repeaters, providing comprehensive support for the management and optimization of mining operations.

[0026] 5. Interaction between virtual and physical mining equipment: Digital twin equipment and physical equipment can achieve real-time synchronization and data fidelity. When there is a deviation between the prediction model of the digital twin system and the actual equipment, the system can intervene in real time to precisely control the mining equipment.

[0027] 6. Reduce environmental risks of deep-sea mining: By precisely controlling and optimizing the operation of mining equipment, damage to the seabed ecosystem can be reduced, thus lowering environmental risks.

[0028] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0029] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram of a deep-sea mining digital twin system based on semi-physical simulation, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a deep-sea mining digital twin system based on semi-physical simulation, according to another embodiment of the present invention. Figure 3 This is a detailed flowchart of selective simulation, one of the core technologies of this invention; Figure 4 This is a schematic diagram of the deep-sea mining operation control method according to an embodiment of the present invention. Detailed Implementation

[0030] The embodiments of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the present invention and are not restrictive.

[0031] This invention provides a digital twin system for deep-sea mining based on semi-physical simulation, such as... Figure 1 As shown, the deep-sea mining digital twin system based on semi-physical simulation in this embodiment includes: a resource layer, an algorithm layer, a platform layer, and an application layer.

[0032] I. Resource Layer The resource layer is used to acquire and store multi-source heterogeneous data. In this embodiment, the multi-source heterogeneous data may include environmental data and data corresponding to different types of physical devices, such as mining ships, mining vehicles, risers, hoses, repeaters, and other equipment data.

[0033] II. Algorithm Layer The algorithm layer is used to selectively simulate deep-sea mining equipment based on a semi-physical simulation engine. This involves combining prediction frames generated from historical data of the physical equipment with simulation frames generated from real-time data of the physical equipment to achieve real-time state synchronization. The semi-physical simulation engine integrates multiple algorithm models, such as... Figure 2 As shown, simulation models such as the mining vessel motion simulation model, hose simulation algorithm model, mining vehicle simulation algorithm model, and ocean wave and current simulation model are used to simultaneously generate hose morphology, mining vessel status, mining vehicle status, and marine environment status. Relevant information regarding the marine environment status includes temperature, salinity, ocean current, and density information at various locations.

[0034] The equipment information of the mining vehicle may include: mode status information (unlocked / locked status, motion mode status); navigation information (manual control, automatic control); relative position (position controlled by buttons in manual control, position calculated by navigation in automatic mode); attitude (yaw angle, pitch angle, roll angle); movement speed; robotic arm rotation status; robotic arm opening / closing status; lighting and supplementary lighting brightness status; camera on / off status and gimbal rotation status; and the status of the upper and rear propellers.

[0035] The status information of the hose can include: the starting point and the ending point.

[0036] The status information of a mining vessel may include: location information and navigation route.

[0037] like Figure 3 As shown, the selective simulation of prediction frames based on historical data of physical devices and simulation frames based on real-time data of physical devices specifically includes the following steps: S1: For each type of physical device, within a preset time period of 25 seconds, acquire the historical data of that type of physical device, establish an LSTM model, divide the historical data into a training set and a validation set, and divide the training set and validation set according to time sequence, with each time point corresponding to a training subset and a validation subset. S2: The training subset is used to train the LSTM model for each time node unit, and the accuracy is verified using the validation subset to obtain the trained LSTM model at each time node. S3: Input the real-time data of the physical device into the current time node of the trained LSTM model to obtain the prediction data of the next time node, and use the semi-physical simulation engine to form a prediction frame, obtain the real-time data of the physical device corresponding to the next time node in the LSTM model, and use the semi-physical simulation engine to form a simulation frame. S4: Use a pre-trained convolutional neural network to identify the physical device region of the simulation frame and the predicted frame at the next time node, and calculate the difference between the two regions. If the number of pixels in the difference exceeds the threshold, select the simulation frame as the frame for the next time node and all subsequent time nodes; otherwise, select the predicted frame as the frame for the next time node and all subsequent time nodes. S5: When the preset time period ends, repeat steps S3-S4 and this step. If the time period does not end, continue to form frames according to the selected simulation method.

[0038] III. Platform Layer The platform layer is used to achieve frame-level synchronization between physical devices and virtual models, as well as command issuance.

[0039] In an optional embodiment of the present invention, the platform layer includes: an interaction server, an algorithm server, and a 3D rendering server.

[0040] An interactive server is used to convert virtual instructions into executable signals for physical devices. In this embodiment, the deep-sea mining digital twin system connects to multiple physical devices via the interactive server at the platform layer to obtain multi-source heterogeneous data and then stores it in the resource layer. The physical devices include one or more of mining vessels, mining vehicles, risers, hoses, and repeaters.

[0041] The algorithm server is used to calculate key mechanical indicators in real time and trigger abnormal state flags. A 3D rendering server is used to visualize the status of physical equipment and the marine environment.

[0042] In this embodiment, the deep-sea mining digital twin system synchronizes its state with the physical mining equipment frame by frame. The system transmits various user operation information as operation signals through an interaction server and an algorithm server. The system obtains the real-time state of the equipment in the current frame through device responses and algorithm calculations, and then responds accordingly. This generates a virtual twin of the physical and virtual equipment states.

[0043] IV. Application Layer The application layer is used to realize 3D visualization, risk warning and intelligent closed-loop control; specifically, the deep-sea mining digital twin system of this embodiment can load digital models of mining ships and mining vehicles, build control modules for mining ships and mining vehicles, and integrate force response calculation models and semi-physical simulation engines to simulate the dynamic evolution process of mining ships and mining vehicles under external forces, respectively. The real-time status of mining ships is displayed through time-series simulation and visualization, and digital twins are realized with the selective simulation.

[0044] In an optional embodiment of the present invention, the deep-sea mining digital twin system communicates with the mining vehicle in real time based on the Modbus protocol, realizing the synchronous mapping of the system's remote control of the mining vehicle and its status feedback; the semi-physical simulation engine uses the mining vehicle prediction model and combines environmental information to dynamically infer the future state of the mining vehicle to support intelligent decision-making and operation optimization, and sends control commands for advancing, retreating, turning, surfacing, and diving to the mining vehicle, as well as control commands for the robotic arm and robotic arm, and adjusts different speed values ​​to achieve the effect of virtual control of the real.

[0045] In an optional embodiment of the present invention, the algorithm layer is also used to establish a three-dimensional digital twin model based on the hose geometric parameters, realize the real-time prediction and visualization rendering of hose spatial configuration, effective tension, curvature and stress by integrating a real-time prediction algorithm for hose dynamic response, and provide a danger warning through the application layer when the hose tension, curvature and stress exceed the threshold.

[0046] This invention proposes a digital twin system for marine mining that integrates hardware control, semi-physical simulation, algorithm prediction, and 3D simulation. The system integrates not only hardware devices such as mining vessels, mining vehicles, risers, hoses, and repeaters, but also a four-layer architecture comprising resource, algorithm, platform, and application layers. By establishing a virtual-physical feedback mechanism between the hardware devices and the software system, digital twins of different dimensions can be achieved. Using the virtual to reflect the real, the equipment and environment are accurately reproduced: The working state of these devices in the deep-sea environment is accurately reproduced through digital means.

[0047] Predicting equipment status accurately using virtual data: By combining physical sensor data and deep learning algorithms, the status of equipment can be predicted in real time through real-time monitoring and simulation of equipment status.

[0048] Virtual control over physical control enables intelligent issuance of equipment commands: control commands are generated based on prediction results and automatically sent to physical equipment, achieving closed-loop optimization of the entire mining process.

[0049] This embodiment realizes a comprehensive simulation and optimization of the entire deep-sea mining process, effectively solving technical problems such as complex environment, difficulty in monitoring equipment status, and large control delay in deep-sea mining.

[0050] This invention also provides a deep-sea mining operation control method applied to the aforementioned deep-sea mining digital twin system based on semi-physical simulation, such as... Figure 4 As shown, the deep-sea mining operation control method of this invention includes: S1: Obtain multi-source heterogeneous data of the current frame, the multi-source heterogeneous data including marine environmental parameters and status information of multiple physical devices; S2: The multi-source heterogeneous data is mapped in real time to the corresponding variables of the physical equipment and environment in the deep-sea mining digital twin system to reproduce the motion trajectory of the physical equipment in real time; Based on the ocean wave and current simulation model, mining vessel motion simulation model, hose simulation algorithm model and mining vehicle simulation algorithm model, hose shape, mining vessel status and mining vehicle status are generated and synchronized to the deep-sea mining digital twin system. S3: Synchronize the simulation information within the system with the interactive server. The interactive server converts the incoming information according to the signal required by the physical device and sends the signal to the corresponding physical device. The physical device responds to the incoming signal and synchronizes its status according to the signal content. At the same time, it feeds back its own status and the collected environmental information.

[0051] The physical equipment status information obtained in step S1 above includes mining vehicle status information, hose status information, and mining vessel status information; the marine environmental parameters include temperature information, salinity information, ocean current information, and density information at each location.

[0052] The equipment information of the mining vehicle may include: mode status information (unlocked / locked status, motion mode status); navigation information (manual control, automatic control); relative position (position controlled by buttons in manual control, position calculated by navigation in automatic mode); attitude (yaw angle, pitch angle, roll angle); movement speed; robotic arm rotation status; robotic arm opening / closing status; lighting and supplementary lighting brightness status; camera on / off status and gimbal rotation status; and the status of the upper and rear propellers.

[0053] The status information of the hose can include: the starting point and the ending point.

[0054] The status information of a mining vessel may include: location information and navigation route.

[0055] Furthermore, after step S3 above, the deep-sea mining digital twin system aligns the measured data of the physical equipment and the virtual simulation data according to the processing stage identifier based on the feedback information, generates a twin data association set, and maps the measured features and simulation features to a unified dimension through feature space unification processing to form a verification feature correspondence set; the twin equipment is adjusted based on the dynamic deviation correction algorithm, and rendered to the graphics card according to different content categories, and the system judges whether the current state is normal based on the feedback and prepares the next frame of data.

[0056] In an optional embodiment of the present invention, all of the above-mentioned information can be synchronized to the algorithm server. The algorithm server predicts the environmental parameters of the next frame based on historical environmental parameters and environmental models, and calculates the force conditions of the hose, mining ship, and mining car based on the hose morphology prediction algorithm, mining ship motion prediction algorithm model, mining car motion prediction algorithm model, and mining car optimal path planning algorithm, and predicts the hose morphology, mining ship, and mining car status.

[0057] For example, the predicted hose morphology includes: stress values, tension values, and curvature values ​​at various locations.

[0058] The predicted states of the mining vessel include: the surfacing and submerging state of the mining vessel, and the rolling and pitching state of the mining vessel.

[0059] The predicted mining vehicle conditions include: traction; buoyancy; center of gravity; ocean current speed and direction; and sediment plume.

[0060] Based on the above information, the status of the corresponding digital twin equipment within the deep-sea mining digital twin system can be updated.

[0061] In an optional embodiment of the present invention, the deep-sea mining digital twin system outputs the predicted driving speed and direction of the mining vehicle, which is synchronized to the physical equipment of the mining vehicle through an interactive server for dynamic deviation adjustment; at the same time, the mining vehicle can be controlled in real time through the digital twin system; based on the real-time control information, the environmental information and the status of the mining vehicle are updated in real time in the deep-sea mining digital twin system.

[0062] Optionally, the following information about the mining vehicle can be controlled: relative position (position controlled by buttons); attitude (yaw angle, pitch angle, roll angle); movement speed; robotic arm rotation status; robotic arm opening and closing status; lighting and supplementary lighting brightness status; camera on / off status and gimbal rotation status; and upper and rear propeller status.

[0063] Overall, the deep-sea mining digital twin system of this invention integrates technologies such as multi-source heterogeneous data analysis, simulation, forecasting models, IoT sensing and communication, and 3D visual rendering to achieve realistic restoration of the mining environment, dynamic monitoring of mining vessels, real-time prediction of hoses, virtual and real control of mining vehicles, and simulation of deep-sea mining operations. Its core modules include marine environment simulation, virtual and real interactive driving of mining equipment, mining operation simulation, and intelligent visualization. The deep-sea mining digital twin system loads a digital model of the mining vessel, constructs a mining vessel control module, and integrates a force response calculation model to simulate the dynamic evolution of the mining vessel under external forces. The real-time status of the mining vessel is displayed through time-series simulation and visualization.

[0064] The deep-sea mining digital twin system in this embodiment establishes a three-dimensional digital twin model based on the hose's geometric parameters. By integrating a real-time prediction algorithm for hose dynamic response, it achieves real-time prediction and visualization of hose spatial configuration, effective tension, curvature, and stress, and provides a hazard warning when hose tension, curvature, and stress exceed thresholds.

[0065] The digital twin intelligent system, based on the Modbus protocol, enables real-time communication with the physical mining vehicle, allowing for remote control and synchronous mapping of the vehicle's status feedback. It integrates a mining vehicle prediction model, dynamically extrapolating the vehicle's future state based on environmental information to support intelligent decision-making and operational optimization. The system can provide commands for the mining vehicle to move forward, backward, turn, rise, and descend, as well as control the robotic arms and manipulators, adjusting different speed values ​​to achieve virtual control of the physical vehicle.

[0066] During operation, the digital twin intelligent system uses information from the mining vehicle's built-in sensors to achieve high-fidelity real-time mapping of key parameters such as the mining vehicle's speed and attitude. It can also use ocean current speed, mining vehicle propulsion command parameters, and cumulative movement time as input features to output predicted speed and attitude values ​​for future moments, assisting operators in anticipating potentially hazardous conditions and optimizing work processes.

[0067] Overall, the digital twin intelligent system in this embodiment can already be integrated with the mining vehicle for joint debugging and testing, achieving simple virtual-real interaction, and has been verified in a small-scale water tank experiment. The system has initially realized intelligent visualization of the mining environment, virtual-real interaction driving of mining equipment, comprehensive real-scene analysis of multi-source data, and simulation of deep-sea mining operations.

[0068] The deep-sea mining digital twin system and method based on semi-physical simulation of this invention has the following innovations: 1. Frame-level synchronization mechanism The system synchronizes the physical devices and the virtual model at fixed time intervals (each frame refresh cycle) via an interactive server. This synchronization covers dynamic parameters such as the switching of mining vehicle movement modes (e.g., transitioning from manual control to automatic navigation), the rotation angle of the robotic arm joints, and the propeller speed. Real-time data from the physical devices (e.g., the mining vehicle's yaw angle and the coordinates of the hose's start and end points) is instantly mapped into the virtual model, ensuring that the device's posture in the 3D scene matches its real-world state. The interactive server converts virtual commands into executable signals for the devices (e.g., parsing the robotic arm's opening / closing angle command into a hydraulic valve opening control current).

[0069] 2. Semi-physics simulation engine The algorithm server constructs the core of dynamic computing: it takes into account real-time environmental data (temperature gradient, ocean current vector) collected by physical sensors and a historical environmental parameter database to establish a multi-dimensional data fusion model. Based on fluid dynamics equations and deep learning prediction algorithms, it outputs key mechanical indicators in real time, including the tension values ​​and radius of curvature of each node of the hose, as well as the rate of change of the rolling / pitch angle of the mining vessel. When the predicted parameters exceed the safety threshold (such as the hose tension exceeding the material limit), an abnormal state flag is automatically triggered.

[0070] 3. Intelligent decision-making closed loop The system establishes an autonomous control link: utilizing the prediction results of a semi-physical simulation engine (such as the trajectory deviation of the mining vehicle within the next 2 seconds), it generates an optimized control command set. Command types include mining vehicle navigation path replanning (avoiding areas with undercurrents), propeller emergency braking (preventing attitude angle loss of control), and repeater position adjustment (compensating for hose deformation). Commands are directly issued to physical devices for execution via an interactive server, requiring no manual intervention.

[0071] 4. Select Simulation An LSTM model trained using historical data from physical devices is used to predict data at any given time point. A semi-physical simulation engine generates prediction frames, which are then compared to simulation frames generated from real-time data in the same engine. The difference between these simulation frames and the identified physical device images within the frames is used to calculate the number of pixels. The simulation mode—whether it's prediction data simulation or real-time data simulation—is selected based on whether a threshold is exceeded.

[0072] The deep-sea mining digital twin system and method based on semi-physical simulation of the present invention has the following effects: 1. Coordinated control of virtual and physical devices Intelligent adaptation of lighting system: When the virtual model detects that the visibility in front of the mining vehicle is below the threshold, it automatically increases the brightness of the physical fill light and adjusts the lighting parameters of the 3D scene in sync.

[0073] Camera-linked control: The gimbal rotation angle command is dynamically generated based on the working range of the mining vehicle's robotic arm in the virtual model, ensuring that critical operating areas are always within the monitoring field of view.

[0074] 2. Safety Protection of Critical Equipment Hose breakage prevention: When the semi-physical simulation engine calculates that the stress value at a certain node of the hose exceeds the safety limit (e.g., >500MPa), the system immediately issues a mining truck deceleration command and a repeater position correction signal to forcibly reduce the hose load.

[0075] Mining vessel maintains stable attitude: When the predicted roll angle exceeds the warning line, the ballast water tank adjustment command is automatically triggered to suppress the risk of capsizing.

[0076] 3. Three-dimensional visualization early warning Environmental risk labeling: A color temperature layer is overlaid on the surface of the virtual mining ship model, and red areas indicate abnormal snorkeling depths (such as sinking speed > 0.5 m / s).

[0077] Equipment status perspective: The section with excessive curvature (curvature > 0.15 rad / m) is highlighted in the 3D model of the hose, helping the operator to quickly locate the potential hazard.

[0078] 4. Select Simulation This allows for a reduction in hardware load while ensuring simulation accuracy.

[0079] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A digital twin system for deep-sea mining based on semi-physical simulation, characterized in that, The system includes: The resource layer is used to acquire and store heterogeneous data from multiple sources. The algorithm layer is used to selectively simulate deep-sea mining equipment based on the semi-physical simulation engine by combining prediction frames formed from historical data of the physical equipment with simulation frames formed from real-time data of the physical equipment, so as to achieve real-time state synchronization. The platform layer is used to achieve frame-level synchronization between physical devices and virtual models, as well as command issuance. The application layer is used to realize 3D visualization, risk warning and intelligent closed-loop control; it loads digital models of mining ships and mining trucks, builds control modules for mining ships and mining trucks, and integrates force response calculation models and semi-physical simulation engines to simulate the dynamic evolution process of mining ships and mining trucks under external forces, respectively. The real-time status of mining ships is displayed through time-series simulation and visualization, and digital twins are realized with the selective simulation.

2. The deep-sea mining digital twin system based on semi-physical simulation according to claim 1, characterized in that, The platform layer includes: An interactive server is used to convert virtual instructions into signals executable by physical devices. The algorithm server is used to calculate key mechanical indicators in real time and trigger abnormal state flags. A 3D rendering server is used to visualize the status of physical equipment and the marine environment.

3. The deep-sea mining digital twin system based on semi-physical simulation according to claim 2, characterized in that, The deep-sea mining digital twin system connects to multiple physical devices via an interactive server at the platform layer to acquire multi-source heterogeneous data and then stores it at the resource layer. The physical equipment includes one or more of the following: mining vessel, mining truck, riser, hose, and repeater; The semi-physical simulation engine includes: a mining vessel motion simulation model, a hose simulation algorithm model, a mining vehicle simulation algorithm model, and an ocean wave and current simulation model, which are used to simultaneously generate hose morphology, mining vessel status, mining vehicle status, and marine environment status.

4. The deep-sea mining digital twin system based on semi-physical simulation according to claim 3, characterized in that, The deep-sea mining digital twin system communicates with the mining vehicle in real time based on the Modbus protocol, realizing the synchronous mapping of the system's remote control of the mining vehicle and its status feedback. The semi-physical simulation engine utilizes predictive models of mining vehicles and mining vessels, combined with environmental information, to dynamically extrapolate the future states of the mining vehicles and mining vessels, in order to support intelligent decision-making and operational optimization. It also sends control commands to the mining vehicles and mining vessels for advancing, retreating, turning, surfacing, and diving, as well as control commands to the robotic arms and manipulators, and adjusts different speed values ​​to achieve the effect of virtual control of reality.

5. The deep-sea mining digital twin system based on semi-physical simulation according to claim 3 or 4, characterized in that, The selective simulation of prediction frames based on historical data of physical devices and simulation frames based on real-time data of physical devices specifically includes the following steps: S1: For each type of physical device, acquire historical data of that type of physical device within a preset time period, establish an LSTM model, divide the historical data into training set and validation set, and divide the training set and validation set according to time sequence, with each time point corresponding to a training subset and a validation subset. S2: The training subset is used to train the LSTM model for each time node unit, and the accuracy is verified using the validation subset to obtain the trained LSTM model at each time node. S3: Input the real-time data of the physical device into the current time node of the trained LSTM model to obtain the prediction data of the next time node, and use the semi-physical simulation engine to form a prediction frame, obtain the real-time data of the physical device corresponding to the next time node in the LSTM model, and use the semi-physical simulation engine to form a simulation frame. S4: Use a pre-trained convolutional neural network to identify the physical device region of the simulation frame and the predicted frame at the next time node, and calculate the difference between the two regions. If the number of pixels in the difference exceeds the threshold, select the simulation frame as the frame for the next time node and all subsequent time nodes; otherwise, select the predicted frame as the frame for the next time node and all subsequent time nodes. S5: When the preset time period ends, repeat steps S3-S4 and this step. If the time period does not end, continue to form frames according to the selected simulation method.

6. The deep-sea mining digital twin system based on semi-physical simulation according to claim 5, characterized in that, The preset time period is 5s-30s.

7. The deep-sea mining digital twin system based on semi-physical simulation according to any one of claims 1-4, 6, characterized in that, The algorithm layer is also used to establish a three-dimensional digital twin model based on the hose's geometric parameters. By integrating a real-time prediction algorithm for hose dynamic response, it enables real-time prediction and visualization of hose spatial configuration, effective tension, curvature, and stress. Furthermore, it provides a hazard warning through the application layer when hose tension, curvature, or stress exceeds a threshold.

8. A deep-sea mining operation control method applied to a deep-sea mining digital twin system based on semi-physical simulation as described in any one of claims 1-7, characterized in that, The method includes: Acquire multi-source heterogeneous data of the current frame, the multi-source heterogeneous data including marine environmental parameters and status information of multiple physical devices; The multi-source heterogeneous data is mapped in real time to the corresponding variables of physical equipment and environment in the deep-sea mining digital twin system, and the motion trajectory of the physical equipment is reproduced in real time. Based on the ocean wave and current simulation model, mining vessel motion simulation model, hose simulation algorithm model and mining vehicle simulation algorithm model, hose shape, mining vessel status and mining vehicle status are generated and synchronized to the deep-sea mining digital twin system. The system synchronizes simulation information with the interactive server. The interactive server converts the incoming information according to the signal required by the physical device and sends the signal to the corresponding physical device. The physical device responds to the incoming signal and synchronizes its status according to the signal content. At the same time, it feeds back its own status and the collected environmental information.

9. The deep-sea mining operation control method according to claim 8, characterized in that, The status information of the physical equipment includes the status information of the mining vehicle, the hose, and the mining vessel; the marine environmental parameters include temperature, salinity, ocean current, and density at various locations. The deep-sea mining digital twin system aligns the measured data of physical equipment with the virtual simulation data according to the processing stage identifier based on feedback information, generates a set of twin data associations, and maps the measured features and simulation features to a unified dimension through feature space unification processing to form a set of verification features correspondence. The twin device is adjusted based on the dynamic deviation correction algorithm, and the content is rendered to the graphics card according to different categories. The system also judges whether the current state is normal based on the feedback and prepares the data for the next frame. The algorithm server predicts the environmental parameters for the next frame based on historical environmental parameters and environmental models. It also calculates the forces on the hose, mining vessel, and mining vehicle based on hose morphology prediction algorithm, mining vessel motion prediction algorithm, mining vehicle motion prediction algorithm model, and mining vehicle optimal path planning algorithm, and predicts the hose morphology and the state of the mining vessel and mining vehicle.

10. The deep-sea mining operation control method according to claim 8 or 9, characterized in that, The method further includes: The deep-sea mining digital twin system outputs the predicted driving speed and direction of the mining vehicle, which is synchronized to the physical equipment of the mining vehicle through an interactive server for dynamic deviation adjustment; at the same time, the mining vehicle can be controlled in real time through the digital twin system; Based on real-time control information, environmental information and mining vehicle status are updated in real time in the deep-sea mining digital twin system.

Images