Group sensing-control integrated twin device and method for fully-mechanized coal mining equipment

Abstract

The invention discloses a fully-mechanized coal mining equipment group sensing-control integrated twinning device and method, and relates to the technical field of coal mine intelligent mining digital twinning. The method comprises a sensing information driving mode, a physical instruction driving mode and a dual-driving mode. When the sensing information driving mode lacks physical instruction information, real-time sensing data of the hydraulic support are collected, sensing-control integration is achieved, and the shape is consistent; the physical instruction driving mode is used for speculating a current posture through a speed parameter training model and an optimal force training model when sensing information is lost, so that virtual-real synchronization is realized; and the dual-drive mode is used for realizing real-time consistent action and regulation of the virtual monitoring model and the virtual control model according to an adaptive control strategy when the physical instruction transmission path and the sensing information transmission path are both intact. According to the method, high-fidelity actions of the virtual control model and the virtual monitoring model can be driven through the physical entity, and virtual reconstruction of the working space is completed in the virtual space.

Description

Technical Field

[0001] The present application relates to the technical field of digital twin technology for intelligent coal mining, and particularly to a sensing-control integrated twin device and method for fully-mechanized mining equipment groups. Background Art

[0002] The hydraulic support group is the core support equipment for the fully-mechanized mining face, responsible for roof support and face advancement. Achieving the intelligent operation of the hydraulic support group is of great significance for the safe, continuous, efficient, and green mining of the working face.

[0003] However, the fully-mechanized mining hydraulic support group has a large number, is arranged compactly, has frequent operations, a high failure rate, and complex spatial pose relationships and cooperative control objectives, resulting in slow research and application of the intelligentization of the hydraulic support group. Most coal mines are still in the stage of automation or semi-automation. Improving the intelligent control level of the hydraulic support group has become a key problem to be solved urgently in the intelligent construction of the fully-mechanized mining face.

[0004] The application of digital twin theory and virtual reality technology in the field of fully-mechanized mining equipment is expected to provide new ideas for the intelligent monitoring and operation mode of fully-mechanized mining equipment underground in coal mines, and bring positive impacts on aspects such as improving production efficiency, reducing production costs, and reducing accident risks underground in coal mines.

[0005] In the prior art, although digital twin theory and virtual reality technology have been applied in fully-mechanized mining equipment, the following defects still exist:

[0006] 1) Due to the complex underground environment, the sensing information transmission path and the physical instruction transmission path are unstable. When there is only a sensing information path or only a physical instruction path, it is impossible to ensure that the action process of the digital twin model is not affected, and there is a large difference between the action of the model and the actual working conditions.

[0007] 2) The virtual monitoring model relies on data drive and can directly reach the target attitude position with a rapid response; while the virtual control model relies on force drive and takes a certain process to reach the target pose, which takes a long time, resulting in asynchronous actions of the two virtual models.

[0008] 3) In the multi-sensor fusion system, the sensing data has redundancy and conflicts that need to be solved urgently.

[0009] 4) When there are both sensing information and physical instruction inputs, there are problems with conflict resolution and redundancy control, and the system cannot perform adaptive regulation to adapt to the current optimal working conditions. Summary of the Invention

[0010] The purpose of the present application is to provide a sensing-control integrated twin device and method for fully-mechanized mining equipment groups, which can drive the high-fidelity actions of the virtual control model and the virtual monitoring model through a physical entity, and complete the virtual reconstruction of the working space in the virtual space.

[0011] To achieve the above object, the present application provides the following solutions:

[0012] In a first aspect, the present application provides an integrated sensing and control twin device for fully-mechanized mining equipment groups, including: an electro-hydraulic controller, a hydraulic support, a sensing module, and a control unit; a virtual monitoring model, a virtual control model, and a dual-drive Agent model are implanted in the control unit.

[0013] When the virtual monitoring model only receives sensing information, the sensing module includes multiple pose sensors, and the multiple pose sensors are used to collect pose data at different positions on the hydraulic support after the electro-hydraulic controller controls the pose of the hydraulic support to change; the virtual monitoring model is used to determine the weights of the driving variables corresponding to each pose data by using the sensor weighted adaptive hybrid algorithm according to the pose data at different positions, and predict the optimal value of the driving variable according to the weights, and drive the virtual support in the virtual monitoring model according to the optimal value of the driving variable; perform a full pose calculation according to the optimal value of the driving variable to obtain the pose state of the virtual support consistent with the current shape of the hydraulic support as the target state, and send it to the virtual control model; the virtual control model is used to determine the difference between the target state of the virtual support and the current state of the virtual support, determine the driving force according to the difference, and drive the virtual support in the virtual control model to act by using the driving force; the virtual support is a digital twin model of the hydraulic support.

[0014] When the virtual monitoring model only receives physical instructions, the sensing module includes multiple pressure sensors, and the multiple pressure sensors are used to monitor the pressure at different positions of the hydraulic support; the virtual control model is used to receive the control instructions issued by the electro-hydraulic controller, and after receiving the control instructions, predict the optimal virtual force according to the pressure at different positions by using the optimal force Agent model, and then drive the virtual support in the virtual control model to act synchronously with the hydraulic support according to the control instructions and the optimal virtual force, and send the state of the virtual support after the synchronous action ends to the virtual monitoring model; the virtual monitoring model is used to perform a full pose calculation on the received state and control the pose state of the virtual support in the virtual monitoring model according to the data after the full pose calculation.

[0015] When the virtual monitoring model receives both sensing information and physical instructions simultaneously, the sensing module includes a plurality of the pose sensors and a plurality of the pressure sensors. The dual-drive Agent model uses an adaptive control strategy to fuse the control instructions and pose data, obtaining the fused pose data. According to the fused pose data, the sensor weighted adaptive hybrid algorithm is used to determine the weights of the driving variables corresponding to each pose data, and the optimal value of the driving variable is predicted based on the weights. The velocity parameter training model is used to predict the velocity parameters of the movement of the hydraulic support cylinders, and the optimal value of the driving variable is interpolated based on the predicted velocity parameters, so that the virtual support in the virtual monitoring model moves according to the velocity of the hydraulic support. The dual-drive Agent model is also used to, after the virtual control model receives the control instructions, predict the optimal virtual force using the optimal force Agent model according to the pressures at different positions, and then drive the virtual support in the virtual control model to move synchronously with the hydraulic support according to the control instructions and the optimal virtual force.

[0016] In a second aspect, the present application provides a comprehensive mining equipment group sensing-control integrated twin method, including:

[0017] When it is detected that only sensing information is input, the virtual monitoring model obtains the pose data at different positions on the hydraulic support. According to the pose data at different positions, the sensor weighted adaptive hybrid algorithm is used to determine the weights of the driving variables corresponding to each pose data, and the optimal value of the driving variable is predicted based on the weights, and the virtual support in the virtual monitoring model is driven based on the optimal value of the driving variable. Full pose calculation is performed according to the optimal value of the driving variable to obtain the pose state of the virtual support that is consistent with the current shape of the hydraulic support, which is used as the target state and sent to the virtual control model. The virtual control model determines the difference between the target state of the virtual support and the current state of the virtual support, determines the driving force according to the difference, and uses the set driving force to drive the virtual support in the virtual control model to move. The virtual support is a virtual model of the hydraulic support.

[0018] When it is detected that only physical instructions are input, the virtual control model receives the control instructions sent by the electro-hydraulic controller, and after receiving the control instructions, predicts the optimal virtual force using the optimal force Agent model according to the pressures at different positions, and then drives the virtual support in the virtual control model to move synchronously with the hydraulic support according to the control instructions and the optimal virtual force, and sends the state of the virtual support after the synchronous movement ends to the virtual monitoring model. The virtual monitoring model performs full pose calculation on the received state and controls the pose state of the virtual support in the virtual monitoring model according to the data after the full pose calculation.

[0019] When sensing information and physical instructions are detected to be input simultaneously, the dual-drive Agent model adopts an adaptive control strategy to fuse the control instructions and pose data, obtaining the fused pose data; according to the fused pose data, it uses the sensor-weighted adaptive hybrid algorithm to determine the weights of the drive variables corresponding to each pose data, and predicts the optimal value of the drive variables based on the weights; it trains the model with speed parameters to predict the speed parameters of the movement of the hydraulic support cylinder, and interpolates the optimal value of the drive variables based on the predicted speed parameters, so that the virtual support in the virtual monitoring model moves at the speed of the hydraulic support; after the virtual control model receives the control instructions, the dual-drive Agent model also predicts the optimal virtual force using the optimal force Agent model according to the pressures at different positions, and then drives the virtual support in the virtual control model to move synchronously with the hydraulic support based on the control instructions and the optimal virtual force.

[0020] According to the specific embodiments provided in the present application, the present application has the following technical effects:

[0021] The present application provides a fully-mechanized mining equipment group sensing-control integrated twin device and method, which provides three working modes: the sensing information driving mode in which the virtual monitoring model only receives sensing information, the physical instruction driving mode in which the virtual monitoring model only receives physical instructions, and the dual-drive mode in which the virtual monitoring model receives both sensing information and physical instructions, ensuring that the action process of the digital twin model is not affected and the actions of the model conform to the actual working conditions; in these three modes, information is exchanged between the virtual monitoring model and the virtual control model to make the actions of the two virtual models synchronous; the sensor-weighted adaptive hybrid algorithm is used to determine the weights of the drive variables corresponding to each pose data, and the optimal value of the drive variables of the virtual monitoring model is predicted, solving the redundancy and conflict of sensing data in the multi-sensor fusion system; when both sensing information and physical instructions are input, an adaptive control strategy is adopted to fuse the control instructions and pose data, solving information conflicts and redundancy, and being able to adaptively regulate and adapt to the current optimal working conditions. The present application drives the high-fidelity actions of the virtual control model and the virtual monitoring model through the physical entity, and completes the virtual reconstruction of the working space in the virtual space. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a schematic technical route diagram of a fully-mechanized mining equipment group sensing-control integrated twin device provided in an embodiment of the present application;

[0024] Figure 2 Schematic diagram of the technical route of the sensing information driving mode provided by an embodiment of the present application;

[0025] Figure 3 Schematic diagram of the technical route of the physical instruction driving mode provided by an embodiment of the present application;

[0026] Figure 4 Schematic diagram of the technical route of the dual driving mode provided by an embodiment of the present application;

[0027] Figure 5 Schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0028] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0029] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0030] In the prior art, the Chinese patent application with the publication number CN115685240A discloses a method for virtual reconstruction of the relative pose of a hydraulic support based on a hybrid drive of knowledge and data, constructs a digital twin model system of the physical prototype of the hydraulic support, and guides the arrangement of sensors in the physical monitoring system through a sensor position planning system; the data of the sensors in the physical monitoring system are collected by a data acquisition module and transmitted into a mechanism model and data processing, and after being corrected by a parameter preprocessing module, the data are transmitted into a relative position calculation module as known quantities for calculation, and the calculation results are mapped into a pose deduction system in real time through a dynamic link; a pose correction module iterates the results of the pose deduction system to construct a reconstruction method driven by a hybrid of knowledge and data; finally, the pose information of the hydraulic support is displayed in a human-computer interaction system, the error of the deduced pose information of the hydraulic support is evaluated in an accuracy evaluation system, and the status of all systems is monitored by using the human-computer interaction system.

[0031] Chinese Patent No. CN112945160A discloses a virtual-reality integrated relative pose test platform and test method for hydraulic supports, which uses a virtual-reality integration method to simulate the working process and support scenarios of adjacent hydraulic supports in a real coal seam environment, including a physical support test system, a virtual-reality data interaction system, and a multi-support virtual test scenario. The multi-support virtual test scenario simulates the action process of adjacent supports in a real underground mine, the real-time relative pose state, and the inclination of the coal seam. The real-time pose image, relative pose data between hydraulic supports, and real-time pressure data of the actual tested hydraulic support are displayed through the alternating actions of adjacent virtual supports and the actual tested hydraulic support. It can take into account the inclination conditions of the coal seam roof and floor and the change of roof pressure during the study of the relative pose state between adjacent hydraulic supports, and solves the problems of low test efficiency, difficulty in real-time observation of the full pose of hydraulic supports, and huge consumption of manpower and material resources during the test of the pose monitoring and judgment algorithm and pose adjustment method for a group of hydraulic supports.

[0032] The published paper "Key Technologies and Systems for Virtual Production of Fully Mechanized Mining Equipment Driven by Digital Twin" obtains the real-time pose data of the fully mechanized mining equipment during operation through various sensors on the physical system of the fully mechanized mining equipment. According to the virtual-reality two-way data interaction technology and virtual monitoring technology, the processed operation attitude data of the fully mechanized mining equipment is assigned to the corresponding variables of the virtual model to drive the real-time action of the virtual fully mechanized mining equipment. In the physical fully mechanized coal mining face, the data acquisition module monitors the change of the potential of the controller at the monitoring and control end, converts it into a control signal, and sends it to the database in the form of a message. The virtual end receives the database data and converts the data into a control signal to control the action and operation of the virtual fully mechanized mining equipment.

[0033] The published paper "Research on Intelligent Control Method for Pose of Hydraulic Support Group Driven by Digital Twin" combines sensing and monitoring data with pose reconstruction methods to obtain a digital description of the pose and operating state of the support. This description is corresponding to the virtual model drive interface, realizing the operation of the virtual model driven by sensing data and restoring the pose state of the physical support, that is, realizing the virtual monitoring of the physical support. The communication between the virtual space and the physical object control system is established to realize the writing of control instructions. A single hydraulic support group in the virtual space makes decisions on its own action behavior according to the distributed control strategy, and converts the result into a control instruction to reverse-control the corresponding physical entity action. The working condition sensing data goes through three key links of "virtual monitoring - virtual decision - reverse control", and is converted into control instructions that can directly guide the operation of physical equipment. The virtual space serves the physical space and provides an analysis and calculation platform for it, and realizes the intelligent operation of real equipment by means of closed-loop linkage between the virtual and real spaces.

[0034] None of the above existing technologies can drive the high-fidelity actions of the virtual control model and the virtual monitoring model to complete the virtual reconstruction of the working space in the virtual space. To solve the problems existing in the existing technologies, in an exemplary embodiment, asFigure 1 As shown, a comprehensive mining equipment group sensing-control integrated twin device is provided, including: an electro-hydraulic controller, a hydraulic support, a sensing module, and a control unit. The control unit is implanted with a virtual monitoring model, a virtual control model, and a dual-drive Agent model.

[0035] When the virtual monitoring model only receives sensing information, the sensing module includes multiple pose sensors. The multiple pose sensors are used to collect the pose data of different positions on the hydraulic support after the electro-hydraulic controller controls the pose of the hydraulic support to change; the virtual monitoring model is used to determine the weights of the driving variables corresponding to the pose data according to the pose data of different positions by using the sensor weighted adaptive hybrid algorithm, and predict the optimal value of the driving variable according to the weights. Drive the virtual support in the virtual monitoring model according to the optimal value of the driving variable; perform a full pose calculation according to the optimal value of the driving variable to obtain the pose state of the virtual support consistent with the current shape of the hydraulic support as the target state and send it to the virtual control model; the virtual control model is used to determine the difference between the target state of the virtual support and the current state of the virtual support, determine the driving force according to the difference, and use the driving force to drive the virtual support in the virtual control model to act; the virtual support is a digital twin model of the hydraulic support.

[0036] When the virtual monitoring model only receives physical instructions, the sensing module includes multiple pressure sensors. The multiple pressure sensors are used to monitor the pressure at different positions of the hydraulic support; the virtual control model is used to receive the control instructions issued by the electro-hydraulic controller, and after receiving the control instructions, predict the optimal virtual force according to the pressure at different positions by using the optimal force Agent model, and then drive the virtual support in the virtual control model to act synchronously with the hydraulic support according to the control instructions and the optimal virtual force, and send the state of the virtual support after the synchronous action is completed to the virtual monitoring model; the virtual monitoring model is used to perform a full pose calculation on the received state and control the pose state of the virtual support in the virtual monitoring model according to the data after the full pose calculation.

[0037] When the virtual monitoring model receives sensing information and physical instructions simultaneously, the sensing module includes a plurality of the pose sensors and a plurality of the pressure sensors. The dual-drive Agent model uses an adaptive control strategy to fuse the control instructions and pose data to obtain the fused pose data; determines the weights of the drive variables corresponding to the respective pose data by using a sensor-weighted adaptive hybrid algorithm according to the fused pose data, and predicts the optimal values of the drive variables according to the weights; trains a model with speed parameters to predict the speed parameters of the movement of the hydraulic support cylinders, and interpolates the optimal values of the drive variables according to the predicted speed parameters, so that the virtual support in the virtual monitoring model moves at the speed of the hydraulic support; the dual-drive Agent model is also used to, after the virtual control model receives the control instructions, predict the optimal virtual force by using the optimal force Agent model according to the pressures at different positions, and then drive the virtual support in the virtual control model to move synchronously with the hydraulic support according to the control instructions and the optimal virtual force.

[0038] Exemplarily, the virtual monitoring model, the virtual control model, and the dual-drive Agent model are constructed in the simulation software Unity3D.

[0039] As an alternative implementation, the pose sensor is an inclination sensor. A plurality of inclination sensors are respectively arranged at the rear connecting rod, the roof beam, and the shield beam of the hydraulic support. The plurality of inclination sensors are used to collect the inclination data of the rear connecting rod, the roof beam, and the shield beam.

[0040] As an alternative implementation, the integrated sensing-control twin device for fully-mechanized mining equipment further includes: an actuator. The electro-hydraulic controller controls the actuator to perform actions according to the control instructions, so that the pose of the hydraulic support is changed.

[0041] As an alternative implementation, the integrated sensing-control twin device for fully-mechanized mining equipment further includes: a data acquisition module; the data acquisition module is implanted in the control unit; the data acquisition module is used to collect the pose data of a plurality of pose sensors and the pressures monitored by a plurality of pressure sensors, and after correcting and filtering the collected pose data and pressures, transmit them to the virtual monitoring model.

[0042] Both the sensing and monitoring data variables in the physical space and the equipment operation control variables can be integrated in the PLC. By reading and writing the data in the DB block of the PLC, the host computer can remotely obtain the sensing data of the hydraulic support group. The data obtained by sensors such as inclination sensors is imported into the simulation software Unity3D through the construction of a real-time communication interface as the subsequent simulation data source. The main programming software used is Visual Studio and third-party libraries such as pcl. The inclination sensor communicates with the host computer through serial communication, and a C# program is written to read the data of the sensor. The sensor data is transmitted to Unity3D using the TCP communication protocol. After the virtual monitoring model receives the sensor data, it modifies the rotation angle or position of the corresponding components of the virtual support to achieve the simulation of the moving support.

[0043] The data processing of the inclination sensor mainly includes data correction and filtering, and a sensor fusion algorithm (such as Kalman filtering or particle filtering) is used to reduce conflicts. The inclination sensor obtains data by measuring the change in the tilt angle of the device relative to the earth's gravity field. Most inclination sensors work on the principle of an accelerometer, measuring the change in the direction of acceleration, which is closely related to the tilt angle of the device. The raw data collected may be affected by noise, so filtering is usually required. A low-pass filter is used to remove high-frequency noise. Kalman filtering is used to optimize the estimation of the noisy data. To ensure the accuracy of the sensor measurement results, the sensor usually needs to be calibrated. The errors of the sensor may include bias error, non-linear error, etc. Calibration usually involves: zero calibration: measuring the output value when the device is in a horizontal state and correcting the deviation. Temperature compensation: adjusting the influence of temperature change on the inclination measurement according to the temperature characteristics of the sensor.

[0044] The communication between the data acquisition module and Unity3D includes a variety of communication methods according to different application scenarios, environmental conditions, data transmission requirements, and real-time requirements.

[0045] The wireless communication network Wi-Fi uses Wi-Fi-enabled sensors or microcontrollers (such as ESP8266 or ESP32). The microcontroller connects to a network via Wi-Fi and runs a web server. The microcontroller can listen on a specified port via the HTTP protocol, receive HTTP requests from Unity3D, and return responses. In Unity3D, use the Unity3DWebRequest class to send HTTP requests and receive data from the microcontroller. Write firmware to send data to the server via the HTTP protocol, and the Unity3D side receives data through the network library. The wireless communication network Wi-Fi is highly flexible, easy to deploy, and supports multi-device connections; underground in mines, wireless communication can avoid the limitations of physical connections. In some places where it is not easy to lay cables, using wireless communication can reduce the construction difficulty and cost. However, this communication method is greatly affected by environmental factors (such as signal interference and obstacles).

[0046] TCP / IP communication uses a microcontroller with an Ethernet interface (such as RaspberryPi). Set up a TCP / IP server on the device to receive sensor data. The Unity3D side uses the TCPClient class to establish a connection and process and visualize the received data. TCP / IP communication is suitable for wired network environments. In the case of a stable wired network, using TCP / IP can achieve high-speed and reliable data transmission. The TCP / IP protocol can ensure the integrity and order of data, making it suitable for applications with high requirements for data transmission reliability. However, this communication method requires cabling and network infrastructure. In a complex network environment, more configuration and management are needed.

[0047] Serial communication connects the serial port interface of the hydraulic support data acquisition module to the USB interface of the computer. Use the RS232 serial protocol for data transmission. This communication method is suitable for short-distance communication and also has a Wi-Fi module for scenarios that require remote monitoring. In the C# script of Unity3D, use the serialPort class to create a serial port object and configure the serial port parameters. Ensure that the serial port settings (COM port, baud rate) in Unity3D are consistent with the data acquisition module. Use UI components to achieve the visualization of monitored objects. Serial communication is suitable for short-distance transmission, generally for occasions where data is transmitted within a few meters; suitable for connecting to simple devices such as single-chip microcontrollers and sensors; simple to implement and low in cost. It has good real-time performance and is suitable for transmitting a small amount of data. However, its transmission distance is limited. It is not suitable for connecting to large-scale or complex networks.

[0048] When selecting the communication method between the data acquisition module and Unity3D, the following factors need to be considered comprehensively: Environmental conditions: Whether there is wireless signal interference, wiring difficulty, etc. Data volume: Whether a large amount of data needs to be transmitted and what the real-time requirements are. Device type: The compatibility between the sensor and the main control device and the interface type. Considering the above factors, flexibly select the most suitable communication method or adopt a combination of multiple communications to ensure the smooth transmission of sensor data into Unity3D. Using one or more of the above communication methods, transmit the data of the inclination angles of the rear connecting rod, the top beam, and the shield beam collected by the inclination sensor to the virtual monitoring model constructed by Unity3D to achieve the communication between the acquisition module and Unity3D.

[0049] The device of this application has three working modes: the sensing information-driven mode, the physical instruction-driven mode, and the dual-driven mode. The sensing information-driven mode is used for the working conditions where pressure sensors and pose sensors are installed on the support. The data acquisition module collects the real-time sensing data of the hydraulic support to facilitate the realization of the "sensing-control" integration and consistent form when the physical instruction information is lacking. The physical instruction-driven mode is used for the working conditions where the sensing information is missing. By training the speed parameter model and the optimal force Agent model, the current posture is inferred to achieve virtual-real synchronization. The dual-driven mode is used for the working conditions where both the physical instruction transmission path and the sensing information transmission path are intact, and according to the adaptive control strategy, the virtual monitoring model and the virtual control model perform real-time consistent actions and regulations.

[0050] The following is a detailed introduction to each working mode.

[0051] (1) Sensing information-driven mode

[0052] As Figure 2As shown in the figure, the sensing information-driven mode means that the electro-hydraulic controller of the hydraulic support issues instructions to control the actuator to perform corresponding actions, the pose of the hydraulic support changes, and multiple sensors installed on it collect information in real time. The information of multiple sensors of multiple hydraulic supports transmits data to the data acquisition module in real time, and the collected data is transmitted to the virtual monitoring model through the serial communication port written by Unity3D. According to the sensor weighted adaptive hybrid algorithm of the sensor, the weight of the driving variable corresponding to each sensing information is calculated, and the optimal value of the driving variable of the virtual monitoring model is predicted. According to this optimal value, full pose calculation is performed in the virtual monitoring model to obtain the pose state of the virtual hydraulic support consistent with the current physical support shape, that is, the target state Sd (Destination state). The current state Sc (Current State) is transmitted to the virtual control model through the MQTT protocol, and the difference is made with the value of each pose of the current state Sc of the virtual control model. According to the size of the difference, different magnitudes of forces are assigned layer by layer. The larger the difference, the greater the force. Thus, the rapid response of the virtual control model to the virtual monitoring model is realized, and both are synchronized with the physical entity.

[0053] The virtual monitoring Agent model includes an MQTT protocol communication module and a sensor weighted adaptive hybrid algorithm. The MQTT protocol communication module first needs to download the MQTTnet.DLL file and import it into the Unity3D project. Write an MQTT protocol proxy (server) for the client through the Factory class in the MQTTnet library; write a client script, set the client theme and topic, and configure the script into the Unity3D project. Run the project in the virtual control model, input the server IP to connect it to the MQTT protocol communication server. Run the project in the virtual monitoring model and set the IP address, and it will connect to the MQTT protocol communication server. Write a communication script in another client, set the server IP and port, subscribe to the theme published by the virtual control model client, and receive the message sent by the virtual control model project. Set the theme and topic in the virtual monitoring model and send messages. In this way, the mutual communication between the two models is realized.

[0054] The sensor weighted adaptive hybrid algorithm uses the Kalman filter method. After establishing the state equation and observation equation of the system, according to the dynamic model of the system, the current state estimate and control input are used to predict the state and its uncertainty (covariance matrix) at the next moment. The new observation values obtained from the sensors are used to update the state estimate and covariance matrix. The key lies in adjusting the weights of different sensors through the error covariance matrix of the sensors. In multi-sensor fusion, the Kalman filter minimizes the error covariance matrices of each sensor through a recursive method, combined with the system dynamic model and observation model, to dynamically adjust the weights, and can optimize the state according to the current predicted value and actual observation value at each step, so as to achieve adaptive weighting.

[0055] Using the weight ratios corresponding to the inclination angles of the rear link, top beam, and shield beam calculated by the sensor weighted adaptive hybrid algorithm, the optimal drive variable value is predicted. It is transmitted to the monitoring model, and the full pose solution is carried out by using the four-bar linkage structure analysis of the hydraulic support. The solved pose is used as the target support state Sd, and is sent to the virtual control model through the MQTT protocol communication module for comparison and difference calculation to achieve the corresponding response of the subsequent control model.

[0056] Specifically, the execution process of the sensor weighted adaptive hybrid algorithm of the sensor includes multiple stages. First, in the initialization stage, the system sets the initial state for each sensor, including the initialization of the state estimate value and the error covariance matrix. The state estimate value is usually the measurement result of the sensor at a certain moment, and the error covariance matrix reflects the magnitude of the sensor estimation error. At the same time, the dynamic model of the system and the mutual relationship between the sensors are set, and the weights of all sensors are initialized to be equal. In the sensor data acquisition and real-time observation stage, multiple sensors acquire data in real time. The data includes the attitude information of different parts of the hydraulic support. Since the timestamps of the sensors may be different, time synchronization is required to ensure the reliability of all sensor data at the same time point. Next, the algorithm dynamically calculates the weights of the sensors according to the performance, reliability, and accuracy of the sensors. The calculation of the weights is based on the error covariance matrix of the sensors. The smaller the error covariance, the higher the weight, indicating that the observation data of this sensor has a greater impact on the final result. Dynamically adjusting the weights ensures that more reliable sensors are preferred under different conditions.

[0057] In the system state prediction stage, the dynamic model of the system is used to predict the current state, taking into account the influence of factors such as acceleration and external disturbances. Based on the state estimation and control input at the previous moment, the predicted value and the predicted error covariance matrix are calculated to provide a reference for the subsequent fusion stage. In the weighted fusion stage, algorithms such as weighted average or Kalman filtering are used to perform weighted fusion on the observation data of multiple sensors according to the dynamically calculated weights, thereby generating a more accurate state estimation value. The purpose of weighted fusion is to reduce the influence of observation noise and system errors. In the state estimation update stage, based on the new observation data and the predicted state, the state estimation of the system is updated. By combining the fused data, the predicted state value, and the error covariance matrix, the Kalman gain is used to optimize the state estimation to make it closer to the actual situation and reduce the bias caused by inaccurate sensor data. As new data is input, the system will dynamically adjust the weights of the sensors. Each time the data is updated, the algorithm will recalculate the weights according to the errors and reliabilities of the sensors to ensure the continuous high precision and reliability of the fusion result.

[0058] The entire process will iterate continuously. When new sensor data arrives, the sensor weights will be continuously adjusted in real time. Through this continuously optimized process, the system can track the state in real time, optimize the state estimation value, ensure that the estimation result is consistent with the sensor data, and adaptively adjust according to environmental changes to cope with situations such as external environment, sensor failures, or accuracy changes.

[0059] The equations involved in the sensor weighted adaptive hybrid algorithm are as follows:

[0060] 1. The state variables are shown in Table 1.

[0061] Table 1 State Variables

[0062]

[0063] 2. The input variables are shown in Table 2.

[0064] Table 2 Input Variables

[0065]

[0066] 3. The observation variables are shown in Table 3.

[0067] Table 3 Observation Variables

[0068]

[0069]

[0070] II. Integrated Mathematical Model

[0071] 1. The state equation (including surrounding rock pressure, valve delay, and cylinder pressure) is:

[0072]

[0073] The parameter descriptions of the state equation are shown in Table 4.

[0074] Table 4 Parameter Descriptions of the State Equation

[0075]

[0076] Dynamic coupling of surrounding rock pressure: Incorporate the non - linear interference of the support inclination angle into the state equation through α1 for the surrounding rock pressure P. Among them, the value of α1 is calibrated by rock mechanics experiments. rock

[0077] Explicit modeling of valve delay: The oil cylinder pressure term P cyl (t - τ valve ) directly represents the influence of the solenoid valve command transmission delay τ valve .

[0078] Vibration interference compensation: The vibration intensity V corrects the dynamic errors of the inclination angle and oil pressure through α2 (calibrated by spectrum analysis) and γ (calibrated by hydraulic pipeline transmission experiment) respectively.

[0079] 2. The observation equation is as follows:

[0080]

[0081] The parameter descriptions of the observation equation are shown in Table 5.

[0082] Table 5 Parameter Descriptions of the Observation Equation

[0083]

[0084] Geometric constraint matrix: Based on the mechanical kinematic constraint relationship constructed by the articulated angle φ of the shield beam (φ is a support structure design parameter), eliminate the measurement coupling error between the top beam and the rear connecting rod sensors.

[0085] Oil cylinder pressure observation channel: Incorporate the pressure sensor data P through η (the pressure - displacement conversion coefficient is calibrated by the oil cylinder size) meas into the observation system to achieve cross - verification of the hydraulic system state and the mechanical attitude.

[0086] III. Noise Covariance Model

[0087] 1. The process noise covariance is as follows:

[0088]

[0089] The parameter descriptions of the process noise covariance are shown in Table 6.

[0090] Table 6 Parameter Descriptions of the Process Noise Covariance

[0091]

[0092]

[0093] Process noise variance Vary with the surrounding rock pressure P rock and the vibration intensity V dynamically, and automatically enhance the robustness of the filter when the roof weighting occurs.

[0094] 2. The measurement noise covariance is as follows:

[0095]

[0096] The parameter description of the measurement noise covariance is shown in Table 7.

[0097] Table 7 Parameter description of the measurement noise covariance

[0098]

[0099] Diagonal elements Are dynamically updated through a weighting algorithm to achieve real-time evaluation of sensor reliability.

[0100] IV. Core algorithm module

[0101] 1. Weighted adaptive hybrid algorithm

[0102] Dynamic weight calculation:

[0103]

[0104] In the formula, |P rock |η i is the surrounding rock pressure interference compensation term of the i-th pose sensor; θ i is the pose data of the i-th pose sensor, is the estimated value of the pose data of the i-th pose sensor; κ|τ valve | is the valve delay compensation term.

[0105] 2. Time-delay compensation observer:

[0106]

[0107] Compensate the pressure accumulation error during the valve delay through an integrator. The parameter description of the time-delay compensation observer is shown in Table 8.

[0108] Table 8 Parameter description of the time-delay compensation observer

[0109] Variable symbol Physical meaning Mathematical expression u(t) Control instruction input (desired oil pressure) Output from the controller ξ Integration variable (dummy variable for time integration) <![CDATA[Integration interval from t - τ valve to t]]>

[0110] Example 1: Core Parameter Calibration Method

[0111] (1) Calibration of Surrounding Rock Pressure - Dip Angle Coupling Coefficient α1

[0112] Experimental Equipment: MTS 815 Rock Mechanics Testing Machine (loading accuracy ±0.5MPa), dip angle sensor array (resolution 0.01°), prototype hydraulic support.

[0113] Experimental Steps:

[0114] 1. Install the support on the testing machine, and make the contact surface between the top beam and the simulated surrounding rock fit.

[0115] 2. Apply gradient surrounding rock pressure (5MPa → 20MPa, step size 5MPa), and keep the vibration intensity V = 0.

[0116] 3. Record the variation data of the inclination angle θ of the rear connecting rod with P rock during steady state.

[0117] 4. Fit the formula by the least - squares method: where the typical value: α1 = 0.12 N·m / Pa.

[0118] Data Processing: Eliminate the vibration interference data and take the average value of three experiments.

[0119] (2) Calibration of Vibration Interference Coefficient α2

[0120] Experimental Equipment: Three - axis vibration table (frequency range 0 - 100Hz), laser displacement sensor, accelerometer.

[0121] Experimental Steps:

[0122] 1. Fix the support on the vibration table and set the vibration intensity V = 1 - 5 m / s 2 (simulating the underground working conditions);

[0123] 2. Turn off the hydraulic system (P cyl = 0), and record the variation curve of θ with V.

[0124] 3. Calculate the inclination angle offset rate caused by vibration: where the typical value: α2 = 0.03 rad / (m / s 2 ).

[0125] Innovation Point: Use white - noise vibration spectrum to simulate the actual random vibration in the underground mine.

[0126] (3) Calibration of Cylinder Pressure Response Coefficient β

[0127] Cited Standard: Section 5.2 "Step Response Method" of "Testing of Dynamic Characteristics of Hydraulic Systems".

[0128] Brief description: Apply a step control signal u to the oil cylinder (t) , and record P through a pressure sensor cyl (t)'s exponential decay curve, calculate: where, t 63％ is the time required for the pressure to reach 63% of the steady-state value, and the typical value of β = 0.81 / s.

[0129] (4) Calibration of the vibration-oil pressure transfer coefficient γ

[0130] Experimental steps:

[0131] 1. Apply a fixed-frequency vibration (f = 10Hz, V = 2m / s 2 ) to the vibration table;

[0132] 2. Measure the amplitude of the oil cylinder pressure fluctuation ΔP cyl ;

[0133] 3. Calculate the transfer coefficient: where, the typical value: γ = 0.05Pa / (m / s 2 ).

[0134] Data verification: Compare the γ values at different frequencies (5Hz, 20Hz, 50Hz) and take the average value.

[0135] Example 2: Calibration of dynamic parameters

[0136] (1) Process noise variance Calibration of dynamic relationship

[0137] Experimental conditions: Apply the surrounding rock pressure (P rock = 5 - 15MPa) and vibration (V = 1 - 4m / s 2 ) synchronously, with a total of 25 working conditions.

[0138] Data processing:

[0139] 1. Record the oil pressure estimation error ΔP cyl = |P cyl - P mes |;

[0140] 2. Establish a model through multiple linear regression:

[0141] 3. Verify that R 2 > 0.85 is considered valid.

[0142] (2) Measurement of the weighting coefficient λ i Calculation rule

[0143] Residual feedback term δ iCalibration: During the dynamic movement of the support, the residuals of each sensor are statistically analyzed:

[0144]

[0145] where N = 1000 sampling points, and δ i <0.1 rad is considered reliable.

[0146] Calibration of the valve delay compensation coefficient κ: Measured through a step response experiment and calculated as: where τ0 = 0.1 s is the reference delay, and k = 0.2 - 0.8.

[0147] Example 3: Description of reference type parameters

[0148] (1) Calculation of the moment of inertia J

[0149] The moment of inertia J of the rear connecting rod is directly obtained through the mass property analysis module of 3D modeling software (SolidWorks 2023), and J = 128.5 kg·m 2 .

[0150] (2) Damping coefficient C d Identification

[0151] Through the free decay experiment of the hydraulic support, the envelope of the angular velocity ω (t) is measured, and C d = 32.4 N·m·s / rad is fitted.

[0152] Therefore, according to the pose data at different positions, the weights of the driving variables corresponding to each pose data are determined by using the sensor weighted adaptive hybrid algorithm, and the optimal value of the driving variable is predicted according to the weights, which can be replaced by the following steps 101 to 103:

[0153] Step 101: According to the pose data at different positions, the state of the hydraulic support is estimated by using the Kalman filter to obtain the estimated value of the pose data.

[0154] Step 102: According to the pose data at different positions and the estimated value of the pose data, use the formula to calculate the weights of the driving variables corresponding to each pose data.

[0155] Step 103: According to the weights of the driving variables corresponding to each pose data, perform weighted fusion of the pose data at different positions to generate the optimal value of the driving variable.

[0156] The full pose calculation uses a seamless linkage method. By analyzing the four-bar mechanism of the hydraulic support, the coordinated analysis of the four-bar mechanism and the roof beam, and the coordinated analysis of the four-bar mechanism, the roof beam, and the front and rear columns, the function of analyzing all pose data of the support based on the inclination angles of the rear connecting rod, the roof beam, and the shield beam monitored by the inclination sensors is realized. Given the structural parameters such as L1, L2, L3, L4, etc. and θ and Then there are:

[0157]

[0158] Among them, L1 is the distance between the connecting pins of the front and rear connecting rods and the shield beam; L2 is the length of the front connecting rod; L3 is the length of the rear connecting rod; L4 is the distance between the connecting pins of the front and rear connecting rods and the base. θ is the inclination angle of the rear connecting rod; is the horizontal inclination angle of the connecting pin line between the front and rear connecting rods and the base; β is the inclination angle of the front connecting rod; γ is the inclination angle of the shield beam; η is the angle between the front column of the base and the base.

[0159] The solution is:

[0160]

[0161] Among them, the intermediate variables a, b, and c are respectively:

[0162]

[0163] Combined with the inclination angle δ of the roof beam, by analyzing the structures of the roof beam and the base respectively, the coordinates of the pin point C at the top of the front column, the pin point D of the rear column, the pin point A at the bottom of the front column body, and the pin point B of the rear column body in the base coordinate system (with the pin point of the rear connecting rod as the origin) can be determined.

[0164]

[0165] Among them: X AC represents the horizontal distance between the pin point A at the bottom of the front column body and the pin point C at the top of the front column, and Y AC represents the vertical distance between the pin point A at the bottom of the front column body and the pin point C at the top of the front column.

[0166]

[0167] L AC原始 represents the original length of the column cylinder before elongation, and L 伸长 represents the difference between before and after the elongation of the column cylinder, that is, the elongation amount.

[0168] In this way, the angle η between the front column and the base can be obtained, as well as the telescopic length of the column during this process. The height H of the hydraulic support can also be obtained.

[0169]

[0170] Using the above full pose calculation method, the full pose of the hydraulic support can be obtained, including the inclination angle of the shield beam, the inclination angle of the top beam, the inclination angles of the front and rear connecting rods, the angles of the front and rear columns, and the elongation.

[0171] The rapid response of the virtual control model is achieved after the full pose calculation of the virtual monitoring model. Through the script mounted on the monitoring model, the monitoring model is driven to the target position in a data-driven manner. The current pose of each component of the hydraulic support in the monitoring model is defined as the target state Sd, and the current state of the support in the control model presented by the virtual sensor in the control model is defined as Sc. Sd is sent to the virtual control model through the MQTT protocol and compared with Sc. According to the size of the difference, the driving force of the virtual control model is set. A large difference results in a large driving force, and a small difference results in a small driving force, thus achieving the rapid response of the virtual control model.

[0172] (2) Physical instruction-driven mode

[0173] As Figure 3 shown, the physical instruction-driven mode means that the electro-hydraulic controller of the physical hydraulic support issues a control instruction digital code, and this signal is transmitted to the Unity3D virtual control model via the MQTT protocol. Each digital code represents an action instruction. Based on the oil cylinder pressure information monitored by the pressure sensor, the surrounding rock pressure, and comprehensively considering factors such as the delay of the switching valve, the optimal force Agent model predicts the magnitude of the driving force of the virtual model that best suits the current working conditions, and commands the virtual control model to act synchronously with the physical entity. The virtual sensor information in the control model is sent to the virtual monitoring model via the MQTT protocol. Through the full pose calculation of the received data, the Transform component of the corresponding component is controlled to control the synchronous action of the virtual monitoring model. Thus, in the case of a lack of sensors, only physical control instructions can also simulate sensor information. The pose data of each component of the virtual monitoring model is the simulated sensing information. It includes the release and transmission of control instructions, the virtual control Agent model, sending virtual sensor information, full pose calculation, and obtaining simulated sensor information.

[0174] The release and transmission of control instructions refer to transmitting the instructions of the electro-hydraulic controller of the hydraulic support to Unity3D. First, the electro-hydraulic controller needs to be connected to the computer through a suitable interface (such as serial communication, TCP / IP protocol, etc.). Secondly, a program for collecting the signals of the electro-hydraulic controller on the computer is written in C# language to achieve data collection and parsing. Through the C# script of Unity3D, the received data is transmitted to Unity3D through TCP / IP, UDP or serial communication. Unity3D supports multiple network communication methods, and the built-in System.IO.Ports.SerialPort class or custom Socket communication of Unity3D can be used. After receiving the instructions, in Unity3D, the transmitted data is parsed according to the written script, and the actions of the virtual hydraulic support model are controlled according to these instructions.

[0175] The virtual control Agent model includes an MQTT protocol communication module, a speed parameter training model, and an optimal force Agent model. The optimal force Agent model uses knowledge such as the positioning of the hydraulic support and the kinematic model of a single support in Unity3D, combined with data such as inclination angle and real-time displacement, to drive the virtual geometric model of the physical hydraulic support, so as to achieve the consistency of the physical hydraulic support and the virtual support in shape. Reserve the association between the attitude drive variable and real-time pose and other data, and drive the virtual hydraulic support model to perform actions synchronously, then the effect of the spatio-temporal consistency between the physical hydraulic support and the hydraulic support model can be achieved. Import the well-modeled hydraulic support model in UG in Unity3D as the rendering part of the virtual control model, and add a collision body to it. Add a driving force to the model, set the range of the force, traverse the magnitudes of the forces within this range, and use GetComponent <rigidbody>().velocity obtains the movement speed of the oil cylinder, and compares it with the speed predicted by the above speed parameter training model. If the difference in velocity.magnitude between the two is within 1%, the traversal of the force stops, and the magnitude of the optimal force is output. It is sent to the virtual control model through the MQTT protocol communication server. When the Unity3D virtual control model receives the action instruction and calls the corresponding action method, the force applied to the model is changed to the above optimal driving force, so as to achieve the highest degree of fidelity. Then, according to the pressures at different positions, the optimal virtual force is predicted using the optimal force Agent model, which can be replaced by the following steps 201 to step 206:

[0176] Step 201: Set the simulation value range of the driving force, and select any driving force within the simulation value range of the driving force.

[0177] Step 202: Add the selected driving force to the virtual support in the virtual control model, and simulate to obtain the corresponding movement speed of the oil cylinder; the oil cylinder includes a left column oil cylinder, a right column oil cylinder, a balance jack oil cylinder, and a push oil cylinder.

[0178] Step 203: According to the oil cylinder pressure from the underground, use the speed parameter training model to predict the movement speed of the oil cylinder; the speed parameter training model is a multi-layer perceptron.

[0179] The speed parameter training model is used to perform speed parameter interpolation on the data-driven process of the monitoring model and provide a speed reference value for the optimal force Agent model. It is a key technical means for the implementation of the dual-drive mode.

[0180] The speed parameter training model first needs to process the existing real column oil cylinder pressures, surrounding rock pressures and corresponding movement speed parameters from the underground, push oil cylinder pressures, surrounding rock pressures and corresponding movement speed parameters, balance jack oil cylinder pressures, surrounding rock pressures and corresponding movement speed parameters. For missing values, select similar values to supplement the group of data, and remove the group of data if there is no approximate value. Then store the processed data in a matrix. To ensure the randomness of the data, a random number generator can be used to shuffle the order of the data. Divide the data set: 70% for the training set, 15% for the validation set, and 15% for the test set.

[0181] Input the partitioned data. Since the Multi-Layer Perceptron (MLP) is relatively easy to implement and train and has a powerful representation ability: with a sufficient number of hidden layers and neurons, it can approximate any continuous function and is suitable for prediction problems, so the neural network model architecture of the multi-layer perceptron is selected. Set 4 neurons in the input layer, corresponding to the pressure of the left leg cylinder, the pressure of the right leg cylinder, the pressure of the balance jack cylinder, and the pressure of the pushing cylinder respectively; 15 neurons in one hidden layer; 4 neurons in the output layer, corresponding to the speed parameters of the left leg cylinder, the speed parameters of the right leg cylinder, the speed parameters of the balance jack cylinder, and the speed parameters of the pushing cylinder respectively.

[0182] Select the most commonly used ReLU (Rectified Linear Unit) activation function. The formula of ReLU is f(x) = max(0, x), and the calculation is very simple, which can accelerate the forward propagation and backward propagation processes. x in the formula of ReLU represents the input, and f() represents the ReLU activation function. At the same time, it can also alleviate the problem of gradient disappearance: in the positive interval, the derivative of ReLU is always 1, which makes it not easy for the network to have the phenomenon of gradient disappearance during training, especially performing well in deep networks. The Mean Squared Error (MSE) loss function is more sensitive to outliers because the squared term will amplify large errors, so the mean squared error loss function is selected.

[0183] L2 regularization is achieved by adding a penalty term of the L2 norm of the weight parameter to the loss function of the model. In L2 regularization, the penalty term is usually defined as the square of the L2 norm of the weight parameter. Specifically, the loss function of L2 regularization can be expressed as:

[0184]

[0185] where: L data is the data loss of the model, which is the error between the predicted value and the true label of the model. λ is the regularization parameter, used to control the strength of regularization. is the square of the L2 norm of the weight vector, expressed as the sum of the squares of each parameter in the weight vector.

[0186] Use the loss function of L2 regularization. The optimization algorithm will consider both the data loss and the regularization term during the optimization process, so as to minimize the size of the model parameters and reduce the complexity of the model while maintaining the fitting ability to the training data.

[0187] Select the Adam optimizer, pass the model parameters to the Adam optimizer, and set the learning rate and other hyperparameters - set the number of epochs to 100, the batch size to 32, and perform multiple iterations (epochs). During each epoch, train on each batch of data. Evaluate the model performance on the test set to monitor overfitting and the generalization ability of the model. The Adam optimizer tunes the hyperparameters based on the model performance: according to the performance on the validation set, adjust the learning rate, batch size, and other hyperparameters to improve the model performance. After training is completed with the most suitable model performance, save the model.

[0188] Use the exportONNXNetwork function in MATLAB to export the trained neural network model to the ONNX format. Unity3D supports ONNX models. Use the Barracuda library in Unity3D to load and run the ONNX model, thus realizing the communication between Unity3D and the neural network training model.

[0189] Step 204: Determine the difference between the simulated cylinder movement speed and the predicted cylinder movement speed.

[0190] Step 205: If the difference in the cylinder movement speed is greater than the preset difference threshold, select another driving force within the driving force simulation value range, and return to Step 202.

[0191] Step 206: If the difference in the cylinder movement speed is less than or equal to the preset difference threshold, determine the currently selected driving force as the optimal virtual force.

[0192] Sending virtual sensor information means constructing a virtual sensor using the transform.localPosition() and transform.localRotation() methods at the control model end and displaying the key poses of the current support. Send the pose data of the driving variables required by the monitoring model to the monitoring model through the MQTT protocol communication module, so that the monitoring model can respond.

[0193] Obtaining simulated sensor information is to obtain all the poses of the hydraulic support after full pose calculation, so that in the case of missing sensing information, simulated sensing information can be obtained, providing an initial pose reference for the next action instruction. Realize the monitorability of the action process of the hydraulic support and the transparency of the fully mechanized mining process.

[0194] (III) Dual-drive mode

[0195] As Figure 4 As shown, the dual-drive mode synthesizes the pressure and pose sensing information on the hydraulic support and the action instructions on the physical entity, and transmits them to the virtual monitoring model and the virtual control model respectively. The sensing information calculates the weights of each sensing data through the sensor weighted adaptive hybrid algorithm of the sensor, and the speed parameters of the cylinder movement under the current cylinder pressure and surrounding rock pressure; predicts the optimal value of the key drive variable of the virtual monitoring model according to the weight; interpolates the drive data according to the speed parameters to make the motion characteristics of the monitoring model closer to the actual support action speed. After receiving the control instruction, the virtual control terminal drives the virtual control model to act according to the optimal virtual force corresponding to the actual cylinder pressure obtained from the optimal force Agent model, and realizes the conflict resolution and redundant control between the sensing information and the physical instruction input through the adaptive control strategy, so as to realize the synchronous and consistent action of the virtual monitoring model and the virtual control model.

[0196] The dual-drive mode interpolates according to the speed parameters predicted by the dual-drive Agent model, so that the action speed of the monitoring model approaches the physical support; makes the action of the virtual control model approach the physical support through the magnitude of the optimal virtual force predicted by the dual-drive Agent model; realizes the conflict resolution and redundant control between the sensing information and the physical instruction input through the adaptive control strategy, thus promoting the synchronous and consistent action of the virtual monitoring model and the virtual control model, which is a necessary condition for realizing the "sensing-control" integration. It includes the issuance of control instructions and the collection of sensing information, the dual-drive Agent model, the speed parameter training model, the speed parameter interpolation method, and the adaptive control strategy.

[0197] The issuance of control instructions and the collection of sensing information include the collection and processing of sensing information in the sensing information drive mode, and the issuance and transmission of control instructions in the physical instruction drive mode. It is applicable to the working conditions where the sensing information path and the physical instruction path exist simultaneously.

[0198] The dual-drive Agent model includes an MQTT protocol communication module, a sensor weighted adaptive hybrid algorithm, and an optimal force Agent model. The sensor weighted adaptive hybrid algorithm is also used to predict the optimal drive variable of the monitoring model, while the optimal force Agent model is used to predict the optimal virtual force to drive the control model to act. The role of the MQTT protocol communication module is to realize the communication between the two, which is convenient for feedback adjustment.

[0199] The speed parameter interpolation method requires obtaining the current speed parameter currentSpeed from the speed parameter training model in each frame, multiplying this speed by Time.deltaTime to get the distance to be moved in each frame. Use the Vector3.MoveTowards method to move the object from the current position towards the target position, while listening to the object's current position and target position. After reaching the final target position, stop the movement, and the virtual monitoring model can achieve high-fidelity actions at an appropriate speed. Then, interpolating the optimal value of the driving variable based on the predicted speed parameter can be replaced by the following steps 301 to 302:

[0200] Step 301: Multiply the predicted speed parameter in each frame by time to obtain the distance to be moved in each frame.

[0201] Step 302: Interpolate the optimal value of the driving variable according to the distance to be moved in each frame.

[0202] The adaptive control strategy is for the dual-drive mode with both sensing information and control instructions. If the inclination sensor data has a high credibility, a higher weight is assigned, and the weight of the control instruction is correspondingly reduced. If the control instruction has a high credibility, a higher weight is assigned, and the weight of the sensor data is correspondingly reduced. If the credibility of both is comparable, the weights are dynamically adjusted according to the adaptive algorithm.

[0203] To implement the above strategy, it is first necessary to clarify the credibility evaluation indicators. For sensing information, there are: Data integrity: Whether the data is complete, whether there are missing or abnormal values. Data consistency: Whether the data is consistent with historical data, other sensor data, or the prediction results of the physical model. Sensor status: Whether the sensor is working properly, whether there are faults or drifts. Environmental interference: Whether environmental factors (such as temperature, humidity, electromagnetic interference) affect the sensor data. For control instructions, there are: Instruction rationality: Whether the instruction conforms to the current state and working logic of the system. Instruction source reliability: Whether the instruction source is reliable, whether there is a risk of being tampered with or misissued. Historical instruction consistency: Whether the current instruction is consistent with the historical instruction sequence, whether there are contradictions.

[0204] Secondly, based on the above indicators, a credibility evaluation model is designed. In terms of qualitative aspects, a rule-based model is designed, that is, according to expert experience and domain knowledge, a series of rules are formulated to judge the credibility of information. For example: if the sensor data exceeds the normal range, the credibility decreases. If the control instruction conflicts with the historical instruction sequence, the credibility decreases. In terms of quantitative aspects, a statistics-based model needs to be designed, that is, using historical data, statistically analyzing the distribution characteristics of sensor data and control instructions, and establishing a probability model to evaluate credibility. For example: using a Bayesian network, according to the prior probability and conditional probability of sensor data and control instructions, calculate the posterior probability as the credibility. Using Kalman filtering, according to the system model and observation data, estimate the system state and observation noise, so as to evaluate the credibility of information. Specifically, first, a Bayesian network needs to be constructed, where each node represents a different variable, such as sensor data and control instructions, and the edges represent the conditional dependence relationships between these variables. In practical applications, when the system obtains real-time sensor data and control instructions, the posterior probabilities of each variable in the network can be updated using Bayes' theorem, so as to evaluate the credibility of the system state. The higher the posterior probability, the more credible the information. This method effectively quantifies the credibility of information through reasoning and probability calculation. Secondly, Kalman filtering further evaluates the credibility of information by combining the system model and observation data to recursively estimate the system state. Kalman filtering is divided into two steps: prediction and update. In the prediction step, based on the state transition model of the system and the state estimate at the previous moment, the system state at the current moment is calculated and the error covariance is calculated. Then, in the update step, the new observation data is used to correct the state estimate, and the prediction error is updated according to the Kalman gain. The updated error covariance matrix can reflect the uncertainty of information, and the smaller the error, the higher the credibility of the information. By combining the Bayesian network and Kalman filtering, the system can comprehensively evaluate the credibility of information and enhance the accuracy and reliability of the estimation.

[0205] The dynamic adjustment of weights mainly includes the following three aspects of settings: First, time decay: The credibility of information decreases over time, so it is necessary to introduce a time decay factor to dynamically adjust the weights. Second, new information update: When new sensor data or control instructions are obtained, the credibility evaluation results need to be updated in a timely manner, and the weights are adjusted accordingly. Third, feedback mechanism: According to the actual operation of the system, feedback and optimization are carried out on the credibility evaluation model and the weight adjustment strategy. The feedback mechanism optimizes the credibility evaluation model and the weight adjustment strategy in real time according to the actual operation of the system. During the operation of the system, by monitoring the deviation between the actual result and the expected result, the weights and the evaluation model are dynamically adjusted. For example, if there are large errors repeatedly in some sensors or control instructions, the system will automatically identify and reduce the weight of this information source. On the contrary, if the performance of some information sources is good, the system will appropriately increase their weights. In addition, the feedback mechanism can also iteratively optimize the evaluation model through historical data and actual operation conditions, gradually improving the weight adjustment strategy, so as to more accurately predict and evaluate the information credibility and ensure the reliability and accuracy of the system in a dynamic environment.

[0206] Therefore, the adaptive control strategy is adopted to fuse the control instructions and the pose data to obtain the fused pose data, which can be replaced by the following steps 401 to 405:

[0207] Step 401: Determine the credibility evaluation index of the control instructions and the credibility evaluation index of the pose data.

[0208] Step 402: According to the credibility evaluation index of the control instructions and the credibility evaluation index of the pose data, use the rule-based model and the statistics-based model to calculate the credibility of the control instructions and the credibility of the pose data.

[0209] Step 403: If the credibility of the control instructions is not equal to the credibility of the pose data, then set the weight of the control instructions and the weight of the pose data according to the ratio of the credibility of the control instructions to the credibility of the pose data.

[0210] Step 404: If the credibility of the control instructions is equal to the credibility of the pose data, then use the sensor weighted adaptive hybrid algorithm to determine the weight of the control instructions and the weight of the pose data.

[0211] Step 405: According to the weight of the control instructions and the weight of the pose data, perform weighted fusion on the control instructions and the pose data to obtain the fused pose data.

[0212] The comparison of the three modes is shown in Table 9. The switching of the three modes is shown in Table 10.

[0213] Table 9 Mode comparison

[0214]

[0215]

[0216] Table 10 Mode Switching Method

[0217]

[0218] Note: ① in Table 10 represents the sensing information driven mode, ② represents the physical instruction control mode, and ③ represents the dual drive mode.

[0219] This application drives the high-fidelity actions of the virtual control model and the virtual monitoring model through physical entities, thereby completing the virtual reconstruction of the working space in the virtual space, and realizing the consistent actions of the two models through the neural network training algorithm and the Agent model.

[0220] Based on the same inventive concept, the embodiment of this application also provides a fully-mechanized mining equipment group sensing-control integrated twin method applied to the fully-mechanized mining equipment group sensing-control integrated twin device involved above. The implementation solutions provided by this method to solve problems are similar to the implementation solutions recorded in the above device. Therefore, the specific limitations in one or more embodiments of the fully-mechanized mining equipment group sensing-control integrated twin method provided below can refer to the limitations on the fully-mechanized mining equipment group sensing-control integrated twin device in the above text, and will not be repeated here.

[0221] In an exemplary embodiment, a fully-mechanized mining equipment group sensing-control integrated twin method is provided, including the following steps 501 to step 503.

[0222] Step 501: When only sensing information input is detected, the virtual monitoring model obtains the pose data of different positions on the hydraulic support; according to the pose data of different positions, the sensor weighted adaptive hybrid algorithm is used to determine the weights of the driving variables corresponding to each pose data, and the optimal value of the driving variable is predicted according to the weights. The virtual support in the virtual monitoring model is driven according to the optimal value of the driving variable; full pose calculation is performed according to the optimal value of the driving variable to obtain the pose state of the virtual support consistent with the current shape of the hydraulic support, which is used as the target state and sent to the virtual control model; the virtual control model determines the difference between the target state of the virtual support and the current state of the virtual support, determines the driving force according to the difference, and uses the set driving force to drive the virtual support in the virtual control model to act; the virtual support is the virtual model of the hydraulic support.

[0223] Step 502: When only physical instructions are detected to be input, the virtual control model receives the control instructions sent by the electro-hydraulic controller. After receiving the control instructions, according to the pressures at different positions, the optimal force Agent model is used to predict the optimal virtual force. Then, based on the control instructions and the optimal virtual force, the virtual support in the virtual control model is driven to act synchronously with the hydraulic support, and the state of the virtual support after the synchronous action is sent to the virtual monitoring model; the virtual monitoring model performs full pose calculation on the received state and controls the pose state of the virtual support in the virtual monitoring model according to the data after the full pose calculation.

[0224] Step 503: When both sensing information and physical instructions are detected to be input, the dual-drive Agent model uses an adaptive control strategy to fuse the control instructions and pose data to obtain the fused pose data; according to the fused pose data, the sensor weighted adaptive hybrid algorithm is used to determine the weights of the drive variables corresponding to each pose data, and the optimal values of the drive variables are predicted based on the weights; the velocity parameter training model is used to predict the velocity parameters of the movement of the hydraulic support cylinder, and the optimal values of the drive variables are interpolated based on the predicted velocity parameters, so that the virtual support in the virtual monitoring model moves at the speed of the hydraulic support; after the virtual control model receives the control instructions, the dual-drive Agent model also uses the optimal force Agent model to predict the optimal virtual force according to the pressures at different positions, and then drives the virtual support in the virtual control model to act synchronously with the hydraulic support based on the control instructions and the optimal virtual force.

[0225] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store sensing information and the state of the virtual support. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a comprehensive mining equipment group sensing-control integrated twin method.

[0226] Those skilled in the art can understand, Figure 5 The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0227] In an exemplary embodiment, a non-transitory computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0228] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0229] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0230] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0231] The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0232] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0233] In this article, specific examples are used to elaborate on the principles and implementation modes of this application. The description of the above embodiments is only used to help understand the method and its core idea of this application. At the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation modes and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.< / rigidbody>

Claims

1. A fully-mechanized mining equipment group sensing-control integrated twin device, characterized in that, The integrated sensing and control twin device of the fully-mechanized mining equipment group includes: an electro-hydraulic controller, a hydraulic support, a sensing module, and a control unit; The control unit is implanted with a virtual monitoring model, a virtual control model, and a dual-drive Agent model; When the virtual monitoring model only receives sensing information, the sensing module includes multiple pose sensors. The multiple pose sensors are used to collect the pose data of different positions on the hydraulic support after the electro-hydraulic controller controls the pose of the hydraulic support to change. The virtual monitoring model is used to determine the weights of the driving variables corresponding to each pose data by using the sensor weighted adaptive hybrid algorithm according to the pose data of different positions, and predict the optimal value of the driving variable according to the weights. The virtual support in the virtual monitoring model is driven according to the optimal value of the driving variable. The full pose calculation is performed according to the optimal value of the driving variable to obtain the pose state of the virtual support consistent with the current shape of the hydraulic support as the target state, and sent to the virtual control model. The virtual control model is used to determine the difference between the target state of the virtual support and the current state of the virtual support, determine the driving force according to the difference, and use the driving force to drive the virtual support in the virtual control model to act. The virtual support is a digital twin model of the hydraulic support; When the virtual monitoring model only receives physical instructions, the sensing module includes multiple pressure sensors. The multiple pressure sensors are used to monitor the pressures at different positions of the hydraulic support. The virtual control model is used to receive the control instructions issued by the electro-hydraulic controller, and after receiving the control instructions, predict the optimal virtual force by using the optimal force Agent model according to the pressures at different positions, and then drive the virtual support in the virtual control model to act synchronously with the hydraulic support according to the control instructions and the optimal virtual force, and send the state of the virtual support after the synchronous action ends to the virtual monitoring model. The virtual monitoring model is used to perform full pose calculation on the received state, and control the pose state of the virtual support in the virtual monitoring model according to the data after the full pose calculation; When the virtual monitoring model receives both sensing information and physical instructions at the same time, the sensing module includes multiple of the pose sensors and multiple of the pressure sensors. The dual-drive Agent model uses an adaptive control strategy to fuse the control instructions and the pose data to obtain the fused pose data. The weights of the driving variables corresponding to each pose data are determined by using the sensor weighted adaptive hybrid algorithm according to the fused pose data, and the optimal value of the driving variable is predicted according to the weights. The velocity parameter training model is used to predict the velocity parameters of the movement of the hydraulic support cylinder, and the optimal value of the driving variable is interpolated according to the predicted velocity parameters, so that the virtual support in the virtual monitoring model moves at the speed of the hydraulic support. The dual-drive Agent model is also used to predict the optimal virtual force by using the optimal force Agent model according to the pressures at different positions after the virtual control model receives the control instructions, and then drive the virtual support in the virtual control model to act synchronously with the hydraulic support according to the control instructions and the optimal virtual force.

2. The integrated group sense-control twin device for fully-mechanized mining equipment according to claim 1, wherein The integrated sensing and control twin device of the fully-mechanized mining equipment group further includes: an actuator; The electro-hydraulic controller controls the actuator to perform actions according to the control instructions, causing the pose of the hydraulic support to change.

3. The integrated perception and control twin device for fully-mechanized mining equipment according to claim 1, wherein, The fully-mechanized mining equipment group-sensing and control integrated twin device further includes: a data acquisition module; The data acquisition module is implanted in the control unit; The data acquisition module is used to collect the pose data of multiple pose sensors and the pressures monitored by multiple pressure sensors. After correcting and filtering the collected pose data and pressures, it is transmitted to the virtual monitoring model.

4. The integrated twin device for group sensing and control of fully mechanized mining equipment according to claim 1, characterized in that, The pose sensor is an inclination sensor; Multiple inclination sensors are respectively arranged at the rear connecting rod, the top beam and the shield beam of the hydraulic support; Multiple inclination sensors are used to collect the rear connecting rod inclination data, the top beam inclination data and the shield beam inclination data.

5. The integrated twin device for group sensing and control of fully-mechanized mining equipment according to claim 1, characterized in that, According to the pose data at different positions, the sensor weighted adaptive hybrid algorithm is used to determine the weights of the driving variables corresponding to each pose data, and the optimal value of the driving variable is predicted according to the weights, including: According to the pose data at different positions, the state of the hydraulic support is estimated by means of Kalman filtering to obtain the estimated value of the pose data; According to the pose data at different positions and the estimated values of the pose data, use the formula to calculate the weights of the driving variables corresponding to each pose data; in the formula, ∈ i is the basic reliability of the i-th pose sensor; η i is the pressure-noise transfer coefficient of the i-th pose sensor, P rock is the surrounding rock pressure, |P rock |η i is the surrounding rock pressure interference compensation term of the i-th pose sensor; δ i is the predicted residual feedback term of the i-th pose sensor, θ i is the pose data of the i-th pose sensor, is the estimated value of the pose data of the i-th pose sensor; κ is the delay sensitivity coefficient, τ valve is the valve control command time delay, κ|τ valve | is the valve delay compensation term; According to the weights of the driving variables corresponding to each pose data, the weighted fusion of the pose data at different positions is performed to generate the optimal value of the driving variable.

6. The integrated twin device for group sensing and control of fully-mechanized mining equipment according to claim 1, characterized in that, The target states include: shield beam inclination, top beam inclination, front connecting rod inclination, rear connecting rod inclination, front column angle, rear column angle, and the elongation of the column cylinder.

7. The integrated perception-control twin device for fully-mechanized mining equipment according to claim 1, wherein According to the pressures at different positions, the optimal virtual force is predicted using the optimal force Agent model, including: Set the simulation value range of the driving force, and select any driving force within the simulation value range of the driving force; Add the selected driving force to the virtual support in the virtual control model, and simulate to obtain the corresponding cylinder movement speed; the cylinders include the left column cylinder, the right column cylinder, the balance jack cylinder, and the push cylinder; According to the cylinder pressures from the underground, use the speed parameter training model to predict the cylinder movement speed; the speed parameter training model is a multi-layer perceptron; Determine the difference between the simulated cylinder movement speed and the predicted cylinder movement speed; If the difference in the cylinder movement speed is greater than the preset difference threshold, select another driving force within the simulation value range of the driving force, and return to the step "Add the selected driving force to the virtual support in the virtual control model, and simulate to obtain the corresponding cylinder movement speed"; If the difference in the cylinder movement speed is less than or equal to the preset difference threshold, determine the currently selected driving force as the optimal virtual force.

8. The integrated group sense-control twin device of fully-mechanized mining equipment according to claim 1, wherein, Interpolate the optimal value of the driving variable according to the predicted speed parameter, specifically including: Multiply the predicted speed parameter in each frame by the time to obtain the distance that should be moved in each frame; Interpolate the optimal value of the driving variable according to the distance that should be moved in each frame.

9. The integrated perception-control twin device for fully-mechanized mining equipment according to claim 1, characterized in that, An adaptive control strategy is used to fuse the control instruction and the pose data to obtain the fused pose data, specifically including: Determine the credibility evaluation index of the control instruction and the credibility evaluation index of the pose data; According to the credibility evaluation index of the control instruction and the credibility evaluation index of the pose data, use the rule-based model and the statistics-based model to calculate the credibility of the control instruction and the credibility of the pose data; If the credibility of the control instruction is not equal to the credibility of the pose data, set the weights of the control instruction and the pose data according to the ratio of the credibility of the control instruction to the credibility of the pose data; If the credibility of the control instruction is equal to the credibility of the pose data, use the sensor weighted adaptive hybrid algorithm to determine the weights of the control instruction and the pose data; According to the weights of the control instruction and the pose data, perform weighted fusion on the control instruction and the pose data to obtain the fused pose data.

10. A twin method for integrated perception and control of a fully-mechanized mining equipment group, characterized in that, Including: When only sensing information is detected to be input, the virtual monitoring model obtains the pose data of different positions on the hydraulic support; According to the pose data of different positions, use the sensor weighted adaptive hybrid algorithm to determine the weights of the driving variables corresponding to each pose data, and predict the optimal value of the driving variable according to the weights, and drive the virtual support in the virtual monitoring model according to the optimal value of the driving variable; Perform full pose calculation according to the optimal value of the driving variable to obtain the pose state of the virtual support consistent with the current shape of the hydraulic support, and use it as the target state and send it to the virtual control model; the virtual control model determines the difference between the target state of the virtual support and the current state of the virtual support, determines the driving force according to the difference, and uses the set driving force to drive the virtual support in the virtual control model to act; the virtual support is a virtual model of the hydraulic support; When only physical instructions are detected to be input, the virtual control model receives the control instructions issued by the electro-hydraulic controller, and after receiving the control instructions, according to the pressures at different positions, uses the optimal force Agent model to predict the optimal virtual force, and then drives the virtual support in the virtual control model to act synchronously with the hydraulic support according to the control instructions and the optimal virtual force, and sends the state of the virtual support after the synchronous action ends to the virtual monitoring model; the virtual monitoring model performs full pose calculation on the received state, and controls the pose state of the virtual support in the virtual monitoring model according to the data after the full pose calculation; When both sensing information and physical instructions are detected to be input, the dual-drive Agent model uses an adaptive control strategy to fuse the control instruction and the pose data to obtain the fused pose data; according to the fused pose data, use the sensor weighted adaptive hybrid algorithm to determine the weights of the driving variables corresponding to each pose data, and predict the optimal value of the driving variable according to the weights; Use the speed parameter training model to predict the speed parameters of the movement of the hydraulic support cylinder, and interpolate the optimal value of the driving variable according to the predicted speed parameters, so that the virtual support in the virtual monitoring model moves at the speed of the hydraulic support; the dual-drive Agent model also, after the virtual control model receives the control instructions, according to the pressures at different positions, uses the optimal force Agent model to predict the optimal virtual force, and then drives the virtual support in the virtual control model to act synchronously with the hydraulic support according to the control instructions and the optimal virtual force.

Images