Efficient virtual-real synchronization method for digital twin of integrated equipment for coal pillar mining and support between faces

Abstract

The present invention belongs to the technical field of coal mine safety monitoring, and relates to an efficient virtual-real synchronization method for digital twins of integrated equipment for inter-face coal pillar excavation and support. It includes: step one, lightweight three-dimensional model of digital twins of equipment; step two, establishing a motion behavior model of digital twins of equipment; step three, establishing an equipment trajectory prediction model: posture prediction of excavation and support equipment twins based on Self-Attention-LSTM; step four, establishing a virtual space equipment drive and correction model, posture correction based on quadratic spline interpolation method and a dynamic update method of digital twin platforms based on state synchronization; step five, associating the above models to establish virtual-real synchronization interaction of digital twin excavation and support equipment. The method combines compressed twin three-dimensional model data and trajectory prediction model to reduce the amount of driving data required for the digital twin system, more efficiently drive the movement of virtual space equipment, and can work stably even when sensor data acquisition is difficult and data communication is limited, with higher reliability and practicality.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal mine safety monitoring, and specifically relates to a digital twin efficient virtual-real synchronization method for integrated equipment for inter-face coal pillar excavation and support. Background Art

[0002] Coal mine intelligence is the core technology support for the high-quality development of the coal industry. Compared with the fully mechanized mining face, the width of the "inter-face coal pillar" between the longwall working faces is generally 15~40m, with narrow space and high dust. At present, it is mainly carried out by manual operation for strip mining, with a low degree of intelligence. The existing video remote intervention control is interfered by factors such as dust, and the remote control effect is not good.

[0003] Digital twin technology is currently a hot topic in the research of intelligent remote control of coal mine equipment. However, in practical applications, due to the complexity of equipment and the large amount of sensor data, the large scale of twin data, and the influence of mine communication delays, the efficiency of virtual-real interaction is low. On the one hand, the lightweight method of three-dimensional twin models does not take into account the maintenance of the mating surfaces between moving parts. When the model mating relationship is retained, the model simplification rate is low, and when the model simplification rate is high, the motion relationship between models needs to be rebuilt. On the other hand, the data-driven method focuses on data transmission optimization and is heavily dependent on mine communication networks. When sensor data is lost and communication is delayed, the virtual-real synchronization consistency, stability and real-time performance driven by digital twins cannot be guaranteed.

[0004] Therefore, the present invention proposes a digital twin efficient virtual-real synchronization method for integrated mining and support equipment, in order to solve the above problems. Summary of the invention

[0005] In order to solve the above problems, the present invention provides an efficient virtual-real synchronization method for digital twins of inter-face coal pillar mining and support integrated equipment. On the one hand, a lightweight method of a three-dimensional twin model based on an improved QEM algorithm is proposed while maintaining the assembly relationship between moving parts. On the other hand, a motion behavior prediction-correction method for integrated mining and support equipment based on Self-Attention-LSTM is proposed, a key motion parameter data interaction model based on multi-threading is established, and the state synchronization technology is used to fuse the optimized three-dimensional twin geometric model and behavior model to solve the problem of poor virtual-real synchronization of digital twin models.

[0006] In order to achieve the above-mentioned invention object, in the first aspect, the present invention proposes a digital twin efficient virtual-real synchronization method for integrated equipment of inter-face coal pillar excavation and support, comprising the following steps:

[0007] S1: establishing a three-dimensional model of a digital twin of equipment, recorded as a first model, and optimizing the first model;

[0008] S2: construct a motion behavior model of the equipment digital twin, recorded as the second model; analyze the second model to obtain a motion behavior parameter mapping matrix, and establish a digital twin driving model of the parameter-driven motion behavior, recorded as the third model;

[0009] S3: Establishing the trajectory prediction model of the equipment digital twin: performing data fusion and drive data division on the sensor system data, predicting the trajectory of the first model, and obtaining the drive parameters of the third model;

[0010] S4: Establishing a virtual space equipment driving and trajectory correction model: The posture data collected by the sensor system is processed by the prediction model in step S3, and then corrected by algorithm processing to obtain smooth driving trajectory data, and the trajectory data is used to drive the first model;

[0011] S5: Build a 3D model of the mining and support robot in a real-to-virtual ratio of 1:1, and after associating steps S1-S4, build a digital twin platform, use multi-threaded design status data, and realize the virtual-real synchronization of the digital twin of the integrated equipment for mining and support of inter-face coal pillars.

[0012] As a preferred technical solution of the present invention, the three-dimensional model in step S1 is lightweight optimized by:

[0013] Step 101, define cooperation-dependent vertices: when performing edge collapse processing, the deletion of vertices and the generation of related new vertices will destroy the mechanical cooperation relationship between components. Such vertices are called cooperation-dependent vertices;

[0014] Step 102, using the BFS algorithm to distinguish the vertex sets of each component of the three-dimensional model assembly according to whether the Mesh grid is continuous, and using the OBB bounding box and edge-face intersection detection to extract the three-dimensional model matching dependent vertices;

[0015] Step 103: Propose a collapsing factor of the dependent vertices based on the quadratic error metric topological compression algorithm, thereby improving the comprehensive topological collapse factor and controlling the collapse process of the edges of the triangle facets of the three-dimensional model;

[0016] Step 104: Evaluate the lightweight optimization effect of the digital twin three-dimensional model.

[0017] As a preferred technical solution of the present invention, the specific method of extracting the three-dimensional model matching dependent vertices in step 102 is:

[0018] Step 10201, fully automatically dividing the three-dimensional model into component vertex sets;

[0019] Step 10202: Use the OBB bounding box to perform component intersection detection, extract component pairs that intersect in space, and count them into set C;

[0020] Step 10203, randomly select a pair C(C1, C2) in the set C, and if any edge e of the C1 component intersects with the OBB bounding box of C2, count it into the set E; if any triangle facet t of the C2 component intersects with the bounding box of C1, count it into the set T;

[0021] Step 10204: Then, each edge e in the set E is checked for intersection with each face t in the set T. The intersection determination criteria are: there is an intersection point between the two and the intersection point is within the triangle t.

[0022] Step 10205: If e and t intersect, the two vertices on the edge are mating-dependent vertices. Calculate the acute angle θ between e and t, and record the vertex coordinates and θ in set D.

[0023] As a preferred technical solution of the present invention, the specific method of behavior prediction in step S3 is:

[0024] Step 301: The motion trajectory data set of the digital twin of the integrated mining and support equipment is: ;

[0025] Among them, Id is the twin equipment identifier, STr represents the trajectory of a single twin equipment, , , J is the sensor identifier on the twin equipment, and the data point Tr represents the trajectory sequence of the sensor detection part. i From the position coordinate X i and posture data Y i composition;

[0026] Step 302: Establish an LSTM network for each twin equipment trajectory prediction task of ID, connect the LSTM networks in sequence, and use the same structure for the trajectory prediction model of multiple J detection points under a single twin equipment;

[0027] Step 303: Use mean absolute error (MAE) to predict trajectories to describe the loss associated with each twin device trajectory: denoted as , ,

[0028] in, Given a trajectory for a single twin device, the observed pose sequence Tr and the predicted pose sequence The loss between them, n is the trajectory length, and |J| is the number of sensors.

[0029] As a preferred technical solution of the present invention, the specific method for driving and trajectory correction of the integrated twin equipment for mining and support in step S4 is:

[0030] Step 401: Define the twin platform time t s toe The predicted pose sequence between fore [h], H is the sampling period of the prediction data, the sensor sampling period is K, and the sequence P is obtained by equidistant sampling. fork ;

[0031] Step 402: define a certain sensor sampling time k, where t s <k<t e , the predicted pose is P fork [k], the real pose data is P real [k], the correction threshold is ε, and the virtual pose should be:

[0032] when hour: ;

[0033] when hour: ,in ;

[0034] In this way, we get a virtual running trajectory P with a period equal to the real data sampling period. vir ;

[0035] Step 403: Get prediction data P fore [h m ], then we get P vir [k-1], P fore [h m ] and P vir [k] Three known points, processing the virtual running trajectory sequence data, we can get a smooth driving trajectory sequence P div [f];

[0036] Step 404: Use trajectory sequence P div [f] Perform motion driving of the first model.

[0037] As a preferred technical solution of the present invention, the specific method for building a digital twin platform for integrated excavation and support equipment between faces in step S5 is:

[0038] Step 501: import the three-dimensional model of mining and support equipment into the virtual space as an object according to the real ratio of 1:1, and then establish a parent-child hierarchical relationship of multiple objects; each object has its own driving script and corresponding variable copy, and the variables are modified independently to realize the motion control of multiple objects with different functions;

[0039] Step 502: Based on the state synchronization concept, a state data interactor is established on the data server. The interactor only transmits key motion parameters, and the calculation and storage of other non-key parameters are performed by the client.

[0040] Step 503: Design a state data interactor using a multi-threaded producer-consumer model.

[0041] As a preferred technical solution of the present invention, the specific method of associating steps S1-S4 in step S5 is:

[0042] Step 50101, three-dimensionally model the mining support robot in accordance with the real-virtual ratio of 1:1, process and compile it into .FBX format in step S1, import it into the engine, process its parent-child relationship, and form a first model;

[0043] Step 50102: Write a driving script based on C# according to the third model in step S2, so that the variable copy of each motion component corresponds one-to-one with the parameters in the third model;

[0044] Step 50103: compile the models in step S3 and step S4 into C# components and mount them on the digital twin platform.

[0045] As a preferred technical solution of the present invention, the inter-face coal pillar mining and support digital twin platform is based on the Unity3D engine, and the database is a MySQL database.

[0046] In a second aspect, the present invention further provides a digital twin efficient virtual-real synchronization system for integrated equipment of inter-face coal pillar excavation and support, which is used to implement the above-mentioned virtual-real synchronization method, and the system includes the following units:

[0047] A model optimization unit, used for lightweight optimization of the topology of a first model, where the first model is a three-dimensional model of a digital twin of the equipment;

[0048] The motion behavior analysis unit is used to construct a motion behavior model of the equipment digital twin, which is recorded as the second model; the second model is analyzed to obtain a motion behavior parameter mapping matrix, and a parameter-driven digital twin driving model of the motion behavior is established, which is recorded as the third model;

[0049] The trajectory prediction unit is used to establish a trajectory prediction model for the equipment digital twin: data fusion and drive data division are performed on the sensor system data to predict the trajectory of the first model and obtain the drive parameters of the third model;

[0050] The driving and trajectory correction unit is used to establish a driving and trajectory correction model for virtual space equipment: the posture data collected by the sensor system is processed by the prediction model in step S3, and then corrected by the algorithm to obtain smooth driving trajectory data, and the trajectory data is used to drive the first model;

[0051] The virtual-reality synchronization unit is used to associate the model optimization unit, motion behavior analysis unit, trajectory prediction unit, and drive and trajectory correction unit to build a digital twin platform, adopt multi-threaded design status data, and realize the virtual-reality synchronization of the digital twin of the integrated equipment for inter-face coal pillar excavation and support.

[0052] In a third aspect, the present invention further provides a computing device, characterized in that the device comprises a memory and a processor:

[0053] The memory is used to store computer programs;

[0054] The processor is used to implement the above-mentioned efficient virtual-real synchronization method of digital twin of integrated equipment for inter-face coal pillar mining and support when executing the computer program.

[0055] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0056] 1. The present invention innovatively combines the optimization of building a digital twin platform with the optimization of twin model driving. On the one hand, while maintaining the assembly relationship between moving parts, the QEM algorithm is improved to achieve lightweight three-dimensional twin models, solving the problem that the large size of the twin model leads to slow drive response and the need to rebuild the twin motion relationship. On the other hand, based on Self-Attention-LSTM, the motion behavior prediction and correction of integrated equipment for inter-face coal pillar mining and support is realized, a key motion parameter data interaction model based on multi-threading is established, and the state synchronization technology is used to fuse the optimized three-dimensional twin geometric model and behavior model; this method can effectively solve the problem of poor virtual-real synchronization of digital twin models.

[0057] 2. In the process of lightweighting the three-dimensional model of integrated equipment for mining and support of inter-face coal pillars, an algorithm based on improved QEM and maintaining the original assembly relationship of the three-dimensional model of integrated equipment for mining and support is proposed. First, the componentization preprocessing is performed: the parts of the manufacturing independent units are reorganized into a single component of the motion independent unit, and the assembly is processed into a multi-motion component model. Then, the breadth-first search algorithm is applied to fully automatically divide the component vertex set of the model. The vertices are distinguished and marked into two types: coordination-dependent vertices and other ordinary vertices. The coordinates of the coordination-dependent vertices are extracted through the intersection detection of the two components. Different error metrics are defined according to the vertex type, and the coordination relationship between components and the geometric characteristics of the components are maintained by calculating the edge collapse cost. This method can effectively save the time of reassembly of the three-dimensional model and greatly improve the fluency of the rendering of the twin equipment platform.

[0058] 3. The present invention proposes a trajectory prediction-correction method based on Self-Attention-LSTM to improve the efficiency of twin platform drive data processing and motion rendering. First, the long short-term memory network (LSTM) is used, which has advantages in learning the time-dependent characteristics of time series data. At the same time, a self-attention mechanism is introduced to evaluate the different degrees of dependency between different trajectory points to improve the accuracy of trajectory prediction. Finally, the difference between the predicted posture and the true posture is compared with the correction threshold, while taking into account the accuracy and timeliness of the twin equipment drive data. This method can effectively reduce the real posture data required for the point, while maintaining the continuity of the drive data, providing an accurate, stable and real-time data source for the drive of the twin equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a flowchart of the method of the present invention;

[0060] Figure 2 is a schematic diagram of an embodiment of the present invention;

[0061] Figure 3 A flowchart of lightweighting a three-dimensional model according to an embodiment of the present invention;

[0062] Figure 4 This is a flowchart of extracting dependent vertices according to an embodiment of the present invention;

[0063] Figure 5 It is a schematic diagram of twin driving and correction of an embodiment of the present invention;

[0064] Figure 6 Schematic diagram of a digital twin platform according to an embodiment of the present invention;

[0065] Figure 7 Schematic diagram of experimental conditions of an embodiment of the present invention;

[0066] Figure 8 (a) is a schematic diagram of the driving effect of an embodiment of the present invention (a);

[0067] Figure 8 (b) is a schematic diagram of the driving effect of an embodiment of the present invention (b). DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0069] First, in order to facilitate the understanding of the present application, the nouns involved in the embodiments of the present application are explained.

[0070] Twin cluster: It is a group of equipment in virtual space, corresponding to the robot cluster of integrated mining and support equipment in real space.

[0071] An equipment or robot cluster refers to a series of identical or different equipment or robots that appear simultaneously in a certain spatial area under unified command to achieve the same goal, generally with a number of more than a dozen.

[0072] Example 1

[0073] like Figure 1-2 As shown, an efficient virtual-real synchronization method for digital twin of integrated mining and support equipment for inter-face coal pillars of the present invention is applied to an example of building a digital twin platform for an integrated mining and support robot cluster, and includes the following steps:

[0074] Step 1: Establish a three-dimensional model of the digital twin of the equipment, recorded as the first model, and optimize the first model; use the improved QEM algorithm to perform topological optimization of the three-dimensional model of the integrated mining and support equipment.

[0075] In this implementation, a 3D model of the mining and support integrated robot cluster was established through Solidworks and assembled. Next, the 3D model was lightweight optimized. The lightweight optimization process is as follows: Figure 3 As shown:

[0076] Step 101, define cooperation-dependent vertices: when performing edge collapse processing, the deletion of vertices and the generation of related new vertices will destroy the mechanical cooperation relationship between components. Such vertices are called cooperation-dependent vertices;

[0077] Step 102, using the BFS algorithm to distinguish the vertex sets of each component of the three-dimensional model assembly according to whether the Mesh grid is continuous, and using the OBB bounding box and edge-face intersection detection to extract the three-dimensional model matching dependent vertices; Figure 4 As shown, the specific process is:

[0078] Step 10201: Apply a breadth first search (BFS) algorithm to fully automatically divide the component vertex set of the three-dimensional model of the mining and support integrated equipment.

[0079] Step 10202: Use the OBB bounding box to perform component intersection detection, quickly extract spatially intersecting component pairs and count them into set C.

[0080] Step 10203, randomly select a pair of C (C1, C2) in C. For any edge e of the C1 component, if it intersects with the OBB bounding box of C2, it is counted into the set E; for any triangle face t of the C2 component, if it intersects with the bounding box of C1, it is counted into the set T.

[0081] Step 10204: Then, each edge e in E is checked for intersection with each face t in T. The intersection criterion is: there is an intersection point between the two and the intersection point is within the triangle t.

[0082] Specifically, the intersection detection of the plane where the line segments e and t are located is performed, and formula 1 is the necessary and sufficient condition for the intersection;

[0083] Formula (1)

[0084] If formula 1 is satisfied, calculate the intersection point I and determine whether the intersection point is inside the triangle patch t. Formula 2 is the necessary and sufficient condition for the intersection point I to be inside the triangle patch t.

[0085] Formula (2)

[0086] Among them, n is the unit normal vector of triangle patch t, ​​V1, V2, V3 are the three vertices of triangle patch t, ​​V i 、V j are the two vertices of edge e, and I is the intersection of e and t;

[0087] If equations (1) and (2) are satisfied in sequence, the edge e intersects with the triangle t;

[0088] Step 10205: If e and t intersect, the two vertices on the edge are mating-dependent vertices. Calculate the acute angle θ between e and t, and record the vertex coordinates and the corresponding θ in set D.

[0089] Step 103: Based on the quadratic error metric topological compression algorithm, a coordinated dependent vertex collapse influence factor is proposed to improve the comprehensive influence factor of topological collapse, which is recorded as , control the collapse process of the triangular face edges of the 3D model;

[0090] In the above formula, is the collapsing cost factor of the dependent vertex according to θ;

[0091] θ is the acute angle between the edge of one component's triangular face and the triangular face of another component;

[0092] u represents the conservative degree of the collocation-dependent vertex collapse factor on edge collapse. In practical applications, this parameter can be adjusted to control the influence of the collocation-dependent vertex on topological compression;

[0093] is the vertex curvature influence factor: , where n i is the normal vector of the triangle surface;

[0094] is the angle between the unit normal vector of the triangle and the vertex normal vector; n miis the normal vector of a vertex of the triangle; It is the flatness influencing factor, which indicates the smoothness of the patch near the vertex;

[0095] are the vertices of the triangle patch; D is the set of matching dependent vertices;

[0096] Step 104: Use the bidirectional Hausdorff distance to evaluate the lightweight optimization effect of the digital twin three-dimensional model.

[0097] When implementing step 1, u is set to 0.7 and the simplification rate is 95%.

[0098] Step 2: construct a motion behavior model of the equipment digital twin, recorded as the second model; analyze the second model to obtain a motion behavior parameter mapping matrix, and establish a digital twin driving model of the parameter-driven motion behavior, recorded as the third model;

[0099] The second model can be further understood as: after analyzing the movement behavior of real equipment in the real physical space, the movement behavior pattern obtained needs to be assigned to the virtual space so that the first model has the same movement behavior. This movement behavior pattern, that is, the second model, exists in both spaces at the same time.

[0100] The third model can be further understood as: after the second model in the virtual space is established, the relationship between the driving and the driven of the parameters in the virtual space is clarified, and a model of parameter-driven motion behavior is established.

[0101] The specific process of step 2 is:

[0102] Step 201: Analyze and establish motion behavior parameter models of mining robots and temporary support robots, and complete the parametric expression of the posture of the mining and support integrated robot cluster.

[0103] Step 202: establish the corresponding relationship between the motion of each component of the mining robot model and the temporary support robot model and each physical sensor, and form a twin model motion mapping matrix, so that the motion effect of the physical space robot cluster and the twin space robot cluster is completely consistent.

[0104] Step 3: Establish a trajectory prediction model for the equipment digital twin: perform data fusion and drive data division on the sensor system data, predict the trajectory of the first model, and obtain the drive parameters of the third model;

[0105] The specific implementation process is as follows:

[0106] Step 301: The real sensor data of the robot cluster is processed, fused and divided into a real motion data set: , where Id is the single robot identifier, STr lRepresents the motion trajectory data of a single robot, , , J is the sensor identifier on the equipment, data point Tr l Represents the trajectory sequence of the sensor detection part, Trl i By position coordinates Xrl i And posture data Yrl i composition.

[0107] At the same time, the motion trajectory dataset of the twin cluster of the integrated mining and support robot is: , where Id is the identifier of a single twin robot, STr represents the trajectory of a single twin robot, , , J is the virtual sensor identifier on the twin robot, and the data point Tr represents the trajectory sequence of the virtual sensor detection part. i From the position coordinate X i and posture data Y i The real motion dataset is used as the driving data when the system is initialized and as the input of the correction model, and the twin dataset is used as the input of the prediction model and the correction model.

[0108] Step 302: Establish an LSTM network for each twin robot trajectory prediction task with ID, connect these LSTM networks sequentially, and introduce a self-attention mechanism; the multi-J detection point trajectory prediction model under a single twin robot adopts the same structure.

[0109] Step 303: Use mean absolute error (MAE) for trajectory prediction to describe the loss associated with the trajectory of each excavation robot twin equipment: , , Given a trajectory for a single twin robot, the observed pose sequence Tr and the predicted pose sequence The loss between , n is the trajectory length, and |J| is the number of sensors. Specifically, the Softmax function is used as the activation function for pose prediction; for the optimization algorithm that performs stochastic gradient descent to minimize the loss function, the root mean square propagation RMSprop method is used.

[0110] In this embodiment, |Id|=13, mining robot: 1*|J|=20, temporary support robot: 12*|J|=14; model training is based on Matlab.

[0111] Step 4: Establish a virtual space equipment driving and trajectory correction model: The posture data collected by the sensor system is processed by the trajectory prediction model in step S3, and then corrected by algorithm processing to obtain smooth driving trajectory data, and the trajectory data is used to drive the first model.

[0112] Under the concept of digital twin, the motion sensing system includes the physical equipment data acquisition system in the real space and the first model data acquisition system in the virtual space.

[0113] like Figure 5 The principle of posture prediction and trajectory correction of the integrated twin equipment for mining and support is shown in the figure. The specific method of driving and trajectory correction of the integrated twin equipment for mining and support is as follows:

[0114] Step 401: Set the twin platform time t s to e The predicted pose sequence between fore [h], H is the sampling period of the prediction data. Since the sensor sampling period K is different from the prediction sequence data sampling period H, equidistant sampling is performed to obtain the prediction sequence P fork .

[0115] Step 402: Assume a certain sensor sampling time k (t s <k<t e ), the predicted pose is P fork [k], the real pose data is P real [k], assuming the correction threshold is ε, the virtual pose should be:

[0116] when hour: ;

[0117] when hour: ,in ; In this way, we get a virtual running trajectory P with a period equal to the real data sampling period vir .

[0118] Step 403: Get prediction data P fore [h m ], then we get P vir [k-1], P fore [h m ] and P vir [k] Three known points, using the quadratic spline interpolation method to process the virtual running trajectory sequence data, a smooth driving trajectory sequence P can be obtained div [f].

[0119] Step 404: Use trajectory sequence P div [f] Perform motion driving of the first model.

[0120] As an exemplary explanation, the posture data of the sensing system includes the data collected by the physical equipment data collection system in the real space and the data collected by the first model data collection system in the virtual space, which are recorded as the first data; the first data is processed by the prediction model in step S3 to obtain the predicted trajectory data, which is recorded as the second data. This data does not directly act on the drive of the first model. The second data needs to be processed with the real data in the first data by an algorithm such as quadratic spline interpolation to obtain the data used to drive the first model, that is, the driving trajectory data, which is recorded as the third data. Since the third data is the operating result of the first model, the third data is downsampled in the application as the source of the posture data of the virtual space sensing system in the first data.

[0121] In this embodiment, the sensing sampling periods K are different. The real motion data period K is 0.2s, and the prediction sequence sampling period H is (1 / 60)s.

[0122] Step 5. Create a three-dimensional model of the mining and support robot in a real-to-virtual ratio of 1:1. After associating steps S1-S4, build a digital twin platform, use multi-threaded design status data, and realize the virtual-real synchronization of the digital twin of the integrated equipment for mining and support of inter-face coal pillars.

[0123] The specific method of associating steps S1-S4 in step S5 is:

[0124] Step 50101, three-dimensionally model the mining support robot in accordance with the real-virtual ratio of 1:1, process and compile it into .FBX format in step S1, import it into the engine, process its parent-child relationship, and form a first model;

[0125] Step 50102: Write a driving script based on C# according to the third model in step S2, so that the variable copy of each motion component corresponds one-to-one with the parameters in the third model;

[0126] Step 50103: compile the models in step S3 and step S4 into C# components and mount them on the digital twin platform.

[0127] The specific method for building the digital twin platform of integrated equipment for face mining and support in step S5 is as follows:

[0128] Step 501: import the three-dimensional model of mining and support equipment into the virtual space as an object according to the real ratio of 1:1, and then establish a parent-child hierarchical relationship of multiple objects; each object has its own driving script and corresponding variable copy, and the variables are modified independently to realize the motion control of multiple objects with different functions;

[0129] Step 502: Based on the state synchronization concept, a state data interactor is established on the data server. The interactor only transmits key motion parameters, and the calculation and storage of other non-key parameters are performed by the client.

[0130] Step 503: Design a state data interactor using a multi-threaded producer-consumer model.

[0131] In this embodiment, the digital twin platform in step 5 is based on the Unity3D engine, the driver script is written in C#, and the driver variables and state interactor databases are all MySQL databases. The database and the Unity3D platform use .dll dynamic libraries for reading and writing. Figure 6 shown.

[0132] Example 2

[0133] In this embodiment, the experimental hardware conditions are a laboratory cutting robot test system and a travel sensor for posture monitoring, the model of which is a pull-rope type XS-500, with a range of 0~500mm and an accuracy of 0.15%FS. Figure 7 shown.

[0134] In this embodiment, the data processor is a computer, the GPU is NVIDIA GeForce RTX 3080, the CPU is AMD R5 5600@3.5Hz, and the development tools and languages ​​are Unity3D, MySQL, Matlab, C# language and SQL language.

[0135] In this embodiment, in order to verify the feasibility of the method in this paper, the virtual-real consistency, operation stability and driving real-time performance of the movement of the cutting arm in the virtual space during the cutting operation are evaluated based on laboratory conditions. According to the actual cutting process, the simulated cutting process in the laboratory is designed as follows: the cutting drum moves vertically downward to simulate coal and rock cutting. After the cutting is completed, the cutting arm retreats and returns to the starting position for the next cutting simulation. The single operation cycle is designed to be 14 minutes, the sensor sampling cycle is 0.2s, and the cylinder data of 5 operation cycles are collected.

[0136] In the experiment of this embodiment, the background gateway is set up to simulate the impact of network delay, jitter and packet loss on virtual-reality synchronization control in real scenarios.

[0137] contrast Figure 8(a) and (b) show that the main reasons for control delay are network delay and rendering delay. When the network fluctuation and packet loss rate increase significantly, the processing delay of Unity increases to 22.78ms, which is 18.96% higher than that of a good network. This is because Unity needs more computing resources for posture prediction and correction. The average total delay of virtual-real synchronization control increases from 50.21ms to 59.88ms, which is 19.26% higher than that of a good network, while the standard deviation only increases by 0.15ms, an increase of 8.88%, indicating that the proposed method can ensure the stability of virtual-real synchronization control. The maximum virtual-real synchronization control delay increases to 66.32ms, and the total delay increases by 16% on average compared to a good network, but it is still kept within 100ms, indicating that the proposed method can ensure the real-time performance of virtual-real synchronization control.

[0138] The present invention also provides a digital twin efficient virtual-real synchronization system for integrated equipment of inter-face coal pillar excavation and support, the system comprising the following modules:

[0139] A model optimization unit, used for lightweight optimization of a first model, where the first model is a three-dimensional model of a digital twin of the equipment;

[0140] The motion behavior analysis unit is used to construct a motion behavior model of the equipment digital twin, which is recorded as the second model; the second model is analyzed to obtain a motion behavior parameter mapping matrix, and a parameter-driven digital twin driving model of the motion behavior is established, which is recorded as the third model;

[0141] The trajectory prediction unit is used to establish a trajectory prediction model for the equipment digital twin: data fusion and drive data division are performed on the sensor system data to predict the trajectory of the first model and obtain the drive parameters of the third model;

[0142] The driving and trajectory correction unit is used to establish a driving and trajectory correction model for virtual space equipment: the posture data collected by the sensor system is processed by the prediction model in step S3, and then corrected by the algorithm to obtain smooth driving trajectory data, and the trajectory data is used to drive the first model;

[0143] The virtual-reality synchronization unit is used to associate the model optimization unit, motion behavior analysis unit, trajectory prediction unit, and drive and trajectory correction unit to build a digital twin platform, adopt multi-threaded design status data, and realize the virtual-reality synchronization of the digital twin of the integrated equipment for inter-face coal pillar excavation and support.

[0144] The present invention also provides a computing device, which includes a memory and a processor:

[0145] Memory for storing computer programs;

[0146] A processor is used to implement the above-mentioned efficient virtual-real synchronization method of the digital twin of the integrated equipment for inter-face coal pillar mining and support when executing a computer program.

[0147] To sum up, the method of the present invention has simple steps, reasonable design and good virtual-reality synchronous driving effect. Under the condition of limited data transmission, the relative error of virtual-reality synchronization does not exceed 2.06%, the standard deviation of the total delay only increases by 1.84ms, and the maximum delay is 66.32ms, which effectively suppresses the increase and fluctuation of the delay of the digital twin platform and realizes accurate and efficient data interaction between virtual models and physical entities.

[0148] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0149] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0150] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The above embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

[0152] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An efficient virtual-real synchronization method for digital twins of integrated equipment for inter-face coal pillar excavation and support, characterized by: The method comprises the following steps: S1: Establish a three-dimensional model of the equipment digital twin, recorded as a first model, and lightweight the first model; S2: construct a motion behavior model of the equipment digital twin, recorded as the second model; analyze the second model to obtain a motion behavior parameter mapping matrix, and establish a digital twin driving model of the parameter-driven motion behavior, recorded as the third model; S3: Establishing the trajectory prediction model of the equipment digital twin: performing data fusion and drive data division on the sensor system data, predicting the trajectory of the first model, and obtaining the drive parameters of the third model; S4: Establishing a virtual space equipment driving and trajectory correction model: The posture data collected by the sensor system is processed by the prediction model in step S3, and then corrected by algorithm processing to obtain smooth driving trajectory data, and the trajectory data is used to drive the first model; S5: 3D model the mining and support robot in a real-to-virtual ratio of 1:1, and after associating steps S1-S4, build a digital twin platform, use multi-threaded design state data, and realize virtual-real synchronization of digital twins of inter-face coal pillar mining and support integrated equipment; In step S1, the first model is lightweighted, and the specific method is as follows: Step 101, define cooperation-dependent vertices: when performing edge collapse processing, the vertices whose deletion and generation of related new vertices will destroy the mechanical cooperation relationship between components are called cooperation-dependent vertices; Step 102, using the BFS algorithm to distinguish the vertex sets of each component of the three-dimensional model assembly according to whether the Mesh grid is continuous, and using the OBB bounding box and edge-face intersection detection to extract the three-dimensional model matching dependent vertices; Step 103: Propose a collapsing factor of the dependent vertices based on the topological compression algorithm, improve the comprehensive impact factor of topological collapse, and control the collapse process of the edges of the triangle facets of the three-dimensional model; Step 104, evaluating the lightweight effect of the first model; The specific method of extracting the three-dimensional model coordination dependent vertices by using the OBB bounding box and edge-face intersection detection in step 102 is: Step 10201, fully automatically dividing the three-dimensional model into component vertex sets; Step 10202: Use the OBB bounding box to perform component intersection detection, extract component pairs that intersect in space, and count them into set C; Step 10203, randomly select a pair C (C1, C2) in the set C, and if any edge e of the C1 component intersects with the OBB bounding box of C2, count it into the set E; if any triangle face t of the C2 component intersects with the bounding box of C1, count it into the set T; Step 10204: Then, each edge e in the set E is checked for intersection with each face t in the set T. The intersection determination criteria are: there is an intersection point between the two and the intersection point is within the triangle t. Step 10205: If e and t intersect, the two vertices on the edge are matching-dependent vertices. Calculate the acute angle θ between e and t, and record the matching-dependent vertex coordinates and θ in set D.

2. According to claim 1, a digital twin efficient virtual-real synchronization method for integrated equipment for inter-face coal pillar excavation and support, characterized in that: The digital twin platform is based on the Unity3D engine and the database is a MySQL database.

3. A digital twin efficient virtual-real synchronization system for integrated equipment of inter-face coal pillar excavation and support, used to implement the virtual-real synchronization method as described in any one of claims 1-2, characterized in that: The system comprises the following units: A model optimization unit, used for lightweight optimization of a first model, where the first model is a three-dimensional model of a digital twin of the equipment; The motion behavior analysis unit is used to construct a motion behavior model of the equipment digital twin, which is recorded as the second model; the second model is analyzed to obtain a motion behavior parameter mapping matrix, and a parameter-driven digital twin driving model of the motion behavior is established, which is recorded as the third model; The trajectory prediction unit is used to establish a trajectory prediction model for the equipment digital twin: data fusion and drive data division are performed on the sensor system data to predict the trajectory of the first model and obtain the drive parameters of the third model; The driving and trajectory correction unit is used to establish a driving and trajectory correction model for virtual space equipment: the posture data collected by the sensor system is processed by the prediction model in step S3, and then corrected by the algorithm to obtain smooth driving trajectory data, and the trajectory data is used to drive the first model; The virtual-reality synchronization unit is used to associate the model optimization unit, motion behavior analysis unit, trajectory prediction unit, and drive and trajectory correction unit to build a digital twin platform, adopt multi-threaded design status data, and realize the virtual-reality synchronization of the digital twin of the integrated equipment for inter-face coal pillar excavation and support.

4. A computing device, characterized in that: The device comprises a memory and a processor: The memory is used to store computer programs; The processor is used to implement the efficient virtual-real synchronization method of digital twin of inter-face coal pillar mining and support integrated equipment as described in any one of claims 1-2 when executing the computer program.

Images