Simulation mine digital twinborn teaching system and method based on virtual reality technology

Abstract

The invention relates to the technical field of virtual reality, in particular to an analog mine digital twinborn teaching system and method based on the virtual reality technology. The method comprises the following steps: generating a virtual mine space on a cloud server; acquiring a virtual model of the coal mining equipment as a digital twinborn body of the coal mining equipment; acquiring real-time model parameters of the coal mining equipment physical model operated by the trainee; according to the real-time model parameters, carrying out online updating processing on a digital twinborn body of the coal mining equipment to obtain an updated digital twinborn body; the cloud server maps a virtual three-dimensional model of the corresponding coal mining equipment in the virtual mine space according to the updated digital twin; and a machine learning algorithm evaluates the coal mining operation of the trainee according to the real-time model parameters. According to the invention, the real experience of trainees during training can be enhanced.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a simulated mine digital twin teaching system and method based on virtual reality technology. Background Art

[0002] Mining engineering is a basic major for national energy security, and the practical teaching of the mining major is crucial for cultivating students' professional abilities. At present, the practical teaching of the mining major mostly adopts the method of allowing students to operate the physical models of coal mining equipment. However, the traditional physical model teaching has the following problems: the physical model can only perform simple actions and cannot fully simulate the real mine environment; the physical model has limited space and cannot meet the diverse practical needs of students; the scenes that students come into contact with are limited, which is quite different from the real mine site, and the practical experience is not real. In addition, due to the complexity of mining operations, it is difficult to evaluate the operations of students timely and accurately with the current teaching methods. Summary of the Invention

[0003] In view of this, the embodiments of the present invention provide a simulated mine digital twin teaching system and method based on virtual reality technology to solve the technical problem that the current teaching method for coal mining is limited by the space scene, resulting in a poor sense of reality in the scene experience of students.

[0004] The technical solution adopted by the present invention is as follows:

[0005] In the first aspect, the present invention provides a multi-scene collaborative simulated mine teaching method based on digital twin, and the method includes:

[0006] S1: Generate a virtual mine space containing a large dip coal seam roadway model according to the layout form of the development roadways in the large dip coal seam mine of actual production;

[0007] S2: Obtain the virtual model of the working face equipment as the digital twin of the working face equipment;

[0008] S3: Obtain the real-time model parameters of the physical models of the working face equipment in multiple different scenes;

[0009] S4: Perform online update processing on the digital twin of the working face equipment according to the real-time model parameters to obtain an updated digital twin;

[0010] S5: Map the corresponding virtual three-dimensional model of the working face equipment in the virtual mine space according to the updated digital twin;

[0011] S6: Evaluate the operation of the working face equipment by using a machine learning algorithm according to the real-time model parameters.

[0012] In a second aspect, the present invention further provides a multi-scenario collaborative simulation mine teaching system based on digital twins, which applies the teaching method described in the first aspect. The system includes a physical model of working face equipment, a cloud server, and a digital twin update module. Sensors are installed on the physical model of the working face equipment. The digital twin update module is used to update the digital twin according to the data collected by the sensors. The digital twin update module is communicatively connected to the cloud server.

[0013] Beneficial effects: The simulation mine digital twin teaching system and method based on virtual reality technology of the present invention can accurately simulate the roadway layout, equipment distribution, and complex geological conditions of an actual mine through the virtual mine space generated by the cloud server, providing an immersive and high-fidelity training environment for students, so that the training of students is no longer restricted by the venue. By means of driving the update of the twin by physical model parameters, the virtual models of coal mining equipment in the present invention, such as hydraulic supports, coal shearers, scraper conveyors, etc., are synchronized with the physical models operated by students, ensuring that the virtual scenario is dynamically consistent with the actual operation, and enhancing the authenticity and intuitiveness of teaching. By collecting the real-time model parameters of the physical model through sensors and updating the digital twin online, seamless linkage between virtual and reality is achieved. The present invention utilizes the real-time model parameters of the physical model and automatically evaluates the operations of students through machine learning algorithms, and can accurately and timely feedback on the training situation of students' coal mining operations, which is beneficial for students to correct their own deficiencies in a timely manner at the training site, thereby significantly improving the teaching effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to these drawings, and all of these are within the protection scope of the present invention.

[0015] Figure 1 It is a schematic flow chart of the simulation mine digital twin teaching method based on virtual reality technology of the present invention;

[0016] Figure 2 It is a schematic layout structure diagram of the virtual mine space in the present invention;

[0017] Figure 3 It is a schematic diagram of the installation position of sensors on the hydraulic support in the present invention;

[0018] Figure 4 It is a schematic diagram of mapping a virtual three-dimensional model in the virtual mine space by using the digital twin in the present invention;

[0019] Figure 5 It is a schematic flow chart of the method for evaluating students' coal mining operations in the present invention;

[0020] Figure 6 It is a schematic flowchart of the method for training the support vector machine algorithm model of the present invention;

[0021] Figure 7 It is a schematic flowchart of the method for evaluating the coal mining operation of trainees by using the machine learning algorithm of the present invention;

[0022] Figure 8 It is a schematic flowchart of the method for reversely updating the physical model of the present invention;

[0023] Figure 9 It is a structural block diagram of the simulated mine digital twin teaching system based on virtual reality technology of the present invention.

[0024] Parts and their numbers in the figure:

[0025] Slope guard angle sensor 11, side guard displacement sensor 12, roof beam attitude sensor 13, infrared sensor 14, shield beam attitude sensor 15, prop displacement sensor 16, base attitude sensor 17, jack displacement sensor 18 for pushing. Specific implementation manner

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. It should be noted that in this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. This is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Moreover, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article, or device. Without further limitation, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article, or device including the said elements. If there is no conflict, the embodiments of the present invention and the various features in the embodiments can be combined with each other, and all are within the protection scope of the present invention.

[0028] Embodiment 1

[0029] As Figure 1 shown, the present invention provides a simulation mine digital twin teaching method based on virtual reality technology. This method mainly includes:

[0030] S1: Generate a virtual mine space on the cloud server;

[0031] The virtual mine space can be obtained after three-dimensional modeling according to the actual mine space. It can also be obtained by making some modifications to the three-dimensional model produced according to the actual mine space as needed. Figure 2 shows the layout of the created virtual mine space. Specifically, during implementation, virtual mine spaces for scenarios meeting different teaching requirements can be pre-generated. Then, the user can select the virtual mine space model to be used for this training according to the training requirements.

[0032] S2: Obtain the virtual model of the coal mining equipment as the digital twin of the coal mining equipment;

[0033] Various coal mining equipment used in coal mining operations can be three-dimensionally modeled in advance to obtain a corresponding virtual model of the coal mining equipment. This virtual model can serve as the digital twin of the coal mining equipment.

[0034] S3: Obtain the real-time model parameters of the physical model of the coal mining equipment operated by the trainee;

[0035] The aforementioned physical model of the coal mining equipment is an actual teaching and training equipment made by imitating the coal mining equipment, and has basically the same structure and function as the actual coal mining equipment operating in a real mine. Trainees can operate the physical model of the coal mining equipment in a safe training venue. When the trainee operates the physical model of the coal mining equipment, the state of the physical model of the coal mining equipment, such as the posture of the model, the relative position relationship and relative posture between various components of the coal mining equipment, etc., will change. The aforementioned parameters reflecting the state of the physical model of the coal mining equipment, the relative position relationship between various components of each coal mining equipment, and the relative posture between various components are the parameters of the aforementioned physical model. In this step, the aforementioned model parameters are collected in real time while the trainee is operating, so as to obtain the model parameters of the physical model of the coal mining equipment.

[0036] S4: Perform online update processing on the digital twin of the mining equipment according to the real-time model parameters to obtain an updated digital twin;

[0037] In this step, the collected real-time model parameters are used to update the digital twin, so that the model parameters of the digital twin are consistent with those of the physical model, so that the state of the digital twin is consistent with the state of the physical model of the coal mining equipment. In this way, the operation of the physical model of the coal mining equipment by the trainee in the training venue can be timely reflected on the digital twin.

[0038] S5: The cloud server maps the corresponding virtual three-dimensional model of the coal mining equipment in the virtual mine space according to the updated digital twin;

[0039] As Figure 4 shown, the initial virtual three-dimensional model of the coal mining equipment can be imported into the virtual mine space before the start of teaching and training. Then, after the digital twin is updated, the virtual three-dimensional model of the coal mining equipment in the virtual mine space is updated in time with the updated digital twin, so that the virtual three-dimensional model of the coal mining equipment in the virtual mine is consistent with the physical model of the coal mining equipment operated by the trainee. In this way, the trainee can experience the effects generated after operating the coal mining equipment in the virtual mine, and the trainees training in the training venue can also have an experience similar to that of coal mining operations in the actual mine space.

[0040] S6: Evaluate the coal mining operation of the trainee according to the real-time model parameters.

[0041] On the one hand, in this embodiment, the digital twin is updated online in real time using real-time model parameters, and the virtual 3D model of the coal mining equipment in the virtual mine is updated in real time using the digital twin. On the other hand, the model parameters of the physical model of the coal mining equipment collected are used to evaluate the coal mining operations of the trainees, so that the trainees can obtain timely feedback results of the training while performing coal mining operation training in a realistic scenario. This is more conducive to the trainees discovering their own deficiencies during the training process and making targeted improvements, thereby significantly improving the training effect.

[0042] As Figure 5 shown, in this embodiment, step S6: evaluating the coal mining operations of the trainees according to the real-time model parameters includes:

[0043] S61: Obtain the real-time model parameters when the trainee operates the physical model of the coal mining equipment;

[0044] In this embodiment, the real-time model parameters obtained when operating the physical model of the coal mining equipment include but are not limited to the height of the hydraulic support column, the angle of the hydraulic support rib protection plate, the position of the hydraulic support, the angle of the shearer boom, the position of the shearer, the shearer traction speed, and the speed of the belt conveyor.

[0045] In order to accurately obtain the real-time model parameters when the trainee operates the physical model of the coal mining equipment, sensors can be installed in advance at the set positions of each physical model of the coal mining equipment, and these sensors are used to collect data reflecting the model parameters. For example, for the hydraulic support, displacement sensors and pressure sensors can be installed on the columns, angle sensors can be installed at the hinge of the rib protection plate, displacement sensors can be installed on the front canopy beam, and displacement sensors can be installed on the jacks; for the shearer, angle sensors can be installed at the boom, and speed sensors can be installed on the drum. Position sensors can also be installed on the scraper conveyor.

[0046] As shown in Figure 3, Figure 3 the positions of the various sensors installed on the hydraulic support are shown, such as the rib protection plate angle sensor 11 installed on the rib protection plate, the side protection plate displacement sensor 12 installed on the side protection plate, the roof beam attitude sensor 13 installed on the roof beam, the infrared sensor 14 installed under the roof beam, the shield beam attitude sensor 15 installed on the shield beam, the support displacement sensor 16 installed on the support, the base attitude sensor 17 installed on the base, and the push jack displacement sensor 18 installed on the push jack.

[0047] S62: Obtain the support vector machine machine learning algorithm model trained with data;

[0048] In this step, the machine learning algorithm model is first trained using the data collected by the sensors, and the parameters in the machine learning algorithm model are optimized through data training so that it can accurately classify the data collected by the sensors. In this embodiment, the support vector machine is used as the machine learning algorithm.

[0049] S63: The machine learning algorithm model outputs an evaluation result of the trainee's coal mining operation according to the input model parameters.

[0050] After obtaining the data reflecting the model parameters detected by the sensors, these data are input into the trained machine learning algorithm model, and the machine learning algorithm outputs an evaluation result of the trainee's coal mining operation according to the input data. The model parameters can indirectly reflect the operation of the trainee on the coal mining equipment. However, the evaluation of whether the trainee's operation meets the standard operation requirements is affected by multiple model parameters, and different model parameters also affect each other. Therefore, there is a relatively complex non-linear relationship between the model parameters and the evaluation results. In this embodiment, the support vector machine algorithm in the machine learning algorithm is used to process the data collected by the sensors to obtain the evaluation result of the trainee's coal mining operation. The evaluation results include but are not limited to: the evaluation of the hydraulic support: whether the support is moved in place, whether the rib protection plate is opened, whether the support resistance reaches the set value, whether the parallelism of adjacent supports meets the requirements, etc.

[0051] The evaluation of the shearer: whether the traction speed and position match the actions of the support, whether the drum height is correct, whether the sequence of the shearer's oblique cutting operation is correct, etc.

[0052] The evaluation of the scraper conveyor: whether the handling after the occurrence of coal accumulation and material scattering is correct, whether the straightness of the scraper conveyor meets the requirements, whether the operation at the end of the scraper conveyor is correct, etc. The evaluation of the end support includes: whether the installation and evacuation operations of the end support are correct, whether the advanced support matches the mining, and whether the length of the advanced support meets the requirements.

[0053] As Figure 6 shown, in this embodiment, the S62: obtaining the support vector machine machine learning algorithm model trained by data includes the following steps:

[0054] S621: Obtain the data of multiple sensors for detecting model parameters when the trainee operates the coal mining equipment, and group the sensor data;

[0055] In this embodiment, sensors installed at set positions of each coal mining device are used to detect and collect model parameters. Since the evaluation results are affected by multiple model parameters in multiple coal mining devices, in this step, the sensor data can be grouped according to the evaluation content, and the sensor data that affects the evaluation content is grouped together. In this way, the data of multiple sensors can be fully utilized, and irrelevant data can be effectively filtered out, thereby improving the evaluation accuracy and increasing the timeliness of the evaluation. For example, the data reflecting the model parameters of the shearer and the support collected within a certain period of time can be grouped as evaluation data for whether the shearer traction speed and position match the support movement.

[0056] S622: Obtain the machine learning algorithm models corresponding to each evaluation item;

[0057] Corresponding machine learning algorithm models can be trained in advance for each evaluation content. For example, for the result of whether the shearer traction speed and position match the support movement, a corresponding machine learning algorithm model can be selected for training, and the parameters of the algorithm model can be optimized. The trained algorithm model is used as the machine learning algorithm model for evaluating items such as whether the shearer traction speed and position match the support movement.

[0058] S623: Obtain the respective evaluation results corresponding to each group of sensor data;

[0059] When this step is specifically implemented, the operator can operate the coal mining device, and then obtain the data of the relevant parameters of the coal mining device detected by the sensor during the operation, and manually determine whether the operation meets the corresponding requirements according to the operation specification requirements. This result is used as the evaluation result of the corresponding sensor data.

[0060] S624: Label each group of sensor data according to the respective evaluation results;

[0061] For example, if the evaluation result corresponding to a certain group of sensor data is that the flatness of the scraper conveyor meets the requirements, a label indicating that the flatness of the scraper conveyor meets the requirements can be added to this group of sensors.

[0062] S625: Train the machine learning algorithm models corresponding to each evaluation result with the labeled sensor data to obtain a support vector machine machine learning algorithm model trained with data.

[0063] In this step, the machine learning algorithm is trained with the labeled sensor data, and the parameters of the model are continuously optimized until the evaluation effect of the machine learning algorithm model meets the requirements.

[0064] Such as Figure 7As shown, in this embodiment, the step S63: the machine learning algorithm model outputs the evaluation result of the trainee's coal mining operation according to the input model parameters, including:

[0065] S631: Obtain a set of sensor data of the coal mining equipment as the target sensor data;

[0066] For example, if it is necessary to evaluate the training operation effect of whether the support is moved in place, and obtain the result of whether the support is moved in place, the set of sensor data reflecting whether the support is moved in place can be used as the target sensor data.

[0067] S632: Obtain a machine learning algorithm corresponding to an evaluation result as the target machine learning algorithm;

[0068] For example, if the evaluation result is whether the support is moved in place, a machine learning algorithm model trained for the evaluation result of whether the support is moved in place can be obtained as the target machine learning algorithm model for classifying the target sensor data later.

[0069] S633: Obtain the data category labels corresponding to each target sensor data;

[0070] Among them, the data category labels include the positive class (represented by the numerical value +1) and the negative class (represented by the numerical value -1).

[0071] The method for obtaining the data category label corresponding to the sensor data can adopt the threshold method, that is, set the threshold according to the safe operation specification of the equipment, and directly judge the category through the sensor data. For example, if the data of the pressure sensor on the hydraulic support column exceeds the rated range, it is marked as the negative class, and if it does not exceed the rated range, it is marked as the positive class. In addition, the sensor data can also be input into the deep learning algorithm model, and the deep learning algorithm model automatically outputs the data category label corresponding to the sensor data.

[0072] S634: Input the target sensor data and the corresponding data category into the target machine learning algorithm model;

[0073] S635: The target machine learning algorithm model outputs the corresponding evaluation result according to the input target sensor data and data category label. Specifically, it includes the following steps:

[0074] Obtain the support vector corresponding to the target sensor data;

[0075] Calculate the kernel function value of the support vector;

[0076] Multiply each kernel function value by the corresponding Lagrange multiplier and category label (if the category label is the positive class, multiply by +1, and if the category label is the negative class, multiply by -1), and then add up all the multiplied results;

[0077] The calculation result is obtained after adding a bias term to the added result.

[0078] The method of this embodiment further includes the following steps:

[0079] The step S63: The machine learning algorithm model outputs an evaluation result of the trainee's coal mining operation according to the input model parameters, and further includes:

[0080] S6301: Collect data on the training progress and training performance of the trainee's training items;

[0081] During the training process, training can be carried out according to the training items. During the trainee's training process, the training progress and training performance are given for these training items, and the system collects data reflecting the training progress and training performance.

[0082] Among them, the training progress can be represented by the ratio of the number of training steps of the completed training items to the total number of steps of the training items. The training performance can be evaluated by a machine learning algorithm. It can also be scored according to the operation accuracy rate and response time. Among them, the operation accuracy rate is the ratio of the number of correct operations to the total number of operations, and different ratios correspond to different scores. The response time is the average time consumed from the issuance of the instruction to the completion of the operation, and different time consumptions correspond to different scores.

[0083] S6302: Determine the optimal classification boundary parameters corresponding to the training progress and training performance according to the classification boundary optimization index of the support vector machine algorithm;

[0084] Among them, the classification boundary optimization index can be maximizing the margin, minimizing the classification error rate, or adding the two according to a certain weight. In specific implementation, a multi-objective optimization algorithm can be used to find the optimal classification boundary parameters that meet the optimization index. The classification boundary parameters are the parameters for determining the support vector machine classification boundary, such as the scale parameter and the bias term.

[0085] S6303: Perform regression analysis on the data of the training progress and training performance and the optimal classification boundary parameters to obtain the mapping relationship between the data of the training progress and training performance and the optimal classification boundary parameters;

[0086] For the support vector machine algorithm using the Sigmoid kernel function, the kernel function calculation formula is

[0087]

[0088] Among them, γ and r can be used as classification boundary parameters.

[0089] S6304: Obtain the current trainee's training progress and training performance when the change rate of the training progress and training performance reaches a preset value;

[0090] The conditions for updating the classification boundary can be set in advance. In this embodiment, the preset change rates reached by the training progress and training performance are used as the conditions for updating the classification boundary.

[0091] S6305: Obtain the optimal classification boundary parameters according to the current training progress and training performance of the trainee and the mapping relationship;

[0092] However, after the preset conditions are met, the optimal classification boundary parameters are obtained according to the corresponding relationship obtained from the previous regression analysis.

[0093] S6306: Determine whether the difference between the current classification boundary parameters and the optimal classification boundary parameters exceeds a preset difference;

[0094] If the difference in the classification boundary before and after the update is too large, it will cause a sudden change in the evaluation result. Therefore, this step uses the preset difference to control the change of the classification boundary.

[0095] S6307: If it exceeds, perform smoothing processing on the optimal classification boundary parameters according to the difference between the current classification boundary parameters and the optimal classification boundary parameters to obtain a set of classification boundary parameters;

[0096] In this step, several gradually changing classification boundary parameters can be found between the current classification boundary parameters and the optimal classification boundary parameters to form a set of classification boundary parameters, and an order is assigned to each classification boundary parameter so that the difference between adjacent classification boundary parameters is less than the preset value.

[0097] S6308: Update the classification boundary of the support vector machine algorithm model according to each classification boundary parameter in the set of classification boundary parameters in turn;

[0098] Update the parameters of the classification boundary in turn according to the order assigned in the previous step, so that the classification boundary is updated in turn.

[0099] S6309: If it does not exceed, update the classification boundary of the support vector machine algorithm model using the optimal classification boundary parameters.

[0100] After the classification boundary is updated, the system uses the updated support vector machine algorithm to evaluate the operations of the trainees.

[0101] The foregoing method of dynamically updating the classification boundary according to the progress and performance can make the evaluation method adapt to the training situation of the trainees and increase the training difficulty scientifically.

[0102] As Figure 8 shown, the method further includes:

[0103] S71: When the evaluation result does not meet the preset requirements, send an operation error prompt message and apply for reverse update permission. For example, when the evaluation result shows that the advanced support length is insufficient to meet the preset requirements, corresponding error prompt messages can be sent to indicate that the current operation is incorrect and the operation has caused the advanced support length to be insufficient to meet the preset requirements. At the same time, the system can send a request for reverse update permission to the operator's terminal device (such as a VR headset) or the teacher's terminal device.

[0104] S72: When receiving the reverse update authorization message, obtain the preset correct state of the virtual 3D model of the coal mining equipment in the virtual mine.

[0105] After receiving the request for reverse update permission, the operator or teacher can decide whether to agree to the reverse update according to the situation, and the result can be sent to the system through the operator's terminal device or the teacher's terminal device. When the system receives the reverse update authorization message, obtain the preset correct state of the virtual 3D model of the coal mining equipment in the virtual mine. For example, make the advanced support length of the hydraulic support reach the correct length that meets the requirements.

[0106] S73: Obtain the operation error of the coal mining equipment according to the preset accurate state of the virtual 3D model of the coal mining equipment and the current state of the virtual 3D model.

[0107] In this step, compare the preset accurate state of the virtual 3D model that meets the requirements with the current state of the virtual 3D model (the incorrect state caused by incorrect operation), and obtain the error between the two as the operation error of the coal mining equipment.

[0108] S74: Update the digital twin to the correct state according to the operation error.

[0109] In this step, perform a reverse update on the digital twin according to the operation error, so that the digital twin is updated to a state consistent with the preset accurate state of the virtual 3D model. For example, update the digital twin of the hydraulic support to the correct length where the advanced support length reaches the requirement.

[0110] S75: Control the physical model of the coal mining equipment to be adjusted to the correct state according to the digital twin after the state update.

[0111] In this step, use the updated digital twin to perform a direction update on the physical model operated by the trainee, so that its state is consistent with the preset accurate state of the virtual 3D model. In this way, the trainee can timely know the accurate state of the coal mining equipment after correct operation, which is conducive to timely correcting the trainee's incorrect operation.

[0112] In this embodiment, the method further includes:

[0113] S71: When an environmental change in the virtual mine space triggers a device failure, inject the corresponding type of failure into the virtual mine space according to the environmental change;

[0114] Environmental changes in the mine space can indeed trigger equipment failures. The mine environment is complex and changeable, and geological conditions, climatic factors, equipment operating status, etc. can all have a direct impact on the equipment, thus leading to failures. For example, rock fractures, roof subsidence, and changes in coal seam hardness can cause a sudden increase in roof pressure, and a sudden increase in roof pressure may lead to insufficient support force of the hydraulic support, triggering support pressure loss. Roof subsidence or rock strata movement may cause jamming of the moving parts of the hydraulic support. If the coal seam thickness is uneven, in fault areas, and in parting layers, it is easy to cause changes in the coal seam thickness. A sudden increase in the coal seam thickness may cause the shearer drum to be overloaded, triggering jamming. Changes in coal seam hardness may cause the traction motor of the shearer to be overloaded. Rock strata creep and support failure can lead to roadway deformation, and roadway deformation may cause the scraper conveyor chain to deviate or jam. Changes in roadway width may cause the equipment to collide with the roadway wall. High temperature may cause the viscosity of the hydraulic oil to decrease and the pressure of the hydraulic system to be insufficient. Changes in coal seam hardness and adjustment of equipment operating speed may cause the equipment load to fluctuate violently. A sudden increase in equipment load may cause the motor to be overloaded. Excessive load on the scraper conveyor may cause the chain to break.

[0115] This step can introduce environmental change factors into the virtual mine space. As the training process progresses, the environment in the virtual space changes. When the environmental change meets the conditions for triggering a device failure, various failures generated under this condition can be injected into the virtual space.

[0116] S72: Determine the corresponding states of each virtual 3D model in the virtual mine space under the failure according to the injected failure type;

[0117] For different failure types, the coal mining equipment will have different states, that is, the positions, postures, and movement speeds of each component of the coal mining equipment will have corresponding states under failure conditions. For example, in the case of support pressure loss, roof subsidence or rock strata movement may cause jamming of the moving parts of the hydraulic support. In this case, the height of the hydraulic support decreases and the movement speed of the components slows down. When the coal seam thickness suddenly increases and causes the shearer to jam, the shearer drum is in a stopped rotating state.

[0118] S73: Update the virtual 3D model according to the corresponding state;

[0119] This step updates the virtual 3D model in the virtual mine space. For example, it reduces the height of the virtual 3D model of the hydraulic support and stops the rotation of the drum of the virtual 3D model of the shearer.

[0120] S74: Determine the action range of the virtual 3D model in the fault state according to the fault type and the updated virtual 3D model;

[0121] For example, the adjustable angle range of the hydraulic support's rib protection plate angle in the fault state is reduced. In the fault state of the shearer, the rotational speed range of the drum is reduced.

[0122] S75: Determine the states of the relevant components of the corresponding actual physical model according to the fault type and the updated virtual 3D model;

[0123] S76: Update the actual physical model according to the states of the relevant components of the actual physical model, and generate a control instruction that restricts the execution actions of the actual physical model according to the action range.

[0124] In this step, the state of the actual physical model is updated to be consistent with the virtual 3D model. For example, the hydraulic valve of the actual physical model of the hydraulic support is adjusted to the low-pressure state consistent with the virtual 3D model, and the support height synchronously drops to the state consistent with the virtual 3D model.

[0125] The control instruction that restricts the execution actions of the actual physical model can use a PWM signal to adjust the rotational speed of the DC motor to simulate the shearer being jammed or the scraper conveyor being stuck. Use a servo motor or a pneumatic device to limit the movement range of the physical model. Adjust the hydraulic flow through a proportional valve to simulate the hydraulic support losing pressure or getting stuck.

[0126] Compared with the training mode of playing fault handling videos in the prior art, in this embodiment, the method of triggering equipment fault injection by environmental changes is used to simulate the scenario of equipment faults in the virtual mine space, enabling trainees to feel the scenario of sudden equipment faults during the mining operation as if they were on the spot. And according to the equipment fault situation in the virtual space, the actual physical model is updated to make the state of the actual physical model consistent with the real equipment fault state, so that trainees can train in the scenario closest to the actual equipment faults, thereby significantly improving the training effect while ensuring safety. And during the process of the fault spreading from the virtual mine space to the training site in real time, the actual physical model and the virtual space remain consistent.

[0127] In this embodiment, the step S4: After performing an online update process on the digital twin of the coal mining equipment according to the real-time model parameters to obtain the updated digital twin further includes:

[0128] S41: Predict the local update range of the virtual 3D model according to the current model parameters and the historical model parameters;

[0129] In this step, the current model parameters and the historical model parameters can be compared to predict the actions to be performed by the virtual 3D model.

[0130] Specifically, it includes predicting the changing part and the fixed part of the virtual 3D model according to the current model parameters and the historical model parameters;

[0131] Predicting the possible changing parameters of the changing part according to the current model parameters and the historical model parameters;

[0132] Predicting the change range of the changing part according to the changing part and the possible changing parameters as the local update range.

[0133] For example, in the current model parameters and the historical model parameters of a hydraulic support, the relevant parameters of the base do not change, and when the length of the prop changes at a certain speed, the base of the hydraulic support is the fixed part, and the rest of the hydraulic support is the changing part. Since the speed of the prop length change will not change suddenly, before the update deadline, the length of the prop elongation is somewhat, and the possible elongation length range of the prop can be determined according to the speed of the length change and the reasonable increase amount. This range is the local update range of the aforementioned virtual 3D model.

[0134] S42: Generating the common difference data within the local update range according to the local update range of the virtual 3D model, and sending the common difference data to the cloud server for local update of the virtual 3D model;

[0135] Among them, the difference data refers to the quantity required for updating the part that changes when the changing part of the virtual 3D model changes. For example, in the aforementioned example, the data of the rest of the hydraulic support except the base is the difference data. Since the motion condition of the changing part is a predicted value and it may be any one of the situations within the predicted range, the data finally used for update cannot be completely determined. However, various situations of the changing part within the predicted range may have common update data, that is, the aforementioned common difference data. For example, in the aforementioned example, the elongation length of the column may be various lengths within the range, but these lengths are all greater than or equal to the length of the minimum elongation speed within the update range. At this time, the minimum elongation length of the column within the update range can be determined for updating the virtual 3D model. Therefore, the update data corresponding to the minimum elongation length of the column within the update range can be used as the common difference data within the local update range.

[0136] S43: Predicting a new local update range according to the latest collected model parameters and the historical model parameters;

[0137] Since the virtual model is constantly changing, as the update deadline approaches, the state of the virtual model is getting closer and closer to the final state, and the local update range is gradually shrinking. The determined common difference data is also constantly increasing. Therefore, in this step, new common difference data can be continuously added according to the new model parameters.

[0138] S44: Determine supplementary common difference data for supplementing the common difference data according to the new local update range;

[0139] The aforesaid newly added common difference data set is the supplementary common difference data.

[0140] S44: Send the supplementary common difference data to the cloud server for local update of the virtual 3D model;

[0141] After the supplementary common difference data is determined, it can be used for continuous update of the virtual 3D model.

[0142] S45: Determine whether the backup data sending condition is satisfied according to the new local update range, the supplementary common difference data, and the supplementary common difference data; specifically including: S451: Determine the data volume of the backup data according to the new local update range, the already sent common difference data, and the supplementary common difference data;

[0143] According to the new local update range, the update data corresponding to the new local update range can be obtained. After removing the already used common difference data and the supplementary common difference data, the data volume of the backup data is obtained.

[0144] S451: Judge the backup data transmission time and the remaining update time of the virtual 3D model according to the data volume of the backup data;

[0145] In this step, the backup data transmission time and the remaining update time of the virtual 3D model can be determined according to the system data transmission speed and the average update speed of the unit data volume of the virtual 3D model. The backup data includes the update data corresponding to each possible motion state of the virtual model within the new local update range. At this time, no matter which state the accurate virtual model reaches, there is corresponding data in the backup data for update. As the local update range continues to shrink, the final possible states of the virtual model also continue to shrink, and at this time the backup data volume also becomes smaller and smaller. Therefore, the backup data transmission time and the remaining update time of the virtual 3D model are also getting shorter and shorter.

[0146] S453: Obtain the preset waiting time for virtual 3D model update;

[0147] The waiting time is the time required to meet the update speed requirement

[0148] S454: Judge whether the data sending condition is satisfied according to the backup data transmission time, the remaining update time of the virtual 3D model, and the preset waiting time for 3D model update.

[0149] If the data volume of the backup data is small enough and the time required for transmitting and updating the backup data can meet the requirement of the preset waiting time, the data sending condition is satisfied.

[0150] S46: If the condition is met, send the current model parameters and backup data to the cloud server for local change update of the virtual 3D model; if not, repeat steps S43 to S46.

[0151] After receiving the backup data, the cloud server selects the data corresponding to the final state in the backup data according to the final state of the virtual 3D device for update.

[0152] In this embodiment, data preprocessing is performed on the locally changed part of the virtual 3D model by predicting the local update range, while the data of the fixed part remains unchanged. And by continuously determining the common difference data and continuously updating the local part, the update amount is reduced, and the update data can be processed in advance. Therefore, not only can the update speed of the virtual 3D model be significantly improved, but also the real-time performance of the training effect can be prevented from being affected by delays.

[0153] Embodiment 2

[0154] As Figure 9 shown, this embodiment provides a simulated mine digital twin teaching system based on virtual reality technology. This teaching system applies the teaching method described in Embodiment 1. The system includes a physical model of coal mining equipment, a cloud server, and a digital twin update module. Sensors are installed on the physical model of the coal mining equipment. The digital twin update module is used to update the digital twin according to the data collected by the sensors. The digital twin update module is communicatively connected to the cloud server. The cloud server generates a virtual mine space according to teaching requirements. Each sensor installed on the physical model of the coal mining equipment collects data reflecting the state of the physical model of the coal mining equipment in real time. The digital twin update module performs online update of the digital twin according to the sensor data. The updated digital twin drives the virtual 3D model of the coal mining equipment mapped in the virtual mine space to perform real-time online update.

[0155] The simulated mine digital twin teaching system based on virtual reality technology in this embodiment further includes VR glasses. The VR glasses are communicatively connected to the cloud server. During the process of students using this system for learning and training, they view the virtual space generated by the cloud server through the worn VR glasses.

[0156] The above is a detailed introduction to a simulated mine digital twin teaching system and method provided by an embodiment of the present invention.

[0157] It should be clear that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present invention.

[0158] The functional blocks shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, and so on. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted via a data signal carried in a carrier wave over a transmission medium or a communication link. A "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0159] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, can be different from the order in the embodiments, or several steps can be executed simultaneously.

[0160] As described above, the above is only the specific implementation manner of the present invention. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, modules, and units can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present invention.

Claims

1. A simulated mine digital twin teaching method based on virtual reality technology, characterized in that: include: S1: Generate a virtual mine space on the cloud server; S2: Obtain a virtual model of the coal mining equipment as a digital twin of the coal mining equipment; S3: obtaining real-time model parameters of the physical model of the coal mining equipment operated by the trainee; S4: performing online updating processing on the digital twin of the coal mining equipment according to the real-time model parameters to obtain an updated digital twin; S5: The cloud server maps a virtual 3D model of the corresponding coal mining equipment in the virtual mine space according to the updated digital twin; S6: The machine learning algorithm evaluates the trainee's coal mining operation based on the real-time model parameters.

2. The simulated mine digital twin teaching method based on virtual reality technology according to claim 1 is characterized in that: The S5 further includes: S51: predicting a local update range of the virtual three-dimensional model according to current model parameters and historical model parameters; S52: generating common difference data within the local update range according to the local update range of the virtual three-dimensional model, and sending the common difference data to the cloud server to perform local update of the virtual three-dimensional model; S53: predicting a new local update range according to the latest collected model parameters and historical model parameters; S54: Determine supplementary common difference data for supplementing the common difference data according to the new local update range; S54: Sending the supplementary common difference data to the cloud server to perform a partial update of the virtual three-dimensional model; S55: judging whether the backup data sending condition is met according to the new local update range, the supplementary common difference data and the supplementary common difference data; S56: If the conditions are met, the current model parameters and backup data are sent to the cloud server to update the local changes of the virtual three-dimensional model. If the conditions are not met, steps S53 to S56 are repeated.

3. The simulated mine digital twin teaching method based on virtual reality technology according to claim 1 is characterized in that: The S6 includes: S61: obtaining real-time model parameters when the trainee operates the physical model of coal mining equipment; S62: Obtain a support vector machine machine learning algorithm model trained with data; S63: The machine learning algorithm model outputs an evaluation result of the trainee's coal mining operation based on the input model parameters.

4. The simulated mine digital twin teaching method based on virtual reality technology according to claim 1 is characterized in that: The evaluation items of S6 include whether the hydraulic support is in place, whether the guard plate of the hydraulic support is open, whether the support resistance of the hydraulic support reaches the set value, whether the parallelism of adjacent hydraulic supports meets the preset requirements; whether the traction speed and position of the coal mining machine match the action of the hydraulic support, whether the drum height of the coal mining machine is correct, and whether the oblique cutting operation sequence of the coal mining machine is correct; whether the handling of the coal piling and material scattering of the scraper conveyor is correct, whether the straightness of the scraper conveyor meets the preset requirements, whether the end operation of the scraper conveyor is correct, whether the installation and evacuation operations of the end support are correct, whether the advance support matches the mining, and whether the length of the advance support meets the preset requirements.

5. The simulated mine digital twin teaching method based on virtual reality technology according to claim 2 is characterized in that: The S62 includes: S621: Acquire data of multiple sensors used to detect model parameters when operating coal mining equipment, and group the sensor data; S622: Obtaining a machine learning algorithm model corresponding to each evaluation item; S623: Obtaining evaluation results of each evaluation item corresponding to each set of sensor data; S624: Labeling each group of sensor data according to each evaluation result; S625: Train the machine learning algorithm model corresponding to each evaluation item using the labeled sensor data to obtain a data-trained support vector machine machine learning algorithm model.

6. The simulated mine digital twin teaching method based on virtual reality technology according to claim 2 is characterized in that: The S63 further includes: Collect data on the training progress and training results of trainees’ training programs; Determine the optimal classification boundary parameters corresponding to the training progress and training results according to the classification boundary optimization index of the support vector machine algorithm; Performing regression analysis on the data of training progress and training results and the optimal classification boundary parameters to obtain a mapping relationship between the data of training progress and training results and the optimal classification boundary parameters; When the change rate of training progress and training results reaches a preset value, the current student's training progress and training results are obtained; Obtaining optimal classification boundary parameters according to the current student's training progress and training results and the mapping relationship; Determine whether the difference between the current classification boundary parameter and the optimal classification boundary parameter exceeds a preset difference; If it exceeds, the optimal classification boundary parameter is smoothed according to the difference between the current classification boundary parameter and the optimal classification boundary parameter to obtain the classification boundary parameter set; Update the classification boundary of the support vector machine algorithm model in sequence according to each classification boundary parameter in the classification boundary parameter set; If it does not exceed, the classification boundary of the support vector machine algorithm model is updated using the optimal classification boundary parameter.

7. The simulated mine digital twin teaching method based on virtual reality technology according to claim 2 is characterized in that: The S63 includes: S631: Acquire a set of sensor data of coal mining equipment as target sensor data; S632: Obtain a machine learning algorithm corresponding to an evaluation item as a target machine learning algorithm model; S633: Obtaining data category labels corresponding to each target sensor data; S634: Input the target sensor data and the corresponding data category label into the target machine learning algorithm model; S635: The target machine learning algorithm model outputs an evaluation result of the corresponding evaluation item according to the input target sensor data and data category label; S636: Repeat steps S631 to S635 until all evaluation results corresponding to each group of sensor data are determined.

8. The simulated mine digital twin teaching method based on virtual reality technology according to claim 1 is characterized in that: The method further comprises: S71: When the evaluation result is that the preset requirements are not met, an operation error prompt message is sent, and a reverse update permission is applied; S72: when receiving the reverse update authorization message, obtaining a preset correct state of the virtual three-dimensional model of the coal mining equipment in the virtual mine; S73: obtaining an operation error of the coal mining equipment according to a preset accurate state of the virtual three-dimensional model of the coal mining equipment and a current state of the virtual three-dimensional model; S74: updating the digital twin to a correct state according to the operation error; S75: Control the physical model of the coal mining equipment to adjust to the correct state according to the digital twin after status update.

9. A simulated mine digital twin teaching system based on virtual reality technology, applying the teaching method described in any one of claims 1 to 8, characterized in that: It includes a physical model of coal mining equipment, a cloud server, and a digital twin update module. The physical model of coal mining equipment is equipped with sensors. The digital twin update module is used to update the digital twin according to the data collected by the sensors. The digital twin update module is communicatively connected to the cloud server.

10. The simulated mine digital twin teaching system based on virtual reality technology according to claim 9 is characterized in that: It also includes VR glasses, which are communicatively connected to the cloud server.

Images