Simulation mine digital twin teaching system and method based on virtual reality technology

Abstract

The application relates to the technical field of virtual reality, in particular to a simulation mine digital twin teaching system and method based on virtual reality technology. The method comprises the following steps: generating a virtual mine space on a cloud server; obtaining a virtual model of a coal mining device as a digital twin of the coal mining device; obtaining real-time model parameters of a physical model of the coal mining device operated by a student; performing online update processing on the digital twin of the coal mining device according to the real-time model parameters to obtain an updated digital twin; the cloud server maps a corresponding virtual three-dimensional model of the coal mining device in the virtual mine space according to the updated digital twin; and a machine learning algorithm evaluates the coal mining operation of the student according to the real-time model parameters. The application can enhance the real experience of the student during training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual reality, in particular to a simulation mine digital twin teaching system and method based on virtual reality technology. BACKGROUND

[0002] Mining engineering is a basic professional for national energy security, and the practical teaching of mining specialty is crucial for the professional ability training of students. At present, the practical teaching of mining specialty mostly adopts the way of letting students operate physical models of coal mining equipment. However, the traditional physical model teaching has the following problems: the physical model can only perform simple actions and cannot completely simulate the real mine environment; the physical model has limited space and cannot meet the diversified practical needs of students; the scene contacted by students is limited and differs greatly from the real mine site, and the practical experience is not realistic. In addition, due to the complexity of mining operation, the current teaching method is difficult to accurately evaluate the operation of students in time. SUMMARY

[0003] Therefore, the embodiments of the present application provide a simulation mine digital twin teaching system and method based on virtual reality technology, which is used to solve the technical problem that the current coal mining teaching method is limited by space scene and causes poor scene experience of students.

[0004] The technical scheme adopted by the present application is:

[0005] In a first aspect, the present application provides a multi-scene collaborative simulation mine teaching method based on digital twin, which comprises:

[0006] S1: generating a virtual mine space containing a large-dip-angle coal seam roadway model according to the arrangement form of the mining-preparation roadway in the actual production of the large-dip-angle coal seam mine;

[0007] S2: obtaining a virtual model of the working face equipment as a digital twin of the working face equipment;

[0008] S3: obtaining real-time model parameters of a plurality of working face equipment physical models in different scenes;

[0009] S4: obtaining an updated digital twin after online updating the digital twin of the working face equipment according to the real-time model parameters;

[0010] S5: mapping a corresponding virtual three-dimensional model of the working face equipment in the virtual mine space according to the updated digital twin;

[0011] S6: evaluating the working face equipment operation according to the real-time model parameters by using a machine learning algorithm.

[0012] In a second aspect, the present application also provides a multi-scenario collaborative simulation mine teaching system based on digital twinning, which applies the teaching method of the first aspect and comprises a working face equipment physical model, a cloud server and a digital twinning body updating module.

[0013] Beneficial effects: The simulation mine digital twinning teaching system and method based on virtual reality technology of the present application can accurately simulate the roadway layout, equipment distribution and complex geological conditions of an actual mine through the virtual mine space generated by the cloud server, thereby providing an immersive and high-fidelity practical training environment for students and making the practical training of trainees no longer limited by the site. The coal mining equipment virtual model such as the hydraulic support, coal mining machine and scraper conveyor in the present application is kept synchronous with the physical model operated by the trainee through the mode of twinning body updating driven by the physical model parameters, thereby ensuring that the virtual scene is consistent with the actual operation and enhancing the authenticity and intuitiveness of the teaching. The real-time model parameters of the physical model are collected through the sensor, the digital twinning body is updated online, and the seamless linkage of the virtual and the real is realized. The real-time model parameters of the physical model are utilized and the operation of the trainee is automatically evaluated through the machine learning algorithm, thereby accurately and timely feeding back the coal mining operation training condition of the trainee, which is beneficial to the trainee to correct the shortcomings in the training site in a timely manner, thereby significantly improving the teaching effect. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly introduced as follows, and other drawings can also be obtained by those of ordinary skill in the art without creative labor on the premise of not paying creative labor, and these are within the protection scope of the present application.

[0015] Figure 1 The flowchart of the simulation mine digital twinning teaching method based on virtual reality technology of the present application is shown in the figure.

[0016] Figure 2 The layout structure diagram of the virtual mine space in the present application is shown in the figure.

[0017] Figure 3 The position diagram of the sensor installed on the hydraulic support in the present application is shown in the figure.

[0018] Figure 4 The diagram of the virtual three-dimensional model mapped out in the virtual mine space by the digital twinning body in the present application is shown in the figure.

[0019] Figure 5 The flowchart of the method for evaluating the coal mining operation of the trainee in the present application is shown in the figure.

[0020] Figure 6 A flowchart of the method for training the support vector machine algorithm model of the present application;

[0021] Figure 7 A flowchart of the method for evaluating the coal mining operation of the student by using the machine learning algorithm of the present application;

[0022] Figure 8 A flowchart of the method for updating the physical model in reverse of the present application;

[0023] Figure 9 A structural block diagram of the simulated mine digital twin teaching system based on virtual reality technology of the present application.

[0024] Parts and their numbers in the figure:

[0025] Guard plate angle sensor 11, side guard plate displacement sensor 12, top beam posture sensor 13, infrared sensor 14, shield beam posture sensor 15, support displacement sensor 16, base posture sensor 17, push jacking displacement sensor 18. DETAILED DESCRIPTION

[0026] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. It should be noted that, in this document, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or sequence between these entities or operations. In the description of the present application, it should be understood that the orientations or positional relationships indicated by terms such as center, upper, lower, front, back, left, right, vertical, horizontal, top, bottom, inner, outer and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. Moreover, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement “include” do not exclude the presence of additional identical elements in the process, method, article or device that includes the elements. If there is no conflict, the embodiments of the present application and the various features in the embodiments can be combined with each other, and all within the scope of protection of the present application.

[0027] Embodiment 1

[0028] As shown in Figure 1 The present application provides a virtual reality technology-based simulation mine digital twin teaching method, which mainly includes:

[0029] S1: generating a virtual mine space on a cloud server;

[0030] The virtual mine space can be obtained after three-dimensional modeling according to the actual mine space. It can also be obtained after making some modifications in the three-dimensional model produced according to the actual mine space as needed. Figure 2 The layout of the created virtual mine space is shown in the figure. In specific implementation, the virtual mine space of the scene meeting different teaching needs can be generated in advance. Then the user selects the virtual mine space model used for this training according to the training needs.

[0031] S2: obtaining a virtual model of a coal mining equipment as a digital twin of the coal mining equipment;

[0032] Various coal mining equipment used in coal mining operations can be pre-modeled in 3D to obtain corresponding virtual models of the equipment. These virtual models can serve as digital twins of the coal mining equipment.

[0033] S3: Obtain real-time model parameters of the physical model of the coal mining equipment operated by the trainee;

[0034] The aforementioned physical model of the coal mining equipment is a teaching and training device modeled after actual coal mining equipment, possessing essentially the same structure and function as the actual coal mining equipment used in real mines. Trainees can operate the physical model in a safe training environment. When trainees operate the physical model, its state, such as the model's posture, and the relative positions and postures of its various components, will change. The parameters reflecting the physical model's state, the relative positions and postures of its components, etc., constitute the parameters of the physical model. This step involves real-time data acquisition of these model parameters while trainees are operating the equipment, thus obtaining the model parameters of the coal mining equipment physical model.

[0035] S4: The updated digital twin of the mining equipment is obtained by performing online update processing based on the real-time model parameters;

[0036] This step uses the collected real-time model parameters to update the digital twin, ensuring that the model parameters of the digital twin are consistent with those of the physical model. This allows the state of the digital twin to be consistent with the state of the physical model of the coal mining equipment, so that the trainees' operations on the physical model of the coal mining equipment in the training field can be reflected in the digital twin in a timely manner.

[0037] S5: The cloud server maps the corresponding virtual 3D model of the coal mining equipment in the virtual mine space based on the updated digital twin;

[0038] like Figure 4 As shown, an initial virtual 3D model of the coal mining equipment can be imported into the virtual mine space before the start of teaching and training. Then, after the digital twin is updated, the updated digital twin is used to update the virtual 3D model of the coal mining equipment in the virtual mine space, ensuring that the virtual 3D model of the coal mining equipment in the virtual mine is consistent with the physical model of the coal mining equipment operated by the trainees. In this way, trainees can experience the effects of operating the coal mining equipment in the virtual mine, giving trainees undergoing practical training in the training venue an experience close to that of coal mining operations in an actual mine space.

[0039] S6: Evaluate the trainee's coal mining operation based on the real-time model parameters.

[0040] This embodiment utilizes real-time model parameters to update the digital twin online in real time, and uses the digital twin to update the virtual 3D model of the coal mining equipment in the virtual mine in real time. On the other hand, it uses the model parameters of the collected physical model of the coal mining equipment to evaluate the trainees' coal mining operations. This allows trainees to receive timely feedback on their training while conducting coal mining operations in a realistic scenario. This is more conducive to trainees identifying their shortcomings during the training process and making targeted improvements, thereby significantly improving the training effect.

[0041] like Figure 5 As shown, in this embodiment, step S6: evaluating the trainee's coal mining operation based on the real-time model parameters includes:

[0042] S61: Obtain real-time model parameters when trainees operate the physical model of coal mining equipment;

[0043] In this embodiment, the real-time model parameters obtained when the physical model of the coal mining equipment is acquired include, but are not limited to, the height of the hydraulic support column, the angle of the hydraulic support side plate, the position of the hydraulic support, the angle of the coal mining machine rocker arm, the position of the coal mining machine, the traction speed of the coal mining machine, and the speed of the belt conveyor.

[0044] To accurately obtain real-time model parameters when trainees operate the physical model of coal mining equipment, sensors can be pre-installed at designated locations on each piece of the physical model. These sensors collect data reflecting the model parameters. For example, for hydraulic supports, displacement and pressure sensors can be installed on the columns, angle sensors at the sidewall hinges, displacement sensors on the front extension beam, and displacement sensors on the jacks. For coal mining machines, angle sensors can be installed on the rocker arms, and speed sensors on the drums. Position sensors can also be installed on scraper conveyors.

[0045] As shown in Figure 3 Figure 3 The image shows the positions of various sensors installed on the hydraulic support, such as the side guard plate angle sensor 11 installed on the side guard plate, the side guard plate displacement sensor 12 installed on the side guard plate, the top beam attitude sensor 13 installed on the top beam, the infrared sensor 14 installed on the lower part of the top beam, the shield beam attitude sensor 15 installed on the shield beam, the support column displacement sensor 16 installed on the support column, the base attitude sensor 17 installed on the base, and the push jack displacement sensor 18 installed on the push jack.

[0046] S62: Obtain the support vector machine machine learning algorithm model trained on the data;

[0047] This step first uses data collected by sensors to train a machine learning algorithm model. Through data training, the parameters of the machine learning algorithm model are optimized so that it can accurately classify the data collected by the sensors. In this embodiment, the machine learning algorithm uses a support vector machine.

[0048] S63: The machine learning algorithm model outputs the evaluation results of the trainee's coal mining operation based on the input model parameters.

[0049] After acquiring the data reflecting model parameters detected by the sensors, this data is input into a trained machine learning algorithm model. The machine learning algorithm outputs the evaluation results of the trainee's coal mining operation based on the input data. Model parameters can indirectly reflect the trainee's operation of the coal mining equipment. However, the evaluation of whether the trainee's operation meets the standard operating requirements is affected by multiple model parameters, and different model parameters can also influence each other. Therefore, there is a relatively complex nonlinear relationship between model parameters and evaluation results. This embodiment uses the support vector machine algorithm in machine learning to process the data collected by the sensors to obtain the evaluation results of the trainee's coal mining operation. The evaluation results include, but are not limited to, the evaluation of hydraulic supports: whether the support is moved in place, whether the side guard plate is opened, whether the support resistance reaches the set value, and whether the parallelism of adjacent supports meets the requirements, etc.

[0050] Evaluation of the coal mining machine includes: whether the traction speed and position match the support movement, whether the drum height is correct, and whether the oblique cutting sequence is correct, etc.

[0051] The evaluation of scraper conveyors includes: whether the handling of coal pile-up and material spillage situations was correct; whether the straightness of the scraper conveyor meets the requirements; and whether the operation of the scraper conveyor end is correct, etc. The evaluation of end support includes: whether the installation and removal of end support were correct; whether the advance support is compatible with mining operations; and whether the length of the advance support meets the requirements.

[0052] like Figure 6 As shown, in this embodiment, step S62: obtaining the support vector machine machine learning algorithm model trained on the data includes the following steps:

[0053] S621: Acquire data from multiple sensors used to detect model parameters when trainees operate coal mining equipment, and group the sensor data;

[0054] The embodiment uses sensors installed at the setting positions of the respective coal mining devices to detect and collect model parameters. Since the evaluation results are affected by multiple model parameters in multiple coal mining devices, this step can group the sensor data according to the evaluation content, and group the sensor data that has an impact on the evaluation content in the same group, so that the data of multiple sensors can be fully utilized, and irrelevant data can be effectively filtered out, thereby improving the evaluation accuracy and increasing the timeliness of the evaluation. For example, the data collected within a time range reflecting the model parameters of the coal mining machine and the model parameters of the support can be grouped as evaluation data for whether the traction speed and position of the coal mining machine match the action of the support.

[0055] S622: Obtain the machine learning algorithm model corresponding to each evaluation item;

[0056] The corresponding machine learning algorithm model can be trained in advance for each evaluation, for example, for the result of whether the traction speed and position of the coal mining machine match the action of the support, the corresponding machine learning algorithm model can be selected for training, and the parameters of the algorithm model can be optimized. The trained algorithm model is used as the machine learning algorithm model for evaluating whether the traction speed and position of the coal mining machine match the action of the support.

[0057] S623: Obtain the respective evaluation results corresponding to each group of sensor data;

[0058] In the specific implementation of this step, an operator can operate the coal mining device, then obtain the data of the parameters of the coal mining device detected by the sensor during the operation, and manually determine whether the operation meets the corresponding requirements according to the operation specification requirements. The result is used as the evaluation result of the corresponding sensor data.

[0059] S624: Label each group of sensor data according to the respective evaluation results;

[0060] For example, the evaluation result corresponding to a group of sensor data is that the flatness of the scraper conveyor meets the requirements, and the group of sensor data can be labeled with the flatness of the scraper conveyor meeting the requirements.

[0061] S625: Train the machine learning algorithm model corresponding to each evaluation result through the labeled sensor data to obtain a support vector machine machine learning algorithm model trained by data.

[0062] This step trains the machine learning algorithm through the labeled sensor data, and continuously optimizes the parameters of the model until the evaluation effect of the machine learning algorithm model meets the requirements.

[0063] For example, the evaluation result corresponding to a group of sensor data is that the flatness of the scraper conveyor meets the requirements, and the group of sensor data can be labeled with the flatness of the scraper conveyor meeting the requirements. Figure 7As shown, in the present embodiment, the S63: the machine learning algorithm model outputs the evaluation result of the learner's coal mining operation according to the input model parameters, which includes the following steps:

[0064] S631: obtaining a set of sensor data of the coal mining equipment as target sensor data;

[0065] For example, the training operation effect of whether the support frame is in place needs to be evaluated, and the result of whether the support frame is in place can be obtained. The set of sensor data reflecting whether the support frame is in place can be used as the target sensor data.

[0066] S632: obtaining a machine learning algorithm corresponding to an evaluation result as a target machine learning algorithm;

[0067] For example, the evaluation result is whether the support frame is in place, and the machine learning algorithm model trained for the evaluation result of whether the support frame is in place can be obtained as the target machine learning algorithm model for classifying the target sensor data.

[0068] S633: obtaining a data class label corresponding to each target sensor data;

[0069] The data class label includes positive class (represented by numerical value +1) and negative class (represented by numerical value -1).

[0070] The method of obtaining the data class label corresponding to the sensor data can adopt threshold method, that is, according to the safety operation specification of the equipment, the threshold is set, and the class is directly judged by the sensor data. For example, if the data of the pressure sensor on the hydraulic support column exceeds the rated range, it is marked as negative class, and if it does not exceed the rated range, it is marked as positive class. In addition, the sensor data can be input into the deep learning algorithm model, and the deep learning algorithm model can automatically output the data class label corresponding to the sensor data.

[0071] S634: inputting the target sensor data and the corresponding data class into the model of the target machine learning algorithm;

[0072] S635: the target machine learning algorithm model outputs the corresponding evaluation result according to the input target sensor data and data class label. Specifically, it includes the following steps:

[0073] Obtaining the support vector corresponding to the target sensor data;

[0074] Calculating the kernel function value of the support vector;

[0075] Multiplying each kernel function value with the corresponding Lagrange multiplier and class label (if the class label is positive class, multiply by +1, if the class label is negative class, multiply by -1), and then adding all the multiplied results.

[0076] The calculation result is obtained after adding a bias term to the result of the addition.

[0077] The method of the embodiment further comprises the following steps:

[0078] The S63: the machine learning algorithm model outputs the evaluation result of the learner's coal mining operation according to the input model parameters further comprises:

[0079] S6301: collect data of training progress and training performance of the training project of the learner;

[0080] During the training process, the training can be carried out according to the training project, and the training progress and the training performance are given to the learner during the training process. The system collects data reflecting the training progress and the training performance.

[0081] The training progress can be represented by the ratio of the number of completed training steps of the training project to the total number of steps of the training project. The training performance can be evaluated by a machine learning algorithm. The operation accuracy and the response time can also be used to score. The operation accuracy is the ratio of the number of correct operations to the total number of operations, and different ratios correspond to different scores. The response time is the average time consumed from the issuance of the instruction to the completion of the operation, and different time consumptions correspond to different scores.

[0082] S6302: determine the optimal classification boundary parameter corresponding to the training progress and the training performance according to the classification boundary optimization index of the support vector machine algorithm;

[0083] The classification boundary optimization index can be maximum margin, minimum classification error rate, or the sum of the two with a certain weight. In specific implementation, a multi-objective optimization algorithm can be used to find the optimal classification boundary parameter that satisfies the optimization index. The classification boundary parameter is a parameter that determines the classification boundary of the support vector machine, such as the scale parameter and the bias term.

[0084] S6303: regression analysis of the data of the training progress and the training performance and the optimal classification boundary parameter to obtain the mapping relationship between the data of the training progress and the training performance and the optimal classification boundary parameter;

[0085] For the support vector machine algorithm using the Sigmoid kernel function, the kernel function calculation formula is

[0086]

[0087] Where γ and r can be used as classification boundary parameters.

[0088] S6304: obtain the current training progress and training performance of the learner when the change rate of the training progress and the training performance reaches a preset value;

[0089] The condition for updating the classification boundary can be set in advance, and in this embodiment, the preset variation rate of the training progress and the training result is taken as the condition for updating the classification boundary.

[0090] S6305: obtaining the optimal classification boundary parameter according to the current training progress and training result of the trainee and the mapping relationship;

[0091] However, after reaching the preset condition, the optimal classification boundary parameter is obtained according to the corresponding relationship obtained by the previous regression analysis.

[0092] S6306: judging whether the difference between the current classification boundary parameter and the optimal classification boundary parameter exceeds the preset difference value;

[0093] If the difference between the classification boundary before and after the update is too large, it will cause the mutation of the evaluation result, and for this, the preset difference value is used to control the change of the classification boundary in this step.

[0094] S6307: if the difference exceeds, the optimal classification boundary parameter is smoothed to obtain a classification boundary parameter set;

[0095] In this step, several gradually changed classification boundary parameters are found between the current classification boundary parameter and the optimal classification boundary parameter to form a classification boundary parameter set, and each classification boundary parameter is assigned an order, so that the difference between adjacent classification boundary parameters is less than the preset value.

[0096] S6308: updating the classification boundary of the support vector machine algorithm model according to each classification boundary parameter in the classification boundary parameter set;

[0097] The parameters of the classification boundary are updated in order according to the order assigned in the previous step, so that the classification boundary is updated in order.

[0098] S6309: if the difference does not exceed, the optimal classification boundary parameter is used to update the classification boundary of the support vector machine algorithm model.

[0099] After updating the classification boundary, the system evaluates the operation of the trainee by using the updated support vector machine algorithm.

[0100] The above-mentioned method of dynamically updating the classification boundary according to the progress and the result can make the evaluation method adapt to the training situation of the trainee and make the training difficulty increase scientifically.

[0101] As shown in Figure 8 The method further comprises:

[0102] S71: send an operation error prompt message when the evaluation result does not meet the preset requirement, and apply for reverse update authority; for example, when the evaluation result is that the length of the advance support is insufficient to meet the preset requirement, the corresponding error prompt message can be sent to prompt that the current operation is incorrect, and the operation causes the length of the advance support to be insufficient to meet the preset requirement. At the same time, the system can send a request for applying for the authority of reverse update to the terminal device of the operator (such as a VR glasses) or the terminal device of the teacher.

[0103] S72: obtain the preset correct state of the virtual three-dimensional model of the coal mining equipment in the virtual mine when receiving the reverse update authorization message.

[0104] The operator or the teacher can decide whether to agree to the reverse update according to the situation after receiving the request for applying for the authority of reverse update, and the result can be sent to the system through the terminal device of the operator or the terminal device of the teacher. When the system receives the reverse update authorization message, the preset correct state of the virtual three-dimensional model of the coal mining equipment in the virtual mine is obtained. For example, the length of the advance support of the hydraulic support is made to reach the correct length that meets the requirement.

[0105] S73: obtain the operation error of the coal mining equipment according to the preset correct state of the virtual three-dimensional model of the coal mining equipment and the current state of the virtual three-dimensional model.

[0106] In this step, the preset correct state of the virtual three-dimensional model that meets the requirement and the state of the current virtual three-dimensional model (an error state caused by incorrect operation) are compared, and the error between the two is obtained as the operation error of the coal mining equipment.

[0107] S74: update the digital twin to the correct state according to the operation error.

[0108] In this step, the digital twin is reversely updated according to the operation error, so that the digital twin is updated to a state consistent with the preset correct state of the virtual three-dimensional model. For example, the digital twin of the hydraulic support is updated to a correct length that meets the requirement of the length of the advance support.

[0109] S75: control the physical model of the coal mining equipment to adjust to the correct state according to the state updated digital twin.

[0110] In this step, the physical model operated by the student is directionally updated by using the updated digital twin, so that the state of the physical model is consistent with the preset correct state of the virtual three-dimensional model. In this way, the student can know the correct state of the coal mining equipment after correct operation in a timely manner, which is beneficial to correcting the incorrect operation of the student in a timely manner.

[0111] In this embodiment, the method further comprises:

[0112] S71: injecting a corresponding type of fault into the virtual mine space according to the environmental change when the environmental change in the virtual mine space triggers a device fault;

[0113] The environmental change in the mine space can indeed cause a device fault. The mine environment is complex and changeable, and geological conditions, climate factors, and device operating states can all have a direct impact on the device, thereby causing a fault to occur. For example, rock stratum fracture, roof subsidence, and coal seam hardness change can cause roof pressure to suddenly increase, which can cause insufficient support force of the hydraulic support, triggering support pressure loss. Roof subsidence or rock stratum movement can cause the moving parts of the hydraulic support to be stuck. Uneven coal seam thickness, fault area, and dirt band can easily cause coal seam thickness to change. Sudden increase in coal seam thickness can cause the shearer drum to be overloaded, triggering stall. Coal seam hardness change can cause the shearer traction motor to be overloaded. Rock stratum creep and support failure can cause roadway deformation, which can cause the chain of the scraper conveyor to be misaligned or stuck. Roadway width change can cause the device to collide with the roadway wall. High temperature can cause the viscosity of hydraulic oil to decrease, causing insufficient pressure of the hydraulic system. Coal seam hardness change and device operating speed adjustment can cause the device load to fluctuate sharply. Sudden increase in device load can cause the motor to be overloaded. Excessive load of the scraper conveyor can cause the chain to break.

[0114] This step can introduce environmental change factors into the virtual mine space. As the training process continues to advance, the environment in the virtual space changes, and when the environmental change meets the conditions for triggering a device fault, various faults generated under the conditions can be injected into the virtual space.

[0115] S72: determining corresponding states of each virtual three-dimensional model in the virtual mine space under the fault according to the injected fault type;

[0116] For different fault types, the coal mining equipment will have different states, that is, the positions, postures, and movement speeds of each component of the coal mining equipment will have corresponding states under the fault. For example, under the condition of support pressure loss, the height of the hydraulic support decreases, and the movement speed of the component slows down. Under the condition of sudden increase in coal seam thickness, which can cause the shearer to stall, the drum of the shearer is in a state of stopping rotation.

[0117] S73: updating the virtual three-dimensional model according to the corresponding state;

[0118] This step updates the virtual three-dimensional model in the virtual mine space, for example, the height of the virtual three-dimensional model of the hydraulic support is lowered, and the drum of the virtual three-dimensional model of the shearer stops rotating.

[0119] S74: determining the action range of the virtual three-dimensional model in the fault state according to the fault type and the updated virtual three-dimensional model;

[0120] For example, the adjustable angle range of the hydraulic support guard plate angle in the fault state is reduced, and the rotating speed range of the drum of the coal mining machine in the fault state is reduced.

[0121] S75: determining the state of the corresponding actual physical model related components according to the fault type and the updated virtual three-dimensional model;

[0122] S76: updating the actual physical model according to the state of the actual physical model related components, and generating control instructions for limiting the execution action of the actual physical model according to the action range.

[0123] This step updates the state of the actual physical model to be consistent with the virtual three-dimensional model. For example, the hydraulic valve of the actual physical model of the hydraulic support is adjusted to a low pressure state consistent with the virtual three-dimensional model, and the support height is synchronously lowered to a state consistent with the virtual three-dimensional model.

[0124] The control instructions for limiting the execution action of the actual physical model can use PWM signals to adjust the rotating speed of the DC motor, simulate the locked rotor of the coal mining machine or the chain jam of the scraper conveyor. The motion range of the actual physical model is limited by using a servo motor or a pneumatic device. The hydraulic flow is adjusted by a proportional valve to simulate the loss of pressure or jam of the hydraulic support.

[0125] Compared with the training mode of playing the fault handling video in the prior art, the embodiment simulates the scene of equipment failure in the virtual mine space by using the method of triggering equipment failure injection by environmental change, so that the trainee can experience the scene of sudden equipment failure in the process of time mining operation, and the actual physical model is updated according to the equipment failure in the virtual space, so that the state of the actual physical model is consistent with the real equipment failure state, so that the trainee can train in the scene closest to the actual equipment failure, thereby significantly improving the training effect while ensuring safety. And in the process of spreading the fault from the virtual mine space to the time training place, the actual physical model and the virtual space are consistent.

[0126] In the embodiment, the S4: online updating the digital twin of the coal mining equipment according to the real-time model parameters to obtain an updated digital twin further comprises:

[0127] S41: predicting the local update range of the virtual three-dimensional model according to the current model parameters and the historical model parameters;

[0128] This step can compare the current model parameters and the historical model parameters, and predict the action to be performed by the virtual three-dimensional model.

[0129] predicting a changing part and a fixed part of the virtual three-dimensional model according to the current model parameters and the historical model parameters;

[0130] predicting possible changing parameters of the changing part according to the current model parameters and the historical model parameters;

[0131] predicting a changing range of the changing part as a local updating range according to the changing part and the possible changing parameters.

[0132] For example, the base of the hydraulic support is the fixed part and the rest of the hydraulic support is the changing part when the relevant parameters of the base in the current model parameters and the historical model parameters of the hydraulic support do not change and the length of the column changes at a certain speed. Since the speed of the column length change will not be suddenly changed, before the updating deadline, the length of the column elongation can be determined according to the speed of the length change and a reasonable increase amount, and the length range of the column elongation can be determined, which is the local updating range of the virtual three-dimensional model.

[0133] S42: generating common difference data in the local updating range according to the local updating range of the virtual three-dimensional model, and sending the common difference data to the cloud server for local updating of the virtual three-dimensional model;

[0134] The difference data refers to the amount of data required for updating the changing part of the virtual three-dimensional model. For example, in the foregoing example, the data of the rest of the hydraulic support except the base is the difference data. Since the motion of the changing part is a predicted value, any one of the predicted ranges can be possible, and therefore the data for updating cannot be completely determined. However, the various conditions of the changing part in the predicted range can have common updating data, that is, the foregoing common difference data. For example, in the foregoing example, the length of the column elongation can be various lengths in the range, but all the lengths are greater than or equal to the length of the minimum elongation speed in the updating range, and the minimum length of the column elongation in the updating range can be determined for updating of the virtual three-dimensional model, and therefore the updating data corresponding to the minimum length of the column elongation in the updating range can be used as the common difference data in the local updating range.

[0135] S43: predicting a new local updating range according to the latest model parameters and the historical model parameters;

[0136] Since the virtual model is constantly changing, as the updating deadline approaches, the state of the virtual model is closer and closer to the final state, and the local updating range is gradually reduced. The common difference data that can be determined is also increasing, and therefore new common difference data can be continuously added according to the new model parameters in this step.

[0137] S44: determining the supplementary common difference data for supplementing the common difference data according to the new local update range;

[0138] The aforementioned new common difference data set is the supplementary common difference data.

[0139] S44: sending the supplementary common difference data to the cloud server for local update of the virtual three-dimensional model;

[0140] The supplementary common difference data can be used for continuous update of the virtual three-dimensional model after being determined.

[0141] S45: determining whether the backup data sending condition is met according to the new local update range, the supplementary common difference data, and the supplementary common difference data; specifically including: S451: determining the data amount of the backup data according to the new local update range, the sent common difference data, and the supplementary common difference data;

[0142] According to the new local update range, the update data corresponding to the new local update range can be obtained, and after removing the common difference data and the supplementary common difference data that have been used for update, the data amount of the backup data is obtained.

[0143] S451: determining the backup data transmission time and the remaining update time of the virtual three-dimensional model according to the data amount of the backup data;

[0144] This step can determine the backup data transmission time and the remaining update time of the virtual three-dimensional model according to the system data transmission speed and the average update speed of the virtual three-dimensional model per unit data amount. The backup data includes the update data corresponding to each possible motion state of the virtual model in the new local update range. At this time, no matter which state the accurate virtual model reaches, there is data corresponding to it in the backup data for update. As the local update range continuously shrinks, the final possible state of the virtual model also continuously shrinks, and at this time, the backup data amount also becomes smaller and smaller. Therefore, the backup data transmission time and the remaining update time of the virtual three-dimensional model also become shorter and shorter.

[0145] S453: obtaining a virtual three-dimensional model update preset waiting time;

[0146] The waiting time is the time required to meet the update speed requirement

[0147] S454: determining whether the data sending condition is met according to the backup data transmission time, the remaining update time of the virtual three-dimensional model, and the virtual three-dimensional model update preset waiting time.

[0148] If the data amount of the backup data is small enough, the time required for transmission and update of the backup data can meet the requirement of the preset waiting time, and the data sending condition is met.

[0149] S46: If the current model parameters, backup data are sent to the cloud server for partial change update of the virtual three-dimensional model, if not, repeat steps S43 to S46.

[0150] After the cloud server receives the backup data, the data corresponding to the final state in the backup data is selected for update according to the final state of the virtual three-dimensional device.

[0151] The embodiment predicts the local update range to pre-process the data of the local change part of the virtual three-dimensional model, and the data of the fixed part remains unchanged. And by continuously determining the common difference data, the local part is continuously updated, which reduces the update amount, and the update data can be processed in advance, so it can not only significantly improve the update speed of the virtual three-dimensional model, but also avoid the influence of the real-time training effect caused by delay.

[0152] Embodiment 2

[0153] As shown in Figure 9 The embodiment provides a simulation mine digital twin teaching system based on virtual reality technology, which applies the teaching method of embodiment 1. The system includes a coal mining equipment physical model, a cloud server, and a digital twin update module. The coal mining equipment physical model is installed with sensors. The digital twin update module is used to update the digital twin according to the data collected by the sensors. The digital twin update module is in communication connection with the cloud server. The cloud server generates a virtual mine space according to the teaching requirements. Each sensor installed on the coal mining equipment physical model collects data reflecting the state of the coal mining equipment physical model in real time. The digital twin update module updates the digital twin online according to the sensor data. The updated digital twin drives the virtual three-dimensional model of the coal mining equipment mapped in the virtual mine space to be updated online in real time.

[0154] The simulation mine digital twin teaching system based on virtual reality technology of the embodiment further includes a VR glasses in communication connection with the cloud server. During the learning and training process using the system, the student watches the virtual space produced by the cloud server through the VR glasses.

[0155] The above is a detailed introduction to the simulation mine digital twin teaching system and method based on virtual reality technology provided by the embodiment of the present application.

[0156] It is to be understood that the application is not limited to particular configurations and processes described herein and shown in the drawings, as such can include any alternatives, modifications, additions or omissions as is readily appreciated by one of ordinary skill in the art. For instance, the methods of the application can be implemented in software, hardware, firmware, or any combination thereof. For the sake of brevity, conventional techniques and methods related to making and using aspects of the application can or can not be described in detail herein. In the above embodiments, a number of specific steps are described and illustrated as examples. However, the methods of the application are not limited to the specific steps described and illustrated herein, but rather include any alternatives, modifications, additions or omissions as is readily appreciated by one of ordinary skill in the art.

[0157] The functional blocks shown in the block diagrams described above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transfer information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via a computer network such as the Internet, an intranet, etc.

[0158] It is also to be understood that the example embodiments mentioned in the present application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the steps described above, that is, the steps can be performed in the order mentioned in the embodiments, or in an order different from the embodiments, or several steps can be performed simultaneously.

[0159] The above merely illustrates the specific implementation of the present application. Those skilled in the art can clearly understand that, for the sake of brevity and conciseness, the specific working processes of the above-described systems, modules and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein. It should be understood that the protection scope of the present application is not limited thereto, and any modifications or replacements within the technical scope disclosed by the present application can be easily conceived by those skilled in the art, and should be included in the protection scope of the present application.

Claims

1. A simulated mine digital twin teaching method based on virtual reality technology, characterized in that, The method comprises the following steps: S1: generating a virtual mine space on a cloud server; S2: obtaining a virtual model of the coal mining equipment as a digital twin of the coal mining equipment; S3: obtaining real-time model parameters of the physical model of the coal mining equipment operated by the trainee; S4: performing online updating processing on the digital twin of the coal mining equipment according to the real-time model parameters to obtain an updated digital twin; S5: mapping a corresponding virtual three-dimensional model of the coal mining equipment in the virtual mine space according to the updated digital twin by the cloud server; S6: evaluating the coal mining operation of the trainee according to the real-time model parameters by a machine learning algorithm; The S6 comprises the following steps: S61: obtaining real-time model parameters when the trainee operates the physical model of the coal mining equipment; S62: obtaining a support vector machine machine learning algorithm model trained by data; S63: outputting an evaluation result of the coal mining operation of the trainee by the machine learning algorithm model according to the input model parameters; The S63 further comprises the following steps: collecting data of training progress and training results of the trainee; determining optimal classification boundary parameters corresponding to the training progress and the training results according to a classification boundary optimization index of a support vector machine algorithm; performing regression analysis on the data of the training progress and the training results and the optimal classification boundary parameters to obtain a mapping relationship between the data of the training progress and the training results and the optimal classification boundary parameters; obtaining current training progress and training results of the trainee when a change rate of the training progress and the training results reaches a preset value; obtaining the optimal classification boundary parameters according to the current training progress and the training results and the mapping relationship; judging whether a difference between a current classification boundary parameter and the optimal classification boundary parameters exceeds a preset difference value; if yes, performing smoothing processing on the optimal classification boundary parameters according to the difference between the current classification boundary parameter and the optimal classification boundary parameters to obtain a classification boundary parameter set; updating the classification boundary of the support vector machine algorithm model according to each classification boundary parameter in the classification boundary parameter set in sequence; if no, updating the classification boundary of the support vector machine algorithm model by using the optimal classification boundary parameters.

2. The virtual reality technology-based simulated mine digital twin teaching method according to claim 1, characterized in that, The S5 further comprises the following steps: S51: predicting a local updating range of the virtual three-dimensional model according to current model parameters and historical model parameters; S52: generating common difference data in the local updating range according to the local updating range of the virtual three-dimensional model, and sending the common difference data to the cloud server for local updating of the virtual three-dimensional model; S53: predicting a new local updating range according to newly collected model parameters and historical model parameters; S54: determining supplementary common difference data for supplementing the common difference data according to the new local updating range; S54: sending the supplementary common difference data to the cloud server for local updating of the virtual three-dimensional model; S55: judging whether a backup data sending condition is met according to the new local updating range, the supplementary common difference data and the supplementary common difference data; S56: if yes, sending the current model parameters and the backup data to the cloud server for local updating of the virtual three-dimensional model; if no, repeating steps S53 to S56.

3. The virtual reality technology-based simulated mine digital twin teaching method according to claim 1, characterized in that, The evaluation items of the S6 include whether the hydraulic support is in place, whether the hydraulic support guard plate is opened, whether the hydraulic support support resistance reaches the set value, whether the parallelism of adjacent hydraulic supports meets the preset requirements; whether the traction speed and position of the coal mining machine match the action of the hydraulic support, whether the drum height of the coal mining machine is correct, whether the oblique cutting operation sequence of the coal mining machine is correct; whether the coal piling and material scattering of the scraper conveyor is handled correctly, whether the flatness of the scraper conveyor meets the preset requirements, whether the end operation of the scraper conveyor is correct, whether the installation and removal operation of the end support is correct, whether the advance support matches the mining, and whether the advance support length meets the preset requirements.

4. The virtual reality technology-based simulated mine digital twin teaching method according to claim 1, characterized in that, The S62 includes: S621: acquiring data of a plurality of sensors for detecting model parameters when operating the coal mining equipment, and grouping the sensor data; S622: acquiring a machine learning algorithm model corresponding to each evaluation item; S623: acquiring an evaluation result of each evaluation item corresponding to each group of sensor data; S624: labeling each group of sensor data according to each evaluation result; S625: training the machine learning algorithm model corresponding to each evaluation item through the labeled sensor data to obtain a support vector machine machine learning algorithm model trained by data.

5. The virtual reality technology-based simulated mine digital twin teaching method according to claim 1, characterized in that, The S63 includes: S631: acquiring a group of sensor data of the coal mining equipment as target sensor data; S632: acquiring a machine learning algorithm corresponding to an evaluation item as a target machine learning algorithm model; S633: acquiring a data class label corresponding to each target sensor data; S634: inputting the target sensor data and the corresponding data class label into the target machine learning algorithm model; S635: the target machine learning algorithm model outputs an evaluation result of the corresponding evaluation item according to the input target sensor data and data class label; S636: repeating steps S631 to S635 until all evaluation results corresponding to each group of sensor data are determined.

6. The virtual reality technology-based simulated mine digital twin teaching method according to claim 1, characterized in that, The method further includes: S71: when the evaluation result does not meet the preset requirements, sending an operation error prompt message and applying for reverse update authority; S72: when receiving the reverse update authorization message, acquiring a preset correct state of a virtual three-dimensional model of the coal mining equipment in the virtual mine; S73: acquiring an operation error of the coal mining equipment according to the preset correct state of the virtual three-dimensional model of the coal mining equipment and the current state of the virtual three-dimensional model; S74: updating the digital twin to the correct state according to the operation error; S75: controlling the coal mining equipment physical model to adjust to the correct state according to the state updated digital twin.

7. A simulated mine digital twin teaching system based on virtual reality technology, applying the teaching method of any one of claims 1 to 6, characterized in that, It includes a coal mining equipment physical model, a cloud server, and a digital twin updating module. The sensor is installed on the coal mining equipment physical model. The digital twin updating module is used to update the digital twin according to the data collected by the sensor. The digital twin updating module is in communication connection with the cloud server.

8. The virtual reality technology-based simulated mine digital twin teaching system according to claim 7, characterized in that, It also includes a VR glasses, which is in communication connection with the cloud server.

Images