Coal mine underground drilling machine training method and device based on VR

Abstract

The invention provides an underground coal mine drilling machine training method and device based on VR. The method comprises the following steps: acquiring working data of a coal mine; according to the working data, a drilling machine training scene of the coal mine is constructed by adopting a virtual reality technology, and the drilling machine training scene is a virtual VR scene used for simulating the working of a drilling machine; a drilling machine training scene is adopted to train and teach a target object, the drilling machine training scene and the target object interact through VR equipment, and the VR equipment comprises one or more of a VR helmet, VR glasses and a VR handle. The method solves the problems that traditional coal mine drilling machine training in the prior art depends on real objects and sites, the training place and time are limited, and variable training requirements cannot be flexibly met.

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and more specifically, to a VR-based underground coal mine drill training method, device, computer program product, and a VR-based underground coal mine drill virtual prototype assessment and training platform. Background Art

[0002] Coal production safety is the cornerstone of ensuring the stable supply of energy in my country, and underground drilling projects are an important technical means to liberate coal resources threatened by gas, water and other disasters, and are the fundamental guarantee for safe mining in mines. The proficiency of underground drillers in operating underground drilling rigs in coal mines and their grasp of safety awareness have a great impact on the safety of underground drilling projects and the guarantee of construction progress. Strengthening the proficiency and safety awareness of drillers in drilling rig operation has played an important role in reducing the occurrence of accidents caused by human factors in coal mine drilling rig operation, promoting safety production in coal mines, accelerating the progress of underground safe construction in coal mines, and ensuring the personal safety of coal mine workers.

[0003] Traditional training relies on physical objects and on-site activities, which limits the location and time of training and cannot flexibly adapt to changing training needs. Summary of the invention

[0004] The main purpose of this application is to provide a VR-based coal mine underground drilling rig training method, device, computer program product and VR-based coal mine underground drilling rig virtual prototype assessment and training platform, so as to at least solve the problem that traditional coal mine drilling rig training in the prior art relies on physical objects and on-site, which limits the location and time of training and cannot flexibly adapt to changing training needs.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a VR-based underground coal mine drilling rig training method is provided, comprising: obtaining working data of the coal mine, wherein the working data includes one or more of drilling rig operation data, coal mine geological data and drilling quality data, the drilling rig operation data includes one or more of the start time, stop time, speed and direction of the drilling rig, the coal mine geological data includes one or more of the rock layer hardness, coal seam thickness, gas content and groundwater level, and the drilling quality data includes one or more of the diameter, depth, straightness and hole wall integrity of the borehole; according to the working data, using virtual reality technology to construct a drilling rig training scene of the coal mine, wherein the drilling rig training scene is a virtual VR scene for simulating the working of the drilling rig; using the drilling rig training scene to train and teach the target object, wherein the drilling rig training scene and the target object interact through a VR device, and the VR device includes one or more of a VR helmet, VR glasses and a VR handle.

[0006] Optionally, based on the working data, virtual reality technology is used to construct a drilling rig training scene for the coal mine, including: based on the working data, using 3D modeling software to construct a virtual three-dimensional environment of the coal mine, wherein the 3D modeling software includes Unreal Engine and / or Unity; using virtual reality technology to integrate a virtual engine in the virtual three-dimensional environment to simulate real interaction to obtain the drilling rig training scene for the coal mine, wherein the virtual engine includes PhysX and / or Havok.

[0007] Optionally, the drilling rig training scenario is used to conduct training and teaching for a target object, including: displaying the drilling rig training scenario in the VR device; using a target prompting method to prompt guidance information for drilling rig operation in the VR device, wherein the target prompting method includes one or more of voice, text, and a virtual instructor; simulating the process of operating a drilling rig through the VR device so as to conduct operation training for the target object.

[0008] Optionally, before using the drilling rig training scenario to train and teach the target object, the method also includes: obtaining target courseware, wherein the target courseware is used to assess the target object, and the target courseware and the drilling rig training scenario correspond one to one; obtaining training operation data, wherein the training operation data is data of virtual drilling rig operation performed by the VR device after the target object selects the target courseware; and scoring the assessment content of the target object based on the training operation data.

[0009] Optionally, obtaining the target courseware includes: obtaining a first operation, wherein the first operation is an operation of randomly selecting a courseware and there are multiple coursewares; in response to the first operation, arbitrarily selecting one courseware from the multiple coursewares to obtain the target courseware; obtaining a second operation, wherein the second operation is an operation of specifying the courseware; in response to the second operation, searching for the specified courseware from the multiple coursewares to obtain the target courseware.

[0010] Optionally, after obtaining the target courseware, the method further includes: obtaining an emergency scenario, wherein the emergency scenario is a simulated emergency situation in the coal mine, and the emergency scenario includes one or more of gas leakage, sudden equipment failure, and tunnel collapse; obtaining a time limit, wherein the time limit is the limited time for the target object to complete the operation and / or problem of the emergency scenario; adding the emergency scenario and the time limit to the target courseware so that the target object can be assessed on emergency events.

[0011] Optionally, after the target object is trained and taught using the drilling rig training scenario, the method further includes: obtaining a first score, wherein the first score is a score obtained by assessing the target object using a scoring algorithm, wherein the scoring algorithm includes one or more of an SVM algorithm, a decision tree algorithm, a random forest algorithm, and a KNN algorithm; obtaining a second score, wherein the second score is a score obtained by manually assessing the target object; and calculating a weighted average of the first score and the second score to obtain a comprehensive score.

[0012] According to another aspect of the present application, a VR-based underground coal mine drilling rig training device is provided, comprising: a first acquisition unit, for acquiring working data of the coal mine, wherein the working data includes one or more of drilling rig operation data, coal mine geological data and drilling quality data, the drilling rig operation data includes one or more of the start time, stop time, speed and direction of the drilling rig, the coal mine geological data includes one or more of the rock layer hardness, coal seam thickness, gas content and groundwater level, and the drilling quality data includes one or more of the diameter, depth, straightness and hole wall integrity of the borehole; a construction unit, for constructing a drilling rig training scene of the coal mine according to the working data using virtual reality technology, wherein the drilling rig training scene is a virtual VR scene for simulating the working of the drilling rig; a training unit, for training and teaching a target object using the drilling rig training scene, wherein the drilling rig training scene and the target object interact through a VR device, and the VR device includes one or more of a VR helmet, VR glasses and a VR handle.

[0013] According to another aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of any one of the VR-based underground coal mine drilling rig training methods.

[0014] According to another aspect of the present application, a VR-based coal mine underground drilling rig virtual prototype assessment and training platform is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any one of the VR-based coal mine underground drilling rig training methods.

[0015] By applying the technical solution of the present application, a full-domain simulation of multiple scenes in a coal mine can be constructed through VR technology to achieve a high simulation of the underground environment, and then VR training and teaching can be carried out for the target objects. This way, there is no limit on the training space and time, and training can be carried out flexibly. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for executing a VR-based underground coal mine drill training method provided in an embodiment of the present application is shown;

[0018] Figure 2 A flow chart of a VR-based underground coal mine drill training method provided according to an embodiment of the present application is shown;

[0019] Figure 3 The schematic diagram of the structure of the virtual prototype assessment and training platform for underground drilling rigs in coal mines is shown;

[0020] Figure 4 The flowchart of the virtual prototype assessment and training method of underground coal mine drilling rigs based on VR is shown;

[0021] Figure 5 A structural block diagram of a VR-based underground coal mine drill training device provided according to an embodiment of the present application is shown.

[0022] The above drawings include the following reference numerals:

[0023] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As introduced in the background technology, traditional coal mine drilling rig training in the prior art relies on physical objects and on-site training, which limits the location and time of training and cannot flexibly adapt to changing training needs. To solve the above problems, the embodiments of the present application provide a VR-based coal mine underground drilling rig training method, device, computer program product and VR-based coal mine underground drilling rig virtual prototype assessment and training platform.

[0028] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal for a VR-based coal mine drilling rig training method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0030] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] In this embodiment, a VR-based coal mine drilling rig training method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0032] Figure 2 1 is a flow chart of a method for training a drill in a coal mine based on VR according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, obtaining the working data of the coal mine, wherein the working data includes one or more of drilling rig operation data, coal mine geological data and borehole quality data, the drilling rig operation data includes one or more of the start time, stop time, speed and direction of the drilling rig, the coal mine geological data includes one or more of the rock layer hardness, coal seam thickness, gas content and groundwater level, and the borehole quality data includes one or more of the diameter, depth, straightness and hole wall integrity of the borehole;

[0034] Specifically, the drilling rig operation data includes the operation information of the drilling rig starting, stopping, speed control, direction adjustment, drilling depth, drilling angle, etc. These data can reflect the worker's proficiency and accuracy in controlling the drilling rig.

[0035] Specifically, geological condition data simulates the drilling environment under different geological conditions, such as rock hardness, coal seam thickness, gas content, groundwater level, etc. These data are used to create realistic drilling scenarios and test workers' drilling skills and decision-making abilities under different geological conditions.

[0036] Specifically, the drilling quality data includes the diameter, depth, straightness, hole wall integrity, etc. These data are used to evaluate the quality control during the drilling process and the degree of compliance of workers with drilling standards.

[0037] Specifically, the above-mentioned working data also includes safety operation data, fault diagnosis and maintenance data, collaborative operation data, operation video and audio data, and environmental data.

[0038] Safety operation data records workers' compliance with safety regulations during operations, such as the use of personal protective equipment, compliance with operating procedures, response speed and handling methods for emergencies, etc. Safety data is a key indicator for assessing workers' safety awareness and emergency response capabilities.

[0039] Fault diagnosis and maintenance data simulates the occurrence of drilling rig failures and records the data of workers diagnosing faults, finding the root causes of problems, and selecting the correct maintenance tools and methods. These data are used to evaluate workers' analytical ability and maintenance skills when facing failures.

[0040] Collaborative operation data If the platform supports multi-person collaborative training, the communication, collaboration and coordination data between multiple people will be recorded, including the transmission of instructions, synchronization of operations, conflict resolution, etc.

[0041] Operation video and audio data record the worker's operation video and audio in the virtual environment for subsequent analysis, review and scoring. Video data can capture the worker's operation details, while audio data can record voice instructions and communication information during the operation.

[0042] Environmental data simulates the temperature, humidity, oxygen content, gas concentration and other environmental parameters underground, testing workers' adaptability and operational decisions under different environmental conditions.

[0043] Step S202, based on the working data, using virtual reality technology to construct the drilling rig training scene of the coal mine, wherein the drilling rig training scene is a virtual VR scene for simulating the working of the drilling rig;

[0044] Specifically, through virtual reality technology, information such as coal mine geological data, drilling rig operating parameters and drilling quality indicators can be transformed into an immersive and interactive 3D environment. This training scene not only includes the appearance and operating interface of the drilling rig, but also simulates different geological conditions and challenges that may be encountered in drilling construction, such as rock formation changes and gas leakage. In this way, trainees can experience and practice drilling rig operation without being restricted by the actual environment.

[0045] Step S203: Use the drilling rig training scenario to train and teach the target object, wherein the drilling rig training scenario and the target object interact through a VR device, and the VR device includes one or more of a VR helmet, VR glasses, and a VR handle.

[0046] Specifically, the target object (usually a coal miner or a drill operator) will interact with the training scenario through VR equipment (including but not limited to VR helmets, glasses and handles). Trainees can operate simulated drills in a virtual environment and deal with various geological and safety issues, while trainers or systems can monitor the trainees' performance in real time and provide instant feedback or guidance. This training method not only improves safety and repeatability, but also allows for detailed recording and analysis of the training process, so that the trainees' skill level and room for improvement can be accurately assessed.

[0047] Through this embodiment, VR technology is used to construct a full-domain simulation of multiple scenes in a coal mine underground, achieving a high simulation of the underground environment, and then VR training and teaching can be carried out for the target objects, so that there is no limit on the training space and time, and training can be carried out flexibly.

[0048] Specifically, the strong support of the state has enabled the rapid development of my country's education and training information industry. With the increasing development of the education and training industry, training modes such as smart classrooms, interactive classrooms, VR / AR classrooms, and holographic classrooms are also developing further. The application of VR technology in the industrial field is mainly concentrated in the petroleum, aerospace, and electric power industries, while the actual application of this VR classroom in coal safety production is relatively rare.

[0049] At present, with the development of the entire coal production safety system towards intensification and intelligence, the VR operation training and assessment system for underground coal mine drilling is also facing a major demand for visualization and diversification of training modes. The traditional underground coal mine drilling rig construction assessment training is restricted by space and time, and cannot meet the needs of multi-person collaborative operation, and cannot review drilling rig equipment failures and repairs. At the same time, there are also safety hazards at the assessment and training site.

[0050] To this end, in view of the above problems, the inventors of this application, through technical development and integrating many years of practical experience in related fields, have developed a VR-based coal mine underground drilling rig virtual prototype assessment and training platform and method to overcome the above difficulties.

[0051] In the specific implementation process, based on the above-mentioned working data, the drilling rig training scene of the above-mentioned coal mine is constructed by using virtual reality technology, which can be achieved through the following steps: based on the above-mentioned working data, a virtual three-dimensional environment of the above-mentioned coal mine is constructed by using 3D modeling software, wherein the above-mentioned 3D modeling software includes Unreal Engine and / or Unity; virtual reality technology is used to integrate a virtual engine in the above-mentioned virtual three-dimensional environment to simulate real interaction, and the above-mentioned drilling rig training scene of the above-mentioned coal mine is obtained, wherein the above-mentioned virtual engine includes PhysX and / or Havok.

[0052] In this solution, through the combined use of 3D modeling software and virtual engines, a virtual space that is highly similar to the real coal mine environment can be created. Not only is the appearance realistic, but the physical behavior is also close to reality, providing trainees with a more immersive training experience.

[0053] Specifically, the coal mine working data is transformed into a virtual three-dimensional training environment. First, a virtual three-dimensional environment of a coal mine is constructed using 3D modeling software such as Unreal Engine or Unity, which means that the working environment of the coal mine, including drilling rigs, underground tunnels, geological structures, etc., is reproduced in the computer in the form of a three-dimensional model. Unreal Engine and Unity are the industry's leading 3D development platforms, which provide a wealth of tools and resources to create highly realistic visual effects and complex physical simulations. Secondly, the physical interactivity of the virtual three-dimensional environment is enhanced by integrating virtual engines (such as PhysX or Havok), so that operations in the virtual environment can simulate the mechanical and dynamic behaviors in the real world. PhysX and Havok are physics engines widely used in game development and virtual reality applications. They can handle collision detection, gravity simulation, and dynamic response of complex objects, making the operation of the virtual drilling rig, the response of the rock formation, and the physical changes during the drilling process more realistic, improving the immersion and authenticity of the training.

[0054] Specifically, a drilling rig training scenario is constructed. Using 3D modeling software, a virtual environment simulating the underground coal mine is established based on the collected data, including tunnel structure, geological section, drilling rig model, ventilation system, lighting system, etc., to ensure the high similarity between the virtual scene and the real working environment. The physical engine is used to simulate the real drilling physical process, such as the friction between the drill bit and the rock, the vibration of the drilling rig, the diffusion of gas leakage, etc. At the same time, an interactive method is designed, such as controlling the start, stop, and direction adjustment of the drilling rig through the handle of the VR helmet, so that workers can interact with the virtual drilling rig as in actual operation. Various possible safety hazards and equipment failures are preset in the scenario, such as gas leakage, rock collapse, drilling rig overheating, hydraulic system leakage, etc., as well as standard operating procedures for dealing with these situations. Through random or self-selected methods, workers are allowed to encounter these scenarios during training to train their safety awareness and fault handling capabilities. The training content is divided into different modules, such as the drilling rig practical operation module, the drilling site hidden danger investigation module, and the drilling rig fault diagnosis and maintenance module. Each module contains specific virtual scenes and operation tasks. Workers can choose different modules for training according to their needs. During the worker's operation, the system will record the operation data in real time, such as operation time, operation steps, fault handling time, etc., and automatically score according to the pre-set scoring rules. At the same time, the system can provide instant operation feedback, point out errors in operation, and help workers improve. According to the worker's training progress and operation level, the difficulty and complexity of the training content are automatically adjusted to provide a personalized training experience. The system can identify the worker's weak links and provide additional training or review content in a targeted manner. If the platform supports multi-person collaborative operation training, workers can communicate in real time through the communication system in the virtual environment to simulate teamwork and emergency coordination scenarios. This helps to improve workers' communication and teamwork skills.

[0055] Specifically, 3D modeling software such as Unreal Engine and Unity are used to build a virtual environment underground in a coal mine, including tunnels, working faces, rock layers, drilling rig models, etc. Each model needs to restore the details of the real world as much as possible to provide a realistic operating experience.

[0056] Integrate physics engines (such as PhysX and Havok) in the virtual environment to simulate real physical interactions, such as the collision effect between the drill and the rock, the vibration during drilling, etc. At the same time, develop interaction logic to ensure that workers can interact with the virtual drill naturally and smoothly through the VR handle, such as starting, stopping, and adjusting the direction of the drill.

[0057] Integrate the designed virtual scenes and interaction logic into VR platforms, such as Oculus Rift, HTC Vive, or VR systems independently developed by enterprises. Ensure that the VR helmet can correctly display the virtual scenes and the controller input can be accurately recognized and responded to.

[0058] Scripts are written to implement specific training functions, such as automatically playing operation guides, setting up simulation tasks for fault diagnosis and emergency handling, and performance statistics and feedback systems. These scripts need to be tightly integrated with the virtual environment and interactive devices to ensure the accuracy and reliability of the functions.

[0059] In some embodiments, the drilling rig training scenario is used to train and teach the target object, which can be specifically achieved through the following steps: displaying the drilling rig training scenario in the VR device; using a target prompt method to prompt the guidance information of the drilling rig operation in the VR device, wherein the target prompt method includes one or more of voice, text and virtual instructor; simulating the process of operating the drilling rig through the VR device, so as to provide operation training for the target object.

[0060] In this solution, the training scene is directly displayed in the VR device, which can provide an immersive experience, allowing trainees to feel the tension and challenge of real operations in a virtual environment, thereby improving the authenticity and efficiency of the training. Through voice, text and virtual instructor prompts, personalized and differentiated guidance can be provided according to the training needs and comprehension abilities of different trainees, helping trainees to master operating skills and safety knowledge more quickly. By simulating the operation of the drilling rig in the VR device, the system can provide instant feedback on the trainees' operation results and scores, which helps trainees understand their own operating results, make timely adjustments and improvements, and ensure the maximum training effect.

[0061] Specifically, workers need to wear VR helmets and corresponding interactive devices (such as handles) before training. Through these devices, workers can "enter" the virtual scene and experience the same operating environment as in a real mine. In the virtual scene, workers can simulate operating a drilling rig to perform drilling operations. The system will provide operating guides, and workers can practice according to the guides, or explore freely and become familiar with the control and workflow of the drilling rig through repeated operations. Before the actual operation, the system can explain the basic operating theory, safety regulations and mine environment knowledge of the drilling rig to the workers through virtual instructors or text or voice prompts to ensure that the workers are fully prepared in theory before operation.

[0062] In the virtual scene, workers follow the operating instructions provided by the system to simulate the operation of the drilling rig for drilling. The system provides interactive methods such as handles or gesture recognition. Workers can control the start, stop, and direction adjustment of the virtual drilling rig, and experience the entire process from preparation to completion of drilling.

[0063] In addition to normal operations, VR scenarios also simulate common equipment failures and safety emergencies. Workers need to perform fault diagnosis and emergency handling in these scenarios, and enhance troubleshooting capabilities and safety awareness through practice. Every operation of the worker in the virtual scene will be recorded by the system and feedback will be given immediately. The system can automatically evaluate the correctness of the operation, point out errors, and help workers correct them. At the same time, through simulated assessments, workers can intuitively see their own operating level and progress. The VR platform can provide personalized training content based on the worker's skill level and training needs. This means that workers can focus on the skill points they need to strengthen and conduct targeted training.

[0064] Common VR (Virtual Reality) devices mainly include the following categories:

[0065] VR Head-Mounted Displays (HMD): These are the most typical VR devices, such as Oculus Rift, HTC Vive, PlayStation VR, Valve Index, Samsung Gear VR, etc. They usually consist of a helmet and a built-in display screen, which can provide users with an immersive visual experience. These helmets are usually equipped with built-in sensors that can track the user's head movement, so that the user's viewpoint in the virtual environment changes as the head moves.

[0066] VR handles or interactive devices: such as Oculus Touch, HTC Vive controllers, Valve Index controllers, etc. These devices provide users with a means to interact with the virtual environment and can simulate actions such as grabbing, pressing buttons, and manipulating virtual objects.

[0067] Peripheral trackers: used to track the movement of other parts of the user's body, such as Oculus sensors, HTC Vive base stations, etc., to ensure that the user's full body movement in the virtual space can be accurately identified and reflected.

[0068] VR motion platforms: Such as Cyberith Virtualizer, KAT Walk, Omni VR, etc. These devices allow users to perform unlimited virtual walking in a limited physical space, simulating the user's movement in the virtual environment through a rotating platform or pulley system.

[0069] Full body suits: Such as Teslasuit, Hardlight VR, etc. This type of equipment can provide full body tactile feedback, including force feedback, vibration feedback, etc., allowing users to feel more realistic physical interactions in virtual environments.

[0070] VR glasses and mobile phone adapters: such as Google Cardboard, Samsung Gear VR, etc. These devices are relatively simple. By placing the mobile phone in a special box, they use the mobile phone's display and accelerometer to provide a basic VR experience.

[0071] Wireless VR devices: such as the Oculus Quest series, HTC Vive Focus, etc. These devices do not require an external PC or game console to run, providing greater freedom of movement and portability.

[0072] Professional-grade VR devices: such as Varjo, StarVR, etc., which usually have higher resolution, more precise tracking system and more complex interactive devices, and can provide the ultimate virtual experience.

[0073] All-in-one VR devices: such as Oculus Quest, Pico Neo, Huawei VR Glass, etc. These devices integrate computing units, sensors, display screens, etc. into helmets. They do not need to be connected to external devices, providing a more convenient VR solution.

[0074] VR audio equipment: such as 3D sound headphones, which can provide surround sound and enhance the immersion of the virtual environment.

[0075] Specifically, workers wear VR helmets and interactive devices (such as VR handles). The helmet is used to display virtual scenes, while the handle is used as a tool to interact with the virtual environment. The system initializes the connection to ensure smooth communication between the VR device and the service platform. Once the device is connected, the worker selects the training module he wants to start through the handle or interactive interface. The system then loads the corresponding virtual scene, such as the tunnel environment under the coal mine, the drilling rig operation interface, etc., and the worker sees and immerses himself in this environment through the helmet. Before starting formal training, the system will guide the worker to calibrate the interactive device to ensure that the position and movement of the handle can be accurately reflected in the virtual scene to achieve a real operating experience. The system provides theoretical knowledge explanations to workers, such as the working principle, operation process, safety regulations, etc. of the drilling rig through voice guidance, text prompts or virtual lecturers. Subsequently, the worker uses the handle (or other interactive devices) to simulate the operation of the drilling rig in the virtual environment and conducts actual operation training. In the practical operation stage, the system will randomly or according to the course objectives add faults or emergency situations to the virtual scene, such as overheating of the drilling rig, gas leakage, etc. Workers need to use the handle for fault diagnosis and emergency treatment under guidance and learn the correct response strategies.

[0076] Specifically, first, the constructed drilling rig training scene is displayed in the VR device, which means integrating the virtual three-dimensional environment, physical simulation and interactive design into the visual and operable interface of the VR helmet and handle, so that the target object (i.e., the trainee) can be fully immersed in the simulated coal mine underground environment and operate the drilling rig just like in a real work site. Then, the trainee is provided with guidance information on drilling rig operation in the VR device through target prompts. The target prompt methods here include but are not limited to voice prompts, text prompts and virtual instructors. Voice prompts can guide trainees' operating steps and safety points in real time through the built-in audio system; text prompts are displayed in text form in the trainee's field of vision, such as above or on both sides of the operation interface, providing operating instructions and precautions; virtual instructors refer to creating a virtual character in a virtual environment, which can guide trainees to operate and answer questions in the form of teaching videos or interactive courses, thereby enhancing the interactivity and educational nature of the training.

[0077] Specifically, when the trainee puts on the VR helmet and starts the training software, the system will automatically load the drilling rig training scene. Trainees can control the view through the handle to preview the entire virtual coal mine environment, including tunnels, drilling rigs, geological structures and potential safety hazards. At the beginning of the training, the virtual instructor guides the trainees to perform basic operations on the drilling rig, such as starting, setting parameters, drilling, etc., through voice and text prompts. As the trainees' skills improve, the virtual instructor will gradually reduce direct guidance and switch to problem guidance to encourage trainees to think and solve problems independently. During the training, the system will randomly or according to plan simulate emergency situations such as gas leaks and rock formation anomalies. The virtual instructor will guide trainees on how to use the virtual handle to operate the drilling rig and take appropriate emergency measures, such as stopping drilling and starting the ventilation system. Trainees can improve their response capabilities by operating in simulated emergency situations.

[0078] During the specific implementation process, before using the above-mentioned drilling rig training scenario to train and teach the target object, the above-mentioned method also includes the following steps: obtaining target courseware, wherein the above-mentioned target courseware is used to assess the above-mentioned target object, and the above-mentioned target courseware and the above-mentioned drilling rig training scenario correspond one to one; obtaining training operation data, wherein the above-mentioned training operation data is data of the above-mentioned target object using the above-mentioned VR device to perform virtual drilling rig operations after selecting the above-mentioned target courseware; and scoring the assessment content of the above-mentioned target object based on the above-mentioned training operation data.

[0079] In this solution, by collecting and analyzing VR operation data, the assessment process can be based on the actual performance of the trainees rather than their subjective impressions, ensuring the accuracy and fairness of the scoring. VR equipment can record every operational detail of the trainees, including tiny hand movements and decision-making processes, which are crucial for evaluating the sophistication of operational skills. Data-based scoring rules can avoid interference from human factors and provide objective assessment results. At the same time, detailed VR operation data can be used as part of feedback to help trainees understand their strengths and areas for improvement.

[0080] Specifically, when the target object (usually the trainee being assessed) selects a specific assessment courseware, the system will collect all relevant data of the virtual drilling rig operation in the VR device, which is called training operation data. These data include all the actions of the trainee in operating the drilling rig in the virtual environment, such as the time to start the drilling rig, the speed of operating the drilling rig, direction adjustment, drilling depth control, reaction time and handling methods for sudden situations underground, etc.

[0081] Specifically, the system evaluates the trainees' operations in the target courseware according to the acquired training operation data and the preset scoring rules to obtain the assessment score. The scoring criteria can cover the accuracy, efficiency, safety and ability to deal with emergencies. In this way, the assessment process is not only comprehensive, but also can accurately reflect the trainees' actual operation level and emergency response capabilities, which improves the objectivity and fairness of the assessment.

[0082] Specifically, before the assessment begins, the system administrator or assessor selects a target courseware through the platform interface. The courseware contains specific assessment scenarios and standard operating procedures, which correspond to the pre-built drilling rig training scenarios. The courseware may involve skill points such as starting and stopping the drilling rig, drilling operations, fault diagnosis and maintenance, etc.

[0083] Specifically, trainees put on VR equipment, enter the selected assessment scene, and start operating the virtual drilling rig. The system automatically records the trainees' operation data, including but not limited to operation time, speed, direction adjustment, drilling depth and straightness control, response time and handling methods for underground emergencies, etc.

[0084] Specifically, after the assessment is completed, the system automatically calculates the trainee's assessment score based on the recorded training operation data and the preset scoring rules. The scoring rules may include weighting of operation accuracy, additional points for safe operation, assessment of emergency handling, etc. At the same time, the system also allows assessors to perform manual scoring to supplement the shortcomings of machine scoring and ensure the comprehensiveness and fairness of the scoring.

[0085] Specifically, the system not only gives the final assessment score, but also generates a detailed VR operation data analysis report, which contains every detail of the trainee's operation and a comparative analysis with the standard operating procedures. Based on this report, the assessor can provide the trainee with specific improvement suggestions or conduct in-depth discussions and analysis on the trainee's operation.

[0086] Specifically, the assessment results and detailed operation data are recorded in the system as a long-term reference for the trainees’ skill levels. These data can also be used in the formulation of subsequent skill improvement plans, or as a reference in the selection of talents and job assignments within the enterprise.

[0087] Specifically, after the assessment, students can re-enter the assessment scene, replay their own operation videos through VR devices, interact with virtual instructors, discuss the advantages and disadvantages of the operation, and deepen their understanding and memory of the operation process.

[0088] In some embodiments, obtaining the target courseware can be specifically achieved through the following steps: obtaining a first operation, wherein the first operation is an operation of randomly selecting a courseware, and there are multiple coursewares; in response to the first operation, arbitrarily selecting one courseware from the multiple coursewares to obtain the target courseware; obtaining a second operation, wherein the second operation is an operation of specifying the courseware; in response to the second operation, searching for the specified courseware from the multiple coursewares to obtain the target courseware.

[0089] In this solution, the operation of randomly selecting courseware provides a high degree of flexibility, ensures the diversity of assessment scenarios, avoids repetition and monotony in the training and assessment process, and improves the adaptability and ability of trainees to deal with different situations. The operation of specifying courseware allows trainers or assessors to select the most appropriate courseware for assessment or training based on the skill level and training needs of trainees, improves the pertinence and controllability of training, and ensures that trainees can get sufficient training on key skill points.

[0090] Specifically, the first operation - random selection of courseware: In this mode, the system will randomly select one of the preset courseware as the target courseware according to the user's instructions. These courseware may contain different underground coal mine scenes, drilling rig failure conditions or drilling tasks. Through random selection, the diversity of scenes for each assessment can be ensured, avoiding students from becoming too familiar with specific scenes too early, thereby improving the authenticity and challenge of the assessment.

[0091] Second operation - specifying courseware: In this mode, users can explicitly specify a specific courseware from multiple courses as the target courseware. This is usually used for intensive training or in-depth assessment of specific skill points. Users (may be assessors or trainers) can select the most appropriate courseware according to training needs or assessment plans to ensure the pertinence and effectiveness of training or assessment content.

[0092] Specifically, the examiner starts the assessment mode, and the system randomly selects a courseware from a courseware library containing 10 different scenarios. This random courseware may simulate a complex rock structure, an area with high gas risk, or a drilling task with unforeseen geological obstacles. The trainee puts on the VR device, enters the random scene, and conducts the operation assessment. After the assessment is completed, the system scores the trainee based on the operation data and provides a detailed feedback report.

[0093] Specifically, after the initial assessment, it was found that a trainee's skills in fault diagnosis and maintenance were relatively weak. The trainer decided to adopt the operation mode of the designated courseware and selected a courseware specifically for drilling rig fault diagnosis to allow the trainee to conduct special skills training. The trainee put on the VR device again, and the system directly entered the designated fault diagnosis courseware. The trainee operated the drilling rig in the virtual environment to identify and repair the fault. During the training process, the system recorded all operating data, including the speed of fault identification and the accuracy of maintenance steps, for subsequent analysis and feedback.

[0094] Specifically, after a period of training, the company decided to conduct a comprehensive assessment for all trainees. The assessors first used randomly selected courseware operations to ensure the diversity of assessment scenarios and test the comprehensive skills of the trainees. Subsequently, based on the weaknesses shown by the trainees in the preliminary assessment, the designated courseware operations were used to conduct targeted assessments on the trainees, such as operation accuracy, safe operation specifications, and fault response capabilities. Finally, the system combined the results of the two assessments to give each trainee a comprehensive score and skill assessment report.

[0095] In the specific implementation process, after obtaining the target courseware, the above method also includes the following steps: obtaining an emergency scenario, wherein the above emergency scenario is a simulated emergency situation in the above coal mine, and the above emergency scenario includes one or more of gas leakage, sudden equipment failure, and tunnel collapse; obtaining a time limit, wherein the above time limit is the limited time for the above target object to complete the operation and / or problem of the above emergency scenario; adding the above emergency scenario and the above time limit to the above target courseware, so as to conduct an emergency incident assessment on the above target object.

[0096] In this program, by simulating emergency scenarios, trainees can experience and practice decision-making and operations under high pressure in a VR environment, effectively improving their emergency response capabilities and psychological endurance in real emergency situations. The time limit setting increases the sense of urgency of the assessment, making the assessment closer to the response requirements of the actual working environment, and improving the authenticity and challenge of the assessment, so as to more accurately evaluate the trainees' actual operation level.

[0097] Specifically, the emergency self-selection mechanism allows workers to encounter simulated emergency situations during the assessment process. These situations are preset based on the actual safety issues and equipment failures that may be encountered in coal mines. Emergency situations may include but are not limited to gas leaks, sudden equipment failures, tunnel collapses, etc. Before or during the assessment, the system will randomly or by the worker's choice decide whether to join the simulation of these emergency situations. When an emergency occurs, workers need to quickly identify problems, assess risks, and take correct countermeasures in a virtual environment, thereby testing and improving their reaction speed, decision-making ability, and emergency response skills under pressure.

[0098] Specifically, time-limited operation is a sub-mode under the emergency self-selection. After an emergency occurs, the system will set a time limit and require workers to complete specific operations or solve problems within the specified time. For example, in the case of a simulated gas leak, the system may set a time limit and require workers to close the leak source, start the ventilation system, or evacuate safely within a limited time. The purpose of time-limited operation is to simulate the time pressure in a real working environment, evaluate the efficiency and accuracy of workers' operations in urgent situations, and improve their time management ability in emergency situations.

[0099] Specifically, the system obtains emergency scenarios from the preset emergency database, which can be common emergency situations in coal mines, such as gas leakage, sudden equipment failure, tunnel collapse, etc. By simulating these scenarios, the system can test the trainees' reaction speed and handling ability when facing various emergency events that may occur in a real working environment.

[0100] Specifically, the system sets a time limit for each emergency scenario, which is the time limit for students to complete the operation or solve the problem. The time limit is set according to the severity and complexity of the emergency, as well as the industry standard operation time, in order to simulate the time pressure of real emergency response and improve the authenticity and urgency of the assessment.

[0101] Specifically, in the selected target courseware, the system integrates emergency scenarios and time limits as part of the assessment. This not only increases the difficulty and challenge of the assessment, but also makes the assessment content closer to the real working environment, helping trainees learn and practice emergency response skills in a virtual environment.

[0102] Specifically, the system administrator can select two emergency scenarios, "gas leakage" and "tunnel collapse", from the system and prepare to add them to the assessment courseware. These scenarios are designed based on real emergency situations in coal mines, and include detailed gas leakage source location, ventilation system startup, personnel evacuation and other operational requirements, as well as rapid detection after tunnel collapse, support reinforcement, escape route selection and other emergency response processes.

[0103] Specifically, according to the coal mine safety operation specifications, the system administrator set time limits for the two scenarios of "gas leakage" and "tunnel collapse", the former is 10 minutes and the latter is 15 minutes. These two time limits are based on industry standards and expert evaluations to ensure the fairness and challenge of the assessment.

[0104] Specifically, the system adds the selected emergency scenarios and time limits to the target assessment courseware, which includes basic drilling rig operation tasks and now also adds simulated emergency assessments. The courseware is presented to trainees through VR devices, and trainees need to complete the drilling rig operation tasks within the time limit and handle the emergency situations accurately and quickly.

[0105] Specifically, trainees put on VR equipment and enter the assessment scene. The system starts timing, and trainees operate the drilling rig in the virtual environment while dealing with emergencies such as "gas leakage" and "tunnel collapse". The system records the trainees' operation data, including the time to deal with the emergency, the decision-making process, and the accuracy of the operation.

[0106] Specifically, after the assessment, the system administrator and trainer will analyze the trainees' response speed, operation efficiency and problem-solving ability in emergency situations by reviewing their operation data. For trainees who respond quickly and operate accurately, positive feedback and encouragement will be given; for trainees who handle improperly or delay operations, specific improvement suggestions and retraining plans will be provided.

[0107] In some embodiments, after the target object is trained and taught using the drilling rig training scenario, the method further includes the following steps: obtaining a first score, wherein the first score is a score obtained by assessing the target object using a scoring algorithm, wherein the scoring algorithm includes one or more of an SVM algorithm, a decision tree algorithm, a random forest algorithm, and a KNN algorithm; obtaining a second score, wherein the second score is a score obtained by manually assessing the target object; and calculating a weighted average of the first score and the second score to obtain a comprehensive score.

[0108] In this solution, by using an automatic scoring algorithm, subjective bias in the scoring process can be reduced, the objectivity and fairness of scoring can be improved, and scoring can be more dependent on the actual operation data of the trainees. The addition of manual scoring makes up for the shortcomings of automatic scoring in capturing operation details and evaluating non-quantitative factors, so that the scoring can fully cover the operation ability and comprehensive quality of the trainees, thus being more detailed and accurate. The calculation of the comprehensive score not only provides quantitative feedback on the trainees' skill level, but also provides specific operation suggestions and improvement directions through manual scoring, which helps trainees improve their skills in a targeted manner.

[0109] Specifically, the collected data is automatically evaluated using a built-in scoring algorithm. For example, if the operation steps are consistent with the standard operating procedures, the system will automatically give a higher score; if the operation time is longer than the standard time, the score will be reduced accordingly. The score for emergency handling is based on the speed at which the worker identifies the problem, whether the measures taken are correct, and whether they comply with safety regulations. This score is usually objective and based on specific data of the operation.

[0110] In addition to machine scoring, the platform also allows for human scoring. Human scorers can be experienced rig operators, safety experts, or training instructors. They make subjective evaluations of workers' operating skills, judgment, and response strategies by watching operation videos or on-site observations (achieved through other means in a virtual environment). Human scorers can evaluate aspects such as the smoothness of the operation, technical details, and safety awareness, which are difficult to accurately evaluate with machine scoring.

[0111] In order to balance the objectivity of machine scoring and the subjectivity of manual scoring, the platform can be set to use a 6:4 scoring ratio for comprehensive evaluation. This means that the machine score accounts for 60% of the total score, while the manual score accounts for 40%. This scoring method ensures the comprehensiveness and fairness of the scoring, taking into account both the actual effect of the operation, the details of the operation and the quality of the operator.

[0112] After the assessment is completed, the system will automatically calculate the final score, i.e. the comprehensive score, and show it to the worker. The score report will include an operation video and a detailed score analysis, pointing out the advantages and disadvantages of the operation and providing suggestions for improvement. Workers can use this to understand their own operation level and clarify the direction of improvement.

[0113] Specifically, the first score is the result of automatic assessment of the target object using a scoring algorithm. The scoring algorithms mentioned here include support vector machine (SVM), decision tree algorithm, random forest algorithm and K nearest neighbor algorithm (KNN), which can automatically calculate a score based on the data of trainees operating the drilling rig in the VR environment, such as operation time, operation accuracy, compliance with safety regulations, etc., to reflect the trainees' operation ability and level of emergency response. The second score is the score manually given by the manual scorer based on the trainees' performance. Manual scoring can take into account some details that the automatic scoring algorithm cannot capture, such as the consistency of operation, the rationality of decision-making, and the ability to handle unstructured problems, providing a more comprehensive and humane perspective for scoring. Finally, the system calculates the weighted average of the first score and the second score to obtain a comprehensive score. The weight distribution can be adjusted according to specific needs. For example, in scenarios that emphasize the objectivity of operation data, the automatic scoring algorithm (first score) can be given a higher weight; while in scenarios that focus on the operation process and decision logic, the manual score (second score) can be given a higher weight.

[0114] For example, the system first uses the random forest algorithm to analyze the trainee's operation data, including operation efficiency, operation accuracy, equipment fault diagnosis speed and accuracy, etc., and automatically gives a score based on the operation data. Assume that the automatic score is 85 points (out of 100 points).

[0115] At the same time, the on-site trainer or assessor manually gives a score based on the trainee's operation process, decision-making logic and performance in dealing with emergencies. Considering the trainee's quick response and teamwork ability in the face of the "tunnel collapse" emergency, the manual score is 90 points.

[0116] Next, the system calculates the weighted average of the first score (automatic score) and the second score (manual score) according to the established scoring rules. Assuming that the weights of automatic score and manual score are 60% and 40% respectively, the comprehensive score is calculated as follows: Comprehensive score = (85*0.6) + (90*0.4) = 51 + 36 = 87 points.

[0117] Finally, the system shows the trainee a comprehensive score and provides a detailed automatic score analysis report and manual score feedback. The automatic score analysis report points out that the trainee performed well in terms of operation accuracy, but was slightly slow in equipment fault diagnosis speed. The manual score feedback emphasized the trainee's quick response ability and teamwork spirit in emergency situations, but also pointed out that more caution could be exercised at certain decision points.

[0118] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the VR-based coal mine drilling rig training method of the present application will be described in detail below in combination with specific embodiments.

[0119] At present, the assessment and training of underground drilling rig construction in coal mines realizes the assessment and training of technical jobs through physical operation, simulated tunnels, assessment and scoring, etc. However, the assessment and training cycle is fixed and inflexible, and the existence of human factors leads to unfair competition and the authenticity of the data is questionable and inaccurate, which is not conducive to the development model of the assessment and training of underground drilling rig construction in coal mines. At the same time, it is also unable to truly bring into play the significance and value of the assessment and training, and will cause potential safety accidents to the later actual underground drilling construction in coal mines.

[0120] The present embodiment relates to a specific coal mine underground drilling rig virtual prototype assessment and training platform, including: a service control module, an assessment module, a training module, and a VR helmet.

[0121] The above-mentioned service control module is used to control the assessment module and the training module, mainly including reading personnel information data, receiving performance data, and receiving operation videos. The above-mentioned assessment module is used for the assessment of underground drilling rig construction operations in coal mines, mainly providing construction scenes, drilling rig construction simulation, performance statistics, and operation video recording. The training module is used for underground drilling rig construction operation training in coal mines, mainly providing construction scenes, drilling rig construction simulation, performance statistics, operation video recording, and text and voice prompts for the operation process. The above-mentioned VR helmet accesses the platform via a wireless network, mainly displaying panoramic videos and providing handle interaction.

[0122] like Figure 3 As shown, the virtual prototype assessment and training platform for underground coal mine drilling rigs includes a service control system, which includes:

[0123] Scenario simulation unit, used to build a global simulation system for multiple scenarios in coal mines;

[0124] The VR unit receives the scene global simulation data information formed by the scene simulation unit and generates a virtual image;

[0125] The training unit conducts virtual teaching based on the virtual images of the VR unit;

[0126] The assessment unit assesses the testee based on the virtual image of the VR unit;

[0127] The scoring module receives the assessment data from the assessment unit and makes a score based on the assessment data;

[0128] The above-mentioned assessment unit includes a random selection module for randomly selecting one or more virtual scenes generated by the VR unit as the assessment scene, a self-selection module for adding emergency situations that may involve hazardous accidents in actual situations to the selected assessment scenes, and a common sense operation module for common sense questions that may not have been taught during the training process. This allows the assessment scene to be closer to the actual operation scene in the later stage, improves the assessment quality, and is also beneficial to the safety of the test takers in their actual on-the-job operations in the later stage.

[0129] The above-mentioned random selection module refers to the courseware randomly generated after entering the assessment courseware, the above-mentioned self-selection module refers to the module for setting the assessment courseware program and selecting the courseware with VR wearable devices, and the above-mentioned common sense operation module refers to some routine assessment point operations that simulate underground construction. The priority of the above-mentioned random selection module is higher than that of the emergency self-selection module, and the priority of the above-mentioned common sense operation module is lower than that of the emergency self-selection module.

[0130] The above-mentioned emergency self-selection module is to enter the above-mentioned self-selection module content and click again on the emergency state simulation module or the time-limited operation module. The above-mentioned emergency state simulation module mode refers to simulating multiple emergency situations again in the determined virtual scene, and the time-limited operation module mode refers to limiting the emergency situation and conducting operation training and assessment for students within a limited time.

[0131] The above-mentioned VR unit includes a communication module that is wirelessly and / or wiredly connected to the service control system, an execution module for generating corresponding virtual scenes according to training or assessment requests, and a screen rendering module for generating virtual scenes, wherein the communication module is mainly effectively connected by wireless WIFI connection and wired USB, DP, and HDMI connection cables, so that the courseware, wearable devices and platforms can interact in real time; the execution module mainly refers to that after the trainees select the training and assessment VR courseware, the virtual scene of the selected courseware will be automatically generated in the VR wearable device for real-time interaction; the screen rendering module refers to the real-time rendering and generation of virtual scene pictures in the VR wearable device.

[0132] The above-mentioned scenario simulation unit includes a drilling rig operation module, a drilling field hidden danger investigation module, and a drilling rig fault diagnosis and maintenance module for constructing coal mine underground drilling rig operation, drilling rig fault diagnosis and maintenance, and drilling field hidden danger investigation. The above-mentioned drilling rig operation module simulates the operation steps of an underground coal mine drilling rig, and conducts training and assessment step by step according to the actual drilling rig operation steps; the above-mentioned drilling field hidden danger investigation module refers to the simulation of designated or randomly generated preset hidden danger points in the underground drilling field, and then the trainees conduct investigation training and assessment on the hidden danger points in the simulated drilling field environment; the above-mentioned drilling rig fault diagnosis and maintenance module refers to the simulation of the vulnerable points of underground coal mine drilling rigs and the steps of replacing conventional accessories, so that the trainees are familiar with the maintenance and assembly steps of underground coal mine drilling rigs, and the framework simulated by the above-mentioned scenario simulation unit is transmitted to the VR unit.

[0133] The above-mentioned training unit includes a duration module for controlling the time of each training session, and a scene switching combination module for combining and switching multiple scenes. The above-mentioned duration module refers to the automatic statistics and control of the duration of each courseware for each training of trainees; the above-mentioned switching combination module means that trainees can click on multiple scenes to switch through VR wearable devices, and then enter the scene for training.

[0134] The above-mentioned scoring modules include a manual scoring module that uses manual scoring and a machine scoring module that uses machine scoring, wherein the scoring ratio of the machine scoring module to the manual scoring module is 6:4. The above-mentioned manual scoring module refers to the manual scoring input of the trainees' training assessment and evaluation by the listeners in the platform; the above-mentioned machine scoring module refers to the assessment courseware that has already made scoring judgments for each step, and assesses and scores the trainees based on the mechanism of scoring correctly and not scoring incorrectly.

[0135] The service control system is also connected to a backup unit, which is connected to the emergency self-selection module and is used to record the emergencies that occur and exist in various actual scenarios.

[0136] This embodiment relates to a specific VR-based coal mine underground drilling rig virtual prototype assessment and training method, including an assessment and training platform, such as Figure 4 As shown, the specific steps include:

[0137] Step 1: The scene simulation unit first inputs the scene data that needs to be trained and assessed, and constructs a virtual scene framework, and then transmits the virtual scene framework to the VR unit, and the VR unit constructs a complete virtual scene based on the virtual scene framework;

[0138] Step 2: Conduct training based on virtual scenarios first, followed by assessment;

[0139] Step 3: After selecting one or more virtual scenes for the assessment, wear the VR device to conduct the assessment. During the assessment, add emergencies that occur in the scene imported by the backup unit, and record the time when the subject is in the emergencies;

[0140] Step 4: Improve scoring accuracy through machine and manual scoring.

[0141] The main technical effects of the above scheme are: ① The construction of a VR-based coal mine underground drilling rig virtual prototype assessment and training platform can realize the full-domain simulation of multiple scenes in coal mines; ② It provides a variety of assessment and training modes, including the actual operation of coal mine underground drilling rigs, drilling rig fault diagnosis and maintenance, and drilling site hidden danger inspection; ③ The platform effectively solves the problems of fixed space and time of the assessment and training venues, safety hazards and data authenticity, which is conducive to ensuring the authenticity and reliability of the assessment and training data, and is also conducive to improving the quality of assessment and training.

[0142] The embodiment of the present application also provides a VR-based underground coal mine drill training device. It should be noted that the VR-based underground coal mine drill training device of the embodiment of the present application can be used to execute the VR-based underground coal mine drill training method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been explained will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0143] The following is an introduction to the VR-based underground coal mine drilling rig training device provided in an embodiment of the present application.

[0144] Figure 5 1 is a block diagram of a VR-based underground coal mine drill training device according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0145] The first acquisition unit 10 is used to acquire the working data of the coal mine, wherein the working data includes one or more of drilling rig operation data, coal mine geological data and borehole quality data, the drilling rig operation data includes one or more of the start time, stop time, speed and direction of the drilling rig, the coal mine geological data includes one or more of the rock layer hardness, coal seam thickness, gas content and groundwater level, and the borehole quality data includes one or more of the diameter, depth, straightness and hole wall integrity of the borehole;

[0146] A construction unit 20 is used to construct the drilling rig training scene of the coal mine by using virtual reality technology according to the working data, wherein the drilling rig training scene is a virtual VR scene for simulating the working of the drilling rig;

[0147] The training unit 30 is used to use the above-mentioned drilling rig training scene to train and teach the target object, wherein the above-mentioned drilling rig training scene and the above-mentioned target object interact through a VR device, and the above-mentioned VR device includes one or more of a VR helmet, VR glasses and a VR handle.

[0148] Through this embodiment, VR technology is used to construct a full-domain simulation of multiple scenes in a coal mine underground, achieving a high simulation of the underground environment, and then VR training and teaching can be carried out for the target objects, which does not limit the training space and time and allows for flexible training.

[0149] In the specific implementation process, the construction unit includes a construction module and an engine module. The construction module is used to construct a virtual three-dimensional environment of the above-mentioned coal mine according to the above-mentioned working data using 3D modeling software, wherein the above-mentioned 3D modeling software includes UnrealEngine and / or Unity; the engine module is used to integrate a virtual engine in the above-mentioned virtual three-dimensional environment using virtual reality technology to simulate real interaction and obtain the above-mentioned drilling rig training scene of the above-mentioned coal mine, wherein the above-mentioned virtual engine includes PhysX and / or Havok.

[0150] In this solution, through the combined use of 3D modeling software and virtual engines, a virtual space that is highly similar to the real coal mine environment can be created. Not only is the appearance realistic, but the physical behavior is also close to reality, providing trainees with a more immersive training experience.

[0151] In some embodiments, the training unit includes a display module, a prompt module and a simulation module. The display module is used to display the above-mentioned drilling rig training scene in the above-mentioned VR device; the prompt module is used to use a target prompt method to prompt the guidance information of the drilling rig operation in the above-mentioned VR device, wherein the above-mentioned target prompt method includes one or more of voice, text and virtual instructor; the simulation module is used to simulate the process of operating the drilling rig through the above-mentioned VR device, so as to perform operation training for the above-mentioned target object.

[0152] In this solution, the training scene is directly displayed in the VR device, which can provide an immersive experience, allowing trainees to feel the tension and challenge of real operations in a virtual environment, thereby improving the authenticity and efficiency of the training. Through voice, text and virtual instructor prompts, personalized and differentiated guidance can be provided according to the training needs and comprehension abilities of different trainees, helping trainees to master operating skills and safety knowledge more quickly. By simulating the operation of the drilling rig in the VR device, the system can provide instant feedback on the trainees' operation results and scores, which helps trainees understand their own operating results, make timely adjustments and improvements, and ensure the maximum training effect.

[0153] During the specific implementation process, the above-mentioned device also includes a second acquisition unit, a third acquisition unit and a scoring unit. The second acquisition unit is used to acquire target courseware before using the above-mentioned drilling rig training scenario to train and teach the target object, wherein the above-mentioned target courseware is used to assess the above-mentioned target object, and the above-mentioned target courseware and the above-mentioned drilling rig training scenario correspond one to one; the third acquisition unit is used to acquire training operation data, wherein the above-mentioned training operation data is data of virtual drilling rig operation performed by the above-mentioned VR device after the above-mentioned target object selects the above-mentioned target courseware; the scoring unit is used to score the assessment content of the above-mentioned target object according to the above-mentioned training operation data.

[0154] In this solution, by collecting and analyzing VR operation data, the assessment process can be based on the actual performance of the trainees rather than their subjective impressions, ensuring the accuracy and fairness of the scoring. VR equipment can record every operational detail of the trainees, including tiny hand movements and decision-making processes, which are crucial for evaluating the sophistication of operational skills. Data-based scoring rules can avoid interference from human factors and provide objective assessment results. At the same time, detailed VR operation data can be used as part of feedback to help trainees understand their strengths and areas for improvement.

[0155] In some embodiments, the second acquisition unit includes a first acquisition module, a first response module, a second acquisition module and a second response module, the first acquisition module is used to acquire a first operation, wherein the first operation is an operation of randomly selecting a courseware, and there are multiple coursewares; the first response module is used to respond to the first operation, select any one of the coursewares from the multiple coursewares, and obtain the target courseware; the second acquisition module is used to acquire a second operation, wherein the second operation is an operation of specifying the courseware; the second response module is used to respond to the second operation, search for the specified courseware from the multiple coursewares, and obtain the target courseware.

[0156] In this solution, the operation of randomly selecting courseware provides a high degree of flexibility, ensures the diversity of assessment scenarios, avoids repetition and monotony in the training and assessment process, and improves the adaptability and ability of trainees to deal with different situations. The operation of specifying courseware allows trainers or assessors to select the most appropriate courseware for assessment or training based on the skill level and training needs of trainees, improves the pertinence and controllability of training, and ensures that trainees can get sufficient training on key skill points.

[0157] During the specific implementation process, the above-mentioned device also includes a fourth acquisition unit, a fifth acquisition unit and a processing unit. The fourth acquisition unit is used to acquire an emergency scene after acquiring the target courseware, wherein the above-mentioned emergency scene is a simulated emergency situation in the above-mentioned coal mine, and the above-mentioned emergency scene includes one or more of gas leakage, sudden equipment failure, and tunnel collapse; the fifth acquisition unit is used to acquire a time limit, wherein the above-mentioned time limit is the limited time for the above-mentioned target object to complete the operation and / or problem of the above-mentioned emergency scene; the processing unit is used to add the above-mentioned emergency scene and the above-mentioned time limit to the above-mentioned target courseware, so as to conduct an emergency incident assessment on the above-mentioned target object.

[0158] In this program, by simulating emergency scenarios, trainees can experience and practice decision-making and operations under high pressure in a VR environment, effectively improving their emergency response capabilities and psychological endurance in real emergency situations. The time limit setting increases the sense of urgency of the assessment, making the assessment closer to the response requirements of the actual working environment, and improving the authenticity and challenge of the assessment, so as to more accurately evaluate the trainees' actual operation level.

[0159] In some embodiments, the device further includes a sixth acquisition unit, a seventh acquisition unit and a calculation unit, wherein the sixth acquisition unit is used to obtain a first score after the target object is trained and taught using the drilling rig training scenario, wherein the first score is a score obtained by assessing the target object using a scoring algorithm, wherein the scoring algorithm includes one or more of an SVM algorithm, a decision tree algorithm, a random forest algorithm, and a KNN algorithm; the seventh acquisition unit is used to obtain a second score, wherein the second score is a score obtained by manually assessing the target object; and the calculation unit is used to calculate a weighted average of the first score and the second score to obtain a comprehensive score.

[0160] In this solution, by using an automatic scoring algorithm, subjective bias in the scoring process can be reduced, the objectivity and fairness of scoring can be improved, and scoring can be more dependent on the actual operation data of the trainees. The addition of manual scoring makes up for the shortcomings of automatic scoring in capturing operation details and evaluating non-quantitative factors, so that the scoring can fully cover the operation ability and comprehensive quality of the trainees, thus being more detailed and accurate. The calculation of the comprehensive score not only provides quantitative feedback on the trainees' skill level, but also provides specific operation suggestions and improvement directions through manual scoring, which helps trainees improve their skills in a targeted manner.

[0161] The VR-based underground coal mine drilling rig training device includes a processor and a memory. The first acquisition unit, the construction unit, the training unit, etc. are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0162] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problem that the traditional coal mine drilling rig training in the prior art relies on the physical object and the site, which limits the location and time of the training and cannot flexibly adapt to the changing training needs.

[0163] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0164] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is running, the device where the computer-readable storage medium is located is controlled to execute the VR-based underground coal mine drilling rig training method.

[0165] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the VR-based underground coal mine drilling rig training method when running.

[0166] The embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the steps of a drill training method for a coal mine based on VR are implemented. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0167] A computer program product includes a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the VR-based underground coal mine drilling rig training method in each embodiment of the present application are implemented.

[0168] The present application provides a VR-based coal mine underground drilling rig virtual prototype assessment and training platform, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the above-mentioned memory and are configured to be executed by the above-mentioned one or more processors, and the above-mentioned one or more programs include a method for executing any one of the above-mentioned VR-based coal mine underground drilling rig training methods.

[0169] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0176] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0178] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A VR-based underground coal mine drilling rig training method, characterized in that: include: Acquire working data of the coal mine, wherein the working data includes one or more of drilling rig operation data, coal mine geological data and borehole quality data, the drilling rig operation data includes one or more of the start time, stop time, speed and direction of the drilling rig, the coal mine geological data includes one or more of the rock layer hardness, coal seam thickness, gas content and groundwater level, and the borehole quality data includes one or more of the diameter, depth, straightness and hole wall integrity of the borehole; According to the working data, a drilling rig training scene of the coal mine is constructed by using virtual reality technology, wherein the drilling rig training scene is a virtual VR scene for simulating the working of the drilling rig; The drilling rig training scenario is used to perform training and teaching on a target object, wherein the drilling rig training scenario and the target object interact through a VR device, and the VR device includes one or more of a VR helmet, VR glasses, and a VR handle.

2. The method according to claim 1, characterized in that Based on the working data, virtual reality technology is used to construct the drilling rig training scene of the coal mine, including: According to the working data, a virtual three-dimensional environment of the coal mine is constructed using 3D modeling software, wherein the 3D modeling software includes Unreal Engine and / or Unity; A virtual engine is integrated into the virtual three-dimensional environment by using virtual reality technology to simulate real interaction and obtain the drilling rig training scene of the coal mine, wherein the virtual engine includes PhysX and / or Havok.

3. The method according to claim 1, characterized in that The drilling rig training scenario is used to conduct training and teaching for the target object, including: Displaying the drilling rig training scene in the VR device; Prompting the guidance information of the drilling rig operation in the VR device by using a target prompting method, wherein the target prompting method includes one or more of voice, text and virtual instructor; The process of operating the drilling rig is simulated by the VR device so as to provide operation training for the target object.

4. The method according to claim 1, characterized in that Before using the drilling rig training scenario to train the target object, the method further includes: Obtaining a target courseware, wherein the target courseware is used to assess the target object, and the target courseware corresponds to the drilling rig training scenario one by one; Acquiring training operation data, wherein the training operation data is data of the target subject performing a virtual drilling rig operation using the VR device after selecting the target courseware; The assessment content of the target object is scored according to the training operation data.

5. The method according to claim 4, characterized in that Obtain target courseware, including: Obtaining a first operation, wherein the first operation is an operation of randomly selecting a courseware, and there are multiple coursewares; In response to the first operation, arbitrarily selecting one of the courseware from the plurality of courseware to obtain the target courseware; Obtaining a second operation, wherein the second operation is an operation for specifying the courseware; In response to the second operation, the designated courseware is searched from the plurality of coursewares to obtain the target courseware.

6. The method according to claim 4, characterized in that After obtaining the target courseware, the method further includes: Acquire an emergency scenario, wherein the emergency scenario is a simulated emergency situation of the coal mine, and the emergency scenario includes one or more of gas leakage, sudden equipment failure, and tunnel collapse; Obtaining a time limit, wherein the time limit is a limited duration for the target object to complete the operation and / or problem of the emergency scenario; The emergency scene and the time limit are added to the target courseware so that the target object can be assessed on the emergency event.

7. The method according to any one of claims 1 to 6, characterized in that: After the target object is trained and taught using the drilling rig training scenario, the method further includes: Obtaining a first score, wherein the first score is a score obtained by using a scoring algorithm to assess the target object, wherein the scoring algorithm includes one or more of a SVM algorithm, a decision tree algorithm, a random forest algorithm, and a KNN algorithm; Obtaining a second score, wherein the second score is a score manually assessed on the target object; A weighted average of the first score and the second score is calculated to obtain a comprehensive score.

8. A VR-based underground coal mine drilling rig training device, characterized in that: include: a first acquisition unit, configured to acquire working data of a coal mine, wherein the working data includes one or more of drilling rig operation data, coal mine geological data, and borehole quality data; the drilling rig operation data includes one or more of the start time, stop time, speed, and direction of the drilling rig; the coal mine geological data includes one or more of the hardness of the rock layer, the thickness of the coal layer, the gas content, and the groundwater level; and the borehole quality data includes one or more of the diameter, depth, straightness, and hole wall integrity of the borehole; A construction unit, configured to construct a drill rig training scene of the coal mine using virtual reality technology according to the working data, wherein the drill rig training scene is a virtual VR scene for simulating the working of the drill rig; A training unit is used to use the drilling rig training scene to train and teach the target object, wherein the drilling rig training scene and the target object interact through a VR device, and the VR device includes one or more of a VR helmet, VR glasses and a VR handle.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the VR-based underground coal mine drill training method described in any one of claims 1 to 7 are implemented.

10. A VR-based coal mine underground drilling rig virtual prototype assessment and training platform, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing the VR-based coal mine drilling training method as described in any one of claims 1 to 7.

Images