Training method, device, equipment and product for mine virtual production line simulation training

Abstract

The invention discloses a training method, device, equipment and product for mine virtual production line simulation training, and relates to the technical field of skill talents, the method is applied to a digital mine training system, the digital mine training system comprises a data acquisition controller, a data processing module and a data processing module, the method comprises the following steps: receiving a training starting instruction of a user; responding to the training starting instruction through the data acquisition controller, and providing a digital mine training task for the user; and acquiring operation information when the user executes the digital mine training task, and evaluating the operation information in real time through a preset evaluation model to obtain an evaluation result. Therefore, the problems that the safety problem exists when the to-be-on-duty personnel are trained through a physical mine, and the training effect is poor due to the fact that no systematic score exists are solved, and the training effect is improved.

Description

Technical Field

[0001] The present application relates to the field of talent training technology, and in particular to a training method, device, equipment and product for virtual production line simulation training in a mine. Background Art

[0002] A mine is a place where ore or other mineral resources are mined. There are many types of mines, including coal mines, iron mines, copper mines, gold mines, etc. Mining usually involves complex engineering technologies, including drilling, blasting, transportation and processing. The operation of mines is not only important for the extraction of resources, but also has an impact on the environment and ecosystem, so it needs to be properly managed and monitored.

[0003] With the continuous development of the energy industry and the advancement of digital transformation, cultivating digital talents has become a key challenge. There are great risks in employee training through actual scenarios, because mines are subject to dangerous situations such as collapse and landslides, harmful gases, dust explosions, high temperature and lack of oxygen during the actual mining process. Therefore, existing training usually uses electronic equipment for training. However, in the process of training, due to the lack of vivid and realistic scenes, the final training effect is not good and personnel cannot meet the job requirements.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of this application is to provide a training method, device, equipment and product for virtual production line simulation training in mines, aiming to solve the technical problems of poor training results caused by safety issues in training of personnel to be employed in physical mines and the lack of systematic scoring.

[0006] To achieve the above purpose, the present application proposes a training method for virtual production line simulation training in a mine, the method is applied to a digital mine training system, the digital mine training system includes a data acquisition controller, and the method includes:

[0007] Receive training start instructions from the user;

[0008] Responding to the training start instruction through the data acquisition controller, providing the user with a digital mine training task;

[0009] The operation information of the user when performing the digital mine training task is collected, and the operation information is evaluated in real time through a preset evaluation model to obtain an evaluation result.

[0010] The data acquisition controller includes a UI interface, an MR helmet and data gloves. Before the step of receiving the training start instruction from the user, the method further includes:

[0011] Detecting the helmet wearing status of the user through the MR helmet, and if the helmet wearing status of the user is complete, detecting the debugging readiness status of the interactive control device of the user through the data glove;

[0012] When the debugging preparation of the interactive control device is completed, a training list is provided to the user through the UI interface, and a training start instruction is received after the user selects the training content based on the training list.

[0013] In one embodiment, the digital mine training system further includes a networking platform, and the step of providing the user with a digital mine training task by responding to the training start instruction through the data acquisition controller includes:

[0014] Parsing the training start instruction to obtain training content;

[0015] Based on the training content, providing a selection menu through the UI interface, the selection menu including one or more of a click menu, a grab menu, a wrist menu, and a voice menu;

[0016] Based on the training content, providing physical feedback through the data glove, the physical feedback including vibration and resistance to fingers and palm;

[0017] Based on the training content, providing visual movement, voice recognition and three-dimensional labeling through the MR helmet;

[0018] Based on the training content, connecting corresponding operating devices in the digital mine training system through the networking platform;

[0019] Based on the selection menu, physical feedback, visual movement and voice recognition, three-dimensional labels and operating devices, training tasks are generated and provided to the user.

[0020] In one embodiment, the step of collecting the operation information of the user when performing the digital mine training task includes:

[0021] Collecting the body movement data of the user when the user performs the digital mine training task through a preset infrared capture device;

[0022] Collecting the finger and palm joint motion data of the user when the user performs the digital mine training task through the data gloves;

[0023] Collecting, through the networking platform, device data of the operating device when the user performs the digital mine training task;

[0024] collecting, by means of the MR helmet, head position data of the user when the user performs the digital mine training task;

[0025] Operation information is output according to the limb movement data, finger joint movement data, head position data and device data.

[0026] In one embodiment, the step of evaluating the operation information in real time using a preset evaluation model to obtain an evaluation result includes:

[0027] Integrating the limb motion data, finger joint motion data and head position data to obtain a user posture sequence;

[0028] Performing time alignment on the user posture sequence and the device sequence of the device data to obtain an alignment result;

[0029] Based on the alignment result, extracting features from the user posture sequence and the device sequence by using a time pyramid algorithm to obtain a feature sequence, and performing multi-scale sampling on the feature sequence to obtain a behavior representation;

[0030] According to the digital mine training task, template extraction is performed on the pre-built operation knowledge graph to obtain a standard template;

[0031] Performing similarity matching on the behavior representation and the standard template using a dynamic time warping algorithm to obtain a matching result;

[0032] The matching result is evaluated according to the operation weight value of the standard template to obtain an evaluation result.

[0033] In one embodiment, after the step of evaluating the operation information in real time using a preset evaluation model to obtain an evaluation result, the method further includes:

[0034] Diagnose the evaluation result to obtain a diagnosis result, wherein the diagnosis result includes one or more of a safety diagnosis result and an erroneous operation diagnosis result;

[0035] Based on the diagnosis result, the standard template is segmented to obtain a standard segment;

[0036] Prompt information is generated according to the standard fragment and the diagnosis result, and the prompt information is sent to the user, wherein the prompt information includes one or more of voice information, video information and animation information.

[0037] In one embodiment, after the step of providing the user with a digital mine training task by the data acquisition controller in response to the training start instruction, the method further comprises:

[0038] Detecting the user's physical condition, and when the physical condition is abnormal, stopping the digital mine training task and sending an alarm message to the corresponding training management terminal; and / or

[0039] The operating device is detected in real time through the networking platform. When an abnormality is detected in the operating device, the digital mine training task is stopped and an alarm message is sent to the corresponding training management terminal.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a training device for virtual production line simulation training in a mine, the device is applied to a digital mine training system, the digital mine training system includes a data acquisition controller, and the device includes:

[0041] A receiving module, used for receiving a training start instruction from a user;

[0042] A response module, used to respond to the training start instruction through the data acquisition controller to provide the user with a digital mine training task;

[0043] The collection module is used to collect the operation information of the user when performing the digital mine training task, and evaluate the operation information in real time through a preset evaluation model to obtain an evaluation result.

[0044] In addition, to achieve the above-mentioned objectives, the present application also proposes a training device for virtual production line simulation training in a mine, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the training method for virtual production line simulation training in a mine as described above.

[0045] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the training method for virtual production line simulation training of the mine as described above.

[0046] One or more technical solutions proposed in this application have at least the following technical effects:

[0047] The embodiment of the present application proposes a training method, device, equipment and computer program product for virtual production line simulation training of a mine, which receives a training start instruction from a user; responds to the training start instruction through the data acquisition controller to provide the user with a digital mine training task; collects the operation information of the user when performing the digital mine training task, and evaluates the operation information in real time through a preset evaluation model to obtain an evaluation result. Thus, after the digital mine training system receives the training start instruction from the user, the data acquisition controller responds to the training start instruction to provide the user with a digital mine training task, collects the user's operation information when the user performs the digital mine training task, and finally evaluates the user's operation information in real time through the evaluation model to obtain an evaluation result, which solves the problem that there are safety issues in the training of personnel waiting to take up their posts through physical mines, and there is no systematic scoring, resulting in poor training results, and improves the training effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 A flow chart of the training method for virtual production line simulation training of the present application provided in Example 1;

[0051] Figure 2 The training method for the virtual production line simulation training of the mine in this application involves a first schematic diagram of a digital mine training task;

[0052] Figure 3 A second schematic diagram of a digital mine training task involving a training method for a virtual mine production line simulation training for this application;

[0053] Figure 4 A schematic diagram of a training scenario involving a training method for a virtual production line simulation training for a mine in this application;

[0054] Figure 5 A flow chart of the second embodiment of the training method for virtual production line simulation training in the present application;

[0055] Figure 6 A training method for virtual production line simulation training for this application involves a schematic diagram of data gloves and UI interface interactive operations;

[0056] Figure 7 A schematic diagram of a standard template for the training method of the virtual production line simulation training for the mine in this application;

[0057] Figure 8 A schematic diagram of prompt information related to the training method for the virtual production line simulation training of the mine applied for this purpose;

[0058] Fig. 9 A schematic diagram of the module structure of a training device for virtual production line simulation training in a mine according to an embodiment of the present application;

[0059] Fig.10 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the training method for virtual mine production line simulation training in the embodiment of the present application.

[0060] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0061] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0062] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0063] The main solution of the embodiment of the present application is: the MR helmet detects the helmet wearing status of the user, and when the helmet wearing status of the user is completed, the data glove detects the debugging preparation status of the interactive control device of the user; when the debugging preparation of the interactive control device is completed, the UI interface provides the user with a training list, and receives the training start instruction sent by the user after selecting the training content based on the training list. The training start instruction is parsed to obtain the training content; based on the training content, a selection menu is provided through the UI interface, and the selection menu includes one or more of a click menu, a grab menu, a wrist menu and a voice menu; based on the training content, physical feedback is provided through the data glove, and the physical feedback includes vibration and resistance to the fingers and palms; based on the training content, the MR helmet provides visual movement, voice recognition and three-dimensional labels; based on the training content, the corresponding operating device in the digital mine training system is connected through the networking platform; based on the selection menu, physical feedback, visual movement and voice recognition, three-dimensional labels and operating devices, a training task is generated and provided to the user. The limb motion data of the user when the user performs the digital mine training task is collected by a preset infrared capture device; the finger and palm joint motion data of the user when the user performs the digital mine training task is collected by the data gloves; the equipment data of the operating device when the user performs the digital mine training task is collected by the networking platform; the head position data of the user when the user performs the digital mine training task is collected by the MR helmet; and operation information is output according to the limb motion data, finger joint motion data, head position data and equipment data. The limb motion data, finger joint motion data and head position data are integrated to obtain a user posture sequence; the user posture sequence and the device sequence of the device data are time-aligned to obtain an alignment result; based on the alignment result, the user posture sequence and the device sequence are feature extracted by a time pyramid algorithm to obtain a feature sequence, and the feature sequence is multi-scale sampled to obtain a behavior representation; according to the digital mine training task, a template is extracted from a pre-constructed operation knowledge graph to obtain a standard template; the behavior representation and the standard template are similarly matched by a dynamic time warping algorithm to obtain a matching result; the matching result is evaluated according to the operation weight value of the standard template to obtain an evaluation result.The evaluation result is diagnosed to obtain a diagnosis result, and the diagnosis result includes one or more of a safety diagnosis result and an erroneous operation diagnosis result; based on the diagnosis result, the standard template is segmented to obtain a standard segment; prompt information is generated according to the standard segment and the diagnosis result, and the prompt information is sent to the user, and the prompt information includes one or more of voice information, video information and animation information. The user's physical state is detected, and when the physical state is abnormal, the digital mine training task is stopped, and an alarm message is sent to the corresponding training management terminal; and / or the operating device is detected in real time through the networking platform, and when the operating device is detected to be abnormal, the digital mine training task is stopped, and an alarm message is sent to the corresponding training management terminal. Thereby, the safety problem of the personnel to be trained through the physical mine is solved, and there is no systematic scoring, resulting in poor training effect, and the training of the personnel to be trained is realized, and the training effect is improved. Based on the solution of the present invention, based on the fact that there are great risks in employee training through actual scenarios, because mines are exposed to dangerous situations such as collapse and landslides, harmful gases, dust explosions, high temperature and hypoxia during the actual mining process, the existing training usually uses electronic equipment for training. However, in the training process, due to the lack of vivid and realistic scenarios for training, the final training effect is not good, thus the effect is low. A training method for virtual production line simulation training of mines is designed, and the effectiveness of the training method for virtual production line simulation training of mines of the present invention is verified when training on-the-job personnel. Finally, the effect of training through the method of the present invention is significantly improved.

[0064] In this embodiment, for the convenience of description, the following description is made by taking the training device for the virtual production line simulation training of a mine as the execution subject.

[0065] Since mines are places where ore or other mineral resources are mined, there are many types of mines, including coal mines, iron mines, copper mines, gold mines, etc. Mining usually involves complex engineering technologies, including drilling, blasting, transportation and processing. Therefore, there are challenges in the safety and effectiveness of training for on-the-job personnel. However, there are great risks in employee training through actual scenarios, because in the actual mining process, mines are subject to dangerous situations such as collapse and landslides, harmful gases, dust explosions, high temperature and lack of oxygen. Therefore, existing training usually uses electronic equipment for training, but in the training process, due to the lack of vivid and realistic scenes for training, the final training effect is not good.

[0066] This application provides a solution that uses data gloves, rendering UI interfaces and MR helmets in a digital mine training system to improve the safety of training. At the same time, the training results of users are scored and recorded to provide users with better services.

[0067] It can be seen from the above embodiments that the present application receives the training start instruction from the user; responds to the training start instruction through the data acquisition controller to provide the user with a digital mine training task; collects the operation information of the user when performing the digital mine training task; scores the operation information through a preset evaluation model to obtain an operation score, and outputs the training result according to the operation score. Thus, after the digital mine training system receives the training start instruction from the user, the data acquisition controller responds to the training start instruction to provide the user with a digital mine training task, collects the user's operation information when the user performs the digital mine training task, and finally evaluates the user's operation information in real time through the evaluation model to obtain an evaluation result, which solves the problem of safety issues in training personnel to be employed through physical mines, and the problem of poor training effect due to the lack of systematic scoring, thereby improving the training effect.

[0068] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a training device for virtual production line simulation training in a mine, etc. The following takes the training device for virtual production line simulation training in a mine as an example to illustrate this embodiment and the following embodiments.

[0069] Based on this, the present application embodiment provides a training method for virtual production line simulation training of a mine, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the training method for virtual production line simulation training in the present application.

[0070] In this embodiment, the training method for the virtual production line simulation training of the mine includes steps S03 to S07:

[0071] Step S03, receiving a training start instruction from a user;

[0072] It should be clear that this embodiment is applied to the digital mine training system, which is mainly applicable to the use scenario of mines. In actual mining and management, it mainly includes:

[0073] (1) Safety management, namely risk assessment (identifying and evaluating potential risks, such as geological hazards, equipment failures, and operational accidents), safety training (training employees on safe operations and emergency response), and monitoring systems (using sensors and monitoring systems to track mine conditions in real time, such as gas concentrations and ground movement);

[0074] (2) Environmental management, namely environmental impact assessment (assessing the impact on the surrounding ecosystem before mining), waste management (treating and disposing of waste materials and wastewater generated by the mine to avoid pollution to the environment) and restoration plans (land reclamation and ecological restoration after mine closure to rebuild the original ecological environment);

[0075] (3) Economic management, namely cost control (monitoring and managing mining costs, including labor, equipment maintenance, and energy consumption), budgeting (preparing detailed budgets and financial plans to ensure maximum economic benefits), and resource optimization (effectively utilizing mineral resources to extend the mining life of the mine);

[0076] (4) Regulatory compliance, namely legal requirements (complying with national and local mining regulations and industry standards to ensure legal and compliant operations) and license management (obtaining and maintaining necessary mining licenses and environmental permits);

[0077] (5) Technology and innovation, namely advanced technology (adopting modern mining techniques, such as automated equipment and data analysis, to improve production efficiency) and research and development (investing in new technologies and methods to improve the safety and environmental friendliness of mining operations);

[0078] (6) Community relations, namely stakeholder communication (maintaining good communication with local communities and other stakeholders to understand their concerns and needs) and social responsibility (supporting local development projects and social responsibility programs to promote harmonious development of the mining area and surrounding areas).

[0079] Therefore, from the above description, it can be learned that there are many regulations and safety requirements in the management of mines, which has certain requirements on the professionalism of the staff. However, in the cultivation of digital talents, there is a great risk in employee training through actual scenarios, because there are dangerous situations such as collapse and landslides, harmful gases, dust explosions, high temperature and hypoxia in the actual mining process of mines. Therefore, the existing training usually uses electronic equipment for training. However, in the process of training, due to the lack of vivid and realistic scenes for training, the final training effect is not good, and the personnel cannot meet the job requirements. In this embodiment, a CAVE immersive virtual simulation system is mainly provided, and participants can use data gloves, force feedback devices, position tracking devices, etc. Virtual reality interactive devices such as trackers can achieve an immersive three-dimensional interactive experience. Among them, the immersive virtual reality system usually requires users to wear MR (or VR) helmets, isolated from the real world, enter the virtual environment, and achieve real-time interaction in a natural way. The super CAVE-U immersive virtual simulation system adopts a "CAVE-U-shaped" immersive physical structure design, supports immersive virtual simulation display, 3D stereo imaging and 6-degree-of-freedom real-time interaction, multi-channel immersive virtual simulation system, desktop virtual reality operation all-in-one machine and enhanced virtual simulation training system, which integrate virtual scenes with real environments, use augmented reality technology to integrate virtual elements into real scenes, and are suitable for operations that require real-virtual interaction.

[0080] In this embodiment, priority is given to receiving training start instructions from users. It should be clear that the digital mine training system in this embodiment is ultimately used for the training of personnel to be employed, so the received training start instructions can be issued by the trainee, or the system administrator can send instructions after confirming that the personnel to be trained have completed the equipment wearing and preparation work.

[0081] Step S04, responding to the training start instruction through the data acquisition controller to provide the user with a digital mine training task;

[0082] Since mining scenarios are risky, during training, the UI interface, MR helmet, and data gloves in this embodiment respond to the training start instruction to provide users with corresponding digital mine training tasks. Specifically, due to the different functions of each device, the training start instruction can be parsed first to obtain the training needs of this time, such as training objectives, training duration, and training content. After receiving the training needs, the UI interface can select the corresponding mode, and can provide corresponding guidance and help when the user feels uncomfortable or is unclear about the operation. The MR helmet is a device for immersive virtual reality experience. It creates a virtual environment. It enables users to interact with computer-generated content in an immersive way, and the data gloves can apply different vibrations and resistances to the user's fingers and palms according to the physical properties of the virtual objects, such as material, weight, collision, etc., so as to obtain simulated tactile sensation. After the above-mentioned multiple devices receive the training start instruction, they can provide users with corresponding digital mine training tasks. The training content in this embodiment includes but is not limited to safety production laws and regulations, safety production knowledge, occupational disease prevention and control, standard job procedures for positions, emergency response to accidents, self-rescue and trauma first aid, basic equipment operation, equipment maintenance manual, and equipment common problem solving through standard behaviors of real production lines.

[0083] In this embodiment, the digital mine training tasks are as follows: Figure 2 As shown in the figure, the current task is to check the working environment (check the top support), and the corresponding prompt is: before entering the post, you must carefully observe the working environment, and immediately deal with any debris such as spalling and phosphorus skin, and then you need to complete the following Figure 3 The task shown is: Check the working environment (check the pitch of the support in the working area). The specific prompt is: There needs to be a pitch lag between the working areas.

[0084] Step S07, collecting the operation information of the user when performing the digital mine training task, and evaluating the operation information in real time through a preset evaluation model to obtain an evaluation result;

[0085] When the user performs the digital mine training task, the corresponding interface is as follows Figure 4 As shown, there are corresponding guidance personnel, objects that need to be handled (number of hazardous sources) and operating tools, etc. During this process, the user needs to check and operate the corresponding hazardous sources. At this time, the user's actions will be collected through the above-mentioned multiple devices to obtain the user's specific operations, and the operation information can be obtained after integration.

[0086] After the training is completed or during the training process, the operation information can be scored through the preset evaluation model. It should be clear that in the digital mine training system, each operation has a corresponding template for comparison. For example, the running speed of the mine car may cause unsafe conditions when it exceeds a certain amount, but the machine being adjusted at this time has certain angle problems. For example, when the horizontal angle is lower than 75%, it is a normal range. In the event of an accident, in order to ensure the safety of personnel, the speed is reduced to 50%, but if it is lower than 30%, the efficiency will be too low. At this time, when operating manually, it is necessary to maintain the range of about 50%, and the angle and action of the pull may also need to be standardized accordingly. Therefore, there is a corresponding operation template for comparison. After each action is compared, the user's operation score can be obtained, and the training results can be output based on this.

[0087] It should be clear that after obtaining the evaluation results, this embodiment can accurately push operation instructions or demonstration videos based on operation errors, thereby realizing a one-to-one intelligent tutor function for skill operation. This function has two modes: simulation training and assessment evaluation, which can solve the automation of operation skill training and assessment evaluation for multiple positions in digital mines.

[0088] In addition, the digital mine training system also effectively cooperates with users to achieve strong interaction and accurately push digital resources. Digital resources include but are not limited to the following data types:

[0089] (1) Text resources: including academic papers, books, articles, reports, meeting minutes, FAQs (Frequently Asked Questions), glossaries, operating instructions, etc.;

[0090] (2) Multimedia resources: composite documents that combine text, images, audio, and video, such as presentations, video lectures, illustrated manuals, etc.;

[0091] (3) Graphic resources: such as photos, charts, diagrams, flow charts, maps, schematic diagrams, etc., used to visually display information;

[0092] (4) Audio resources: such as lecture recordings, audiobooks, podcasts, interview recordings, etc., which provide information in audio form;

[0093] (5) Video resources: including instructional videos, product demonstrations, how-to videos, interview videos, etc., combining visual and auditory elements;

[0094] (6) 3D models and animations: 3D visualization resources used to display complex structures or dynamic processes;

[0095] (7) Database: contains structured data sets, such as research data sets, statistical data, market data, etc., which can be retrieved and analyzed through queries;

[0096] (8) Software and Tools: The knowledge base may contain or link to software applications, computational tools, and simulation tools for specific tasks;

[0097] (9) Websites and web pages: Knowledge bases may contain links to external resources or integrate web page content as part of the knowledge base;

[0098] (10) User-generated content: knowledge, experience sharing, comments, ratings, and feedback contributed by the community or users.

[0099] Specifically, before the step S03 of receiving the training start instruction from the user, the method further includes:

[0100] Step S01, detecting the helmet wearing status of the user through the MR helmet, and if the helmet wearing status of the user is complete, detecting the debugging readiness status of the interactive control device of the user through the data glove;

[0101] Step S02, when the debugging preparation of the interactive control device is completed, providing the user with a training list through the UI interface, and receiving a training start instruction sent by the user after selecting training content based on the training list.

[0102] After understanding the structure and function of the digital mine training system, the user's wearing condition can be detected. In actual use, improper wearing may lead to poor final imaging effect and training effect experience. Therefore, in this embodiment, the MR helmet is used to detect the user's helmet wearing condition. When the user's helmet wearing condition is complete, the data gloves are used to detect the user's glove wearing condition. It should be clear that the wearing condition should be determined according to the user's senses. For example, when the user's imaging effect is clear and the rotation condition is good, it can be output that the wearing is complete, and the same applies to data gloves.

[0103] After both the data gloves and the MR helmet are put on, the corresponding interface will pop up through the UI interface for the user to select. It should be clear that the training in this embodiment may include safety issue training, skill issue training, and routine issue training, etc. In addition to the training function, the present embodiment can also provide a corresponding examination function. For example, after the user completes the training of multiple tasks, the degree of mastery can be assessed in various levels. For example, in special task situations, safety issues take precedence over skill issues. At this time, the safety issue ratio of the current operation can be set to 70%, skill issues to 25%, and routine issues to 5%, and corresponding assessments can be conducted.

[0104] The specific process is as follows: after the user has completed wearing, he enters the mode selection interface (select training or assessment). After the selection is completed, the corresponding device will be selected, such as a mining machine, a mine car, and a transportation system, and then the corresponding proportion will be selected, such as the safety, skills, and routine proportions mentioned above. After the selection is completed, a training start instruction can be issued, and the digital mine training system will respond to obtain a digital mine training task.

[0105] It should be clear that the digital mine training system also includes a networking platform, the MR helmet includes a VR helmet and AR glasses, and the data acquisition controller includes one or more of a UI interface, a VR helmet and data gloves. Therefore, in step S04, the step of providing the user with a digital mine training task by responding to the training start instruction through the data acquisition controller includes:

[0106] Step S041, parsing the training start instruction to obtain training content;

[0107] Step S042, based on the training content, providing a selection menu through the UI interface, wherein the selection menu includes one or more of a point menu, a grab menu, a wrist menu, and a voice menu;

[0108] Step S043, providing physical feedback through the data glove based on the training content, wherein the physical feedback includes vibration and resistance to the fingers and palm;

[0109] Step S044, based on the training content, providing visual movement conditions, voice recognition and three-dimensional labels through the MR helmet;

[0110] Step S045, based on the training content, connecting the corresponding operating device in the digital mine training system through the networking platform;

[0111] Step S046, generating a training task based on the selection menu, physical feedback, visual movement and voice recognition, three-dimensional labels and operating equipment, and providing it to the user.

[0112] Since the digital mine training system in this embodiment also includes a networking platform, the MR helmet includes a VR helmet and AR glasses, the UI interface is used to provide one or more of a grab menu, a wrist menu and a voice menu for the user to choose when the user performs a task, the data gloves are used to provide vibration and resistance to the user's fingers and palms so that the user can get simulated tactile sensation, the VR helmet is used to provide visual movement and voice recognition, the AR glasses are used to generate three-dimensional labels so that the user can view them in real time, and the networking platform is used to connect several operating devices in the digital mine training system, so these devices are further described. Among them, the head position and hand movements of the user wearing the VR helmet can be accurately tracked, and these data can be mapped to perspective changes and gesture interaction events in the virtual scene. For example, when the user turns his head, the scene perspective moves accordingly. When the user makes a specific gesture, it can trigger operations such as switching equipment and grabbing tools. At the same time, the voice Voice recognition technology allows users to control the system through oral commands, and when the AR glasses in the VR helmet are worn, the physical equipment in front of them will be recognized and reconstructed in three dimensions by the system. On this basis, the platform obtains the real-time status data of the equipment and displays it on the surface of the equipment in the form of three-dimensional labels for users to view in real time. The labels can display the nameplate parameters, operating parameters, energy consumption levels and fault warnings of the equipment. When changing the viewing angle, the labels will remain aligned with the equipment. The user interface in the VR environment can avoid using traditional two-dimensional menus, but instead adopt natural interaction paradigms such as circular ray menus, grab menus, wrist menus, and voice menus to reduce the user's cognitive burden. The clickable areas of all virtual buttons are made larger and wider to improve the selection hit rate. Key information is always kept in the center of the field of vision. When an abnormality occurs, prompts are given by flashing, color changes, etc. The operating devices include but are not limited to one or more virtual devices, digital video resources, semi-physical simulation devices, or real operating devices.

[0113] More specifically, after the step S04, in which the data acquisition controller responds to the training start instruction to provide the user with a digital mine training task, the method further includes:

[0114] Step S05, detecting the user's physical condition, and when the physical condition is abnormal, stopping the digital mine training task and sending an alarm message to the corresponding training management terminal; and / or

[0115] Step S06, performing real-time detection on the operating device through the networking platform, and when an abnormality is detected in the operating device, stopping the digital mine training task and sending an alarm message to the corresponding training management terminal.

[0116] During the training or assessment process, the safety of users and equipment needs to be given priority. Since the training scene in this embodiment may involve full virtualization (i.e., operations are performed in a specific space) or semi-virtualization (i.e., corresponding operations are performed in the actual scene), the safety of users is particularly important at this time. This embodiment provides two detection methods: equipment safety and user safety, which can be combined or selected alone. For the user's physical condition, this embodiment uses MR helmets, data gloves, and infrared detection equipment to detect the user's physical condition. In the event of a fall or dizziness, the digital mine training task is stopped, and an alarm message is sent to the corresponding training management end (i.e., training management personnel, etc.);

[0117] In addition, this embodiment also detects the equipment for specific operations. Since many operating devices are connected to the networking platform, these operating devices are detected in real time through the networking platform to avoid user misoperation or various emergencies of the device. When an abnormality is detected in the operating device, the digital mine training task is stopped and an alarm message is sent to the corresponding training management terminal.

[0118] This embodiment adopts the above scheme, specifically by receiving the training start instruction of the user; responding to the training start instruction through the data acquisition controller to provide the user with a digital mine training task; collecting the operation information of the user when performing the digital mine training task, and evaluating the operation information in real time through a preset evaluation model to obtain an evaluation result. Therefore, after the digital mine training system receives the training start instruction of the user, the data acquisition controller responds to the training start instruction to provide the user with a digital mine training task, collects the user's operation information when the user performs the digital mine training task, and finally evaluates the user's operation information in real time through the evaluation model to obtain an evaluation result, which solves the problem of safety problems in training personnel to be employed through physical mines, and the problem of poor training effect due to the lack of systematic scoring, thereby improving the training effect.

[0119] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 5 , step S07, collecting the operation information of the user when performing the digital mine training task, the training method of the mine virtual production line simulation training also includes steps S071 to S075:

[0120] Step S071, collecting body movement data of the user when the user performs the digital mine training task through a preset infrared capture device;

[0121] Step S072, collecting the finger and palm joint motion data of the user when the user performs the digital mine training task through the data glove;

[0122] Step S073, collecting, through the networking platform, equipment data of the operating device when the user performs the digital mine training task;

[0123] Step S074, collecting the head position data of the user when the user performs the digital mine training task through the MR helmet;

[0124] Step S075, outputting operation information according to the limb movement data, finger joint movement data, head position data and device data.

[0125] In the training scenario, the data gloves and UI interface in this embodiment are as follows: Figure 6 As shown in the figure, the UI interface can display the tools (wrench, screwdriver, flashlight, etc.) provided during the current training operation, and the data gloves can sense the user's operating area, as shown in the green line segment in the figure;

[0126] In the above-mentioned embodiments, the functions of each device in the digital mine system have been explained. Subsequently, the body movements of the user when performing the digital mine training task can be collected through a preset infrared capture device (such as Kinect, etc.) to obtain body movement data. Among them, Kinect is a motion capture device that captures the user's whole body movements through a depth camera and an infrared sensor. Without the need for a handheld controller, Kinect can track the user's posture and movement, and convert it into the scene in real time, providing an immersive interactive experience;

[0127] For the user's finger operation, this embodiment collects data through data gloves to obtain the user's finger joint movement data, and for the operating device involved in the user's digital mine training task, this embodiment collects the device data of the operating device through the networking platform;

[0128] Subsequently, in a specific task scenario, the user needs to perform operations such as walking and turning, so this embodiment also collects the user's head position data when the user performs the digital mine training task through the MR helmet, including the user's rotation angle of view and focus object, etc.;

[0129] Finally, operation information can be output based on limb motion data, finger joint motion data, head position data and device data.

[0130] Specifically, Figure 1 In the illustrated embodiment, the step S07, evaluating the operation information in real time by using a preset evaluation model, and obtaining the evaluation result comprises:

[0131] Step S076, integrating the limb motion data, finger joint motion data and head position data to obtain a user posture sequence;

[0132] Step S077, integrating the limb motion data, finger joint motion data and head position data to obtain a user posture sequence;

[0133] Step S078, time-aligning the user posture sequence and the device sequence of the device data to obtain an alignment result;

[0134] Step S079, based on the alignment result, extracting features from the user posture sequence and the device sequence by using a time pyramid algorithm to obtain a feature sequence, and performing multi-scale sampling on the feature sequence to obtain a behavior representation;

[0135] Step S080, according to the digital mine training task, extracting a template from the pre-built operation knowledge graph to obtain a standard template;

[0136] Step S081, performing similarity matching on the behavior representation and the standard template using a dynamic time warping algorithm to obtain a matching result;

[0137] Step S082, evaluating the matching result according to the operation weight value of the standard template to obtain an evaluation result.

[0138] After obtaining the user's operation information, it is necessary to classify the items in the operation information. Since the device information and posture information are different, in this embodiment, the collected user motion sequence (posture sequence, including limb movement data, finger joint motion data and head position data) and device state sequence (including device data) are aligned in time and then feature extraction is performed. It should be clear that the motion sequence extracts physical features such as speed, acceleration, and trajectory smoothness, while the device sequence extracts threshold events and slope features. Then, the time pyramid algorithm is used to perform multi-scale sampling on the feature sequence to obtain macro and micro behavior representations under different time spans. Among them, the time pyramid algorithm is a technology for processing and analyzing time series data. It decomposes the time series data into multiple different time scales (or "pyramid levels") so that it can be processed at different time resolutions.

[0139] After obtaining the user's behavior representation, it is necessary to have a corresponding best score judgment. Therefore, this embodiment extracts the template of the pre-built operation knowledge graph according to the digital mine training task, and obtains the following Figure 7The standard template shown includes the step-by-step instructions and animation demonstrations for virtual disassembly and assembly. The dynamic time warping algorithm can then be used to perform similarity matching on the behavior representation and the standard template to obtain a matching result. The operation graph is a hierarchical operation knowledge graph constructed for each standard operation. The top-level nodes of the graph are coarse-grained operations, such as "host startup", the middle-level nodes are various steps, such as "close the power switch", and the bottom-level nodes are various actions, such as "raise the right arm and hold the switch handle". Each node records the spatiotemporal feature sequence of the operation to form a behavior template (standard template). The dynamic time warping (DTW) algorithm is a technology for comparing and aligning two time series. It aligns them on the time axis by calculating the minimum distance between the sequences in order to find the best match. It matches the feature sequence generated by the user's actual operation with the behavior template of the standard operation. The dynamic time warping (DTW) algorithm is used to calculate the similarity of the two multidimensional time series and give a fine-grained matching score. Template matching can be trained end-to-end based on the twin network in deep learning.

[0140] Finally, since corresponding weight information is also set in the actual operation process, for example, when performing circuit work, safety takes priority, so the safety ratio is 70% and the operation specification ratio is 30%. More points will be deducted for safety issues in the operation, and fewer points will be deducted for operational specifications. In this way, the matching results are scored accordingly to obtain the scoring results, and the training results are obtained by combining the user's single or multiple scoring results.

[0141] It should be noted that this embodiment can also provide feedback information while the user is operating, and the feedback information includes but is not limited to operation scores, detailed lists of operation conditions, whether the operation steps are correct or not, prompts of operation errors, demonstration videos of operation content, etc. for users to learn accurately and train repeatedly. At the end of the digital mine training task, the training results are provided, including operation scores, training time, knowledge and skills completion, etc.

[0142] It should be noted that for some special usage scenarios, the parameters need to be changed by twisting the fingers, but the glasses correspond to observing the instruments on the left side of the body, so the fingers are out of sight. For such operations, further attention should be paid to safety and operational norms.

[0143] More specifically, after the step S082 of evaluating the matching result according to the operation weight value of the standard template to obtain the evaluation result, the method further includes:

[0144] Step S083, diagnosing the training result to obtain a diagnosis result, wherein the diagnosis result includes one or more of a safety diagnosis result and an erroneous operation diagnosis result;

[0145] Step S084, performing segmentation on the standard template based on the diagnosis result to obtain a standard segment;

[0146] Step S085, generating prompt information according to the standard clip and the diagnosis result, and sending the prompt information to the user, wherein the prompt information includes one or more of voice information, video information and animation information.

[0147] In this embodiment, the user's operation can also be traced back for errors. For operations with scores below the threshold, the cause of the error can be diagnosed and visualized. For example, the order of action execution is reversed (the order of the wrong action can be marked), key steps are missing (the incomplete nodes in the graph can be located), and the action amplitude is too large or too small (can be magnified and rendered through skeletal animation), etc. According to the type of error, the system gives corresponding voice, video, and animation prompts (i.e., prompt information) to guide the user to standardize the operation. The prompt information in this embodiment is as follows: Figure 8 As shown, the operation flow of step 2 is pointed out, and the actual operation is analyzed to find out where the error problem is and the corresponding impact it will cause.

[0148] In addition, the digital mine training system in this embodiment also has:

[0149] (1) Open architecture: The system strictly follows the Industrial Internet Architecture (IIRA). All layers are interoperable through standard protocols. Flexible north-south interfaces are reserved in data collection, edge computing, artificial intelligence, knowledge base, etc., which can easily access various new scenarios of smart mines.

[0150] (2) Customized working conditions: Thanks to digital twin technology, the system can quickly build a new digital scene of the mine. Engineers only need to adjust some parameters and scripts to meet differentiated requirements such as equipment model, mining process, geological conditions, and failure mode, greatly shortening the delivery cycle.

[0151] (3) Network isolation: The system is divided into multiple defense-in-depth areas, including the production control area, the management information area, and the external service area. Each area is isolated by firewalls and one-way network switches. The simulation training system is physically isolated from the Internet and is only connected to the industrial Ethernet to prevent network intrusion.

[0152] (4) Access control: The system is connected to the enterprise's unified identity authentication system, which strictly limits account permissions. Ordinary employees can only access training content related to their positions. Training instructors can import courseware and publish assignments. Only system administrators have the right to adjust system parameters. In addition, the system has a built-in operation audit function that automatically records each user's login and operation log to ensure security and traceability.

[0153] The digital mine training system in this embodiment has been tested for many rounds. Compared with the traditional manual training method, the results achieved in terms of training efficiency, training quality, safety accidents and cost savings are shown in the following table:

[0154] index promote Average training time per person 50% shorter Knowledge mastery rate 30% improvement Operation qualification rate 35% improvement Retraining rate 80% reduction Safety incidents 90% reduction Training cost per person 60% reduction

[0155] This embodiment uses the above scheme to collect the user's body movement data when the user performs the digital mine training task through a preset infrared capture device; collect the user's finger and palm joint movement data when the user performs the digital mine training task through the data gloves; collect the device data of the operating device when the user performs the digital mine training task through the networking platform; collect the user's head position data when the user performs the digital mine training task through the MR helmet; and output operation information according to the body movement data, finger joint movement data, head position data and device data. Therefore, after the digital mine training system receives the user's training start instruction, the data acquisition controller responds to the training start instruction, provides the user with a digital mine training task, collects the user's operation information when the user performs the digital mine training task, and finally evaluates the user's operation information in real time through the evaluation model to obtain the evaluation result, which solves the problem that the personnel to be employed are trained through the physical mine and there is no systematic scoring, resulting in poor training effect, and improves the training effect.

[0156] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the training method of the virtual production line simulation training of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0157] This application also provides a training device for virtual production line simulation training in mines, please refer to Fig. 9 , the training device for the virtual production line simulation training of the mine includes:

[0158] A receiving module 10, used to receive a training start instruction from a user;

[0159] A response module 20, configured to respond to the training start instruction through the data acquisition controller to provide the user with a digital mine training task;

[0160] The collection module 30 is used to collect the operation information of the user when performing the digital mine training task, and evaluate the operation information in real time through a preset evaluation model to obtain an evaluation result.

[0161] The training device for virtual production line simulation training of mines provided in the present application adopts the training method for virtual production line simulation training of mines in the above-mentioned embodiments, which can solve the technical problems that there are safety issues in training personnel to be employed through physical mines, and there is no systematic scoring, resulting in poor training effects. Compared with the prior art, the beneficial effects of the training device for virtual production line simulation training of mines provided in the present application are the same as the beneficial effects of the training method for virtual production line simulation training of mines provided in the above-mentioned embodiments, and the other technical features of the training device for virtual production line simulation training of mines are the same as the features disclosed in the above-mentioned embodiment methods, which will not be repeated here.

[0162] The present application provides a training device for virtual production line simulation training in a mine, and the training device for virtual production line simulation training in a mine includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the training method for virtual production line simulation training in the mine in the above-mentioned embodiment one.

[0163] Reference below Fig.10 , which shows a schematic diagram of the structure of a training device suitable for implementing the virtual production line simulation training of a mine in the embodiment of the present application. The training device for the virtual production line simulation training of a mine in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.10 The training equipment for virtual mine production line simulation training shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0164] like Fig.10As shown, the training equipment for virtual production line simulation training in a mine may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. Various programs and data required for the operation of the training equipment for virtual production line simulation training in a mine are also stored in RAM1004. The processing device 1001, ROM1002, and RAM1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the training device for the virtual production line simulation training of the mine to communicate wirelessly or wired with other devices to exchange data. Although the figure shows the training device for the virtual production line simulation training of the mine with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0165] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0166] The training equipment for virtual production line simulation training of mines provided by the present application adopts the training method for virtual production line simulation training of mines in the above-mentioned embodiment, which can solve the technical problems that there are safety issues in training personnel to be employed through physical mines, and there is no systematic scoring, resulting in poor training effects. Compared with the prior art, the beneficial effects of the training equipment for virtual production line simulation training of mines provided by the present application are the same as the beneficial effects of the training method for virtual production line simulation training of mines provided by the above-mentioned embodiment, and the other technical features of the training equipment for virtual production line simulation training of mines are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0167] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0168] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0169] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the training method for virtual production line simulation training in the mine in the above-mentioned embodiment.

[0170] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0171] The computer-readable storage medium may be included in the training device for the virtual production line simulation training of a mine; or may exist independently without being assembled into the training device for the virtual production line simulation training of a mine.

[0172] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the training equipment for virtual production line simulation training of mines, the training equipment for virtual production line simulation training of mines: receives the training start instruction of the user; responds to the training start instruction through the data acquisition controller to provide the user with a digital mine training task; collects the operation information of the user when performing the digital mine training task; scores the operation information through a preset evaluation model to obtain an operation score, and outputs the training result according to the operation score.

[0173] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0174] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0175] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0176] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned training method for virtual production line simulation training in mines, which can solve the technical problems that there are safety issues in training personnel to be employed through physical mines, and there is no systematic scoring, resulting in poor training effects. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the training method for virtual production line simulation training in mines provided in the above-mentioned embodiments, and will not be elaborated here.

[0177] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the training method for virtual production line simulation training in a mine as described above.

[0178] The computer program product provided by this application can solve the technical problems that there are safety issues in training personnel in physical mines, and there is no systematic scoring, resulting in poor training effects. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the training method for virtual production line simulation training in mines provided in the above embodiments, and will not be elaborated here.

[0179] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A training method for virtual production line simulation training in a mine, characterized in that: The method is applied to a digital mine training system, the digital mine training system includes a data acquisition controller, and the method includes: Receive training start instructions from the user; Responding to the training start instruction through the data acquisition controller, providing the user with a digital mine training task; The operation information of the user when performing the digital mine training task is collected, and the operation information is evaluated in real time through a preset evaluation model to obtain an evaluation result.

2. The method according to claim 1, characterized in that The data acquisition controller includes a UI interface, an MR helmet and data gloves. Before the step of receiving the training start instruction from the user, the method further includes: Detecting the helmet wearing status of the user through the MR helmet, and if the helmet wearing status of the user is complete, detecting the debugging readiness status of the interactive control device of the user through the data glove; When the debugging preparation of the interactive control device is completed, a training list is provided to the user through the UI interface, and a training start instruction is received after the user selects the training content based on the training list.

3. The method according to claim 2, characterized in that The digital mine training system further includes a networking platform, and the step of providing the user with a digital mine training task by responding to the training start instruction through the data acquisition controller includes: Parsing the training start instruction to obtain training content; Based on the training content, providing a selection menu through the UI interface, the selection menu including one or more of a click menu, a grab menu, a wrist menu, and a voice menu; Based on the training content, providing physical feedback through the data glove, the physical feedback including vibration and resistance to fingers and palm; Based on the training content, providing visual movement, voice recognition and three-dimensional labeling through the MR helmet; Based on the training content, connecting corresponding operating devices in the digital mine training system through the networking platform; Based on the selection menu, physical feedback, visual movement and voice recognition, three-dimensional labels and operating devices, training tasks are generated and provided to the user.

4. The method according to claim 3, characterized in that The step of collecting the operation information of the user when performing the digital mine training task includes: Collecting the body movement data of the user when the user performs the digital mine training task through a preset infrared capture device; Collecting the finger and palm joint motion data of the user when the user performs the digital mine training task through the data gloves; Collecting, through the networking platform, device data of the operating device when the user performs the digital mine training task; collecting, by means of the MR helmet, head position data of the user when the user performs the digital mine training task; Operation information is output according to the limb movement data, finger joint movement data, head position data and device data.

5. The method according to claim 4, characterized in that The step of evaluating the operation information in real time by using a preset evaluation model to obtain an evaluation result comprises: Integrating the limb motion data, finger joint motion data and head position data to obtain a user posture sequence; Performing time alignment on the user posture sequence and the device sequence of the device data to obtain an alignment result; Based on the alignment result, extracting features from the user posture sequence and the device sequence by using a time pyramid algorithm to obtain a feature sequence, and performing multi-scale sampling on the feature sequence to obtain a behavior representation; According to the digital mine training task, template extraction is performed on the pre-built operation knowledge graph to obtain a standard template; Performing similarity matching on the behavior representation and the standard template using a dynamic time warping algorithm to obtain a matching result; The matching result is evaluated according to the operation weight value of the standard template to obtain an evaluation result.

6. The method according to claim 5, characterized in that After the step of evaluating the matching result according to the operation weight value of the standard template to obtain the evaluation result, the method further comprises: Diagnose the evaluation result to obtain a diagnosis result, wherein the diagnosis result includes one or more of a safety diagnosis result and an erroneous operation diagnosis result; Based on the diagnosis result, the standard template is segmented to obtain a standard segment; Prompt information is generated according to the standard fragment and the diagnosis result, and the prompt information is sent to the user, wherein the prompt information includes one or more of voice information, video information and animation information.

7. The method according to claim 3, characterized in that After the step of providing the user with a digital mine training task by the data acquisition controller responding to the training start instruction, the method further includes: Detecting the user's physical condition, and when the physical condition is abnormal, stopping the digital mine training task and sending an alarm message to the corresponding training management terminal; and / or The operating device is detected in real time through the networking platform. When an abnormality is detected in the operating device, the digital mine training task is stopped and an alarm message is sent to the corresponding training management terminal.

8. A training device for virtual production line simulation training in a mine, characterized in that: The device is applied to a digital mine training system, the digital mine training system includes a data acquisition controller, and the device includes: A receiving module, used for receiving a training start instruction from a user; A response module, used to respond to the training start instruction through the data acquisition controller to provide the user with a digital mine training task; The collection module is used to collect the operation information of the user when performing the digital mine training task, and evaluate the operation information in real time through a preset evaluation model to obtain an evaluation result.

9. A training device for virtual production line simulation training in a mine, characterized in that: The device comprises: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the training method for virtual production line simulation training of a mine as described in any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the training method for virtual production line simulation training of a mine as claimed in any one of claims 1 to 7 are implemented.

Images