Safety training simulation rehearsal method and system based on digital twinning

Abstract

The invention provides a safety training simulation rehearsal method and system based on digital twinning, and the method comprises the steps: collecting three-dimensional posture image data when a user executes a safety training plot task, mapping the data to a safety operation virtual scene, and generating a corresponding three-dimensional virtual posture image; and recognizing three-dimensional operation posture characteristics in the three-dimensional virtual posture image through a posture recognition model, and judging whether the three-dimensional operation posture characteristics meet the posture operation requirements of the corresponding safety training plot task or not. Safety training simulation rehearsal learning can be carried out on a user, AI identification can be carried out on a three-dimensional virtual posture in safety training, and whether a safety training operation action of the user reaches the standard or not is judged in a virtual environment, so that a function of carrying out evaluation according to an actual action posture of an entity model in a virtual system is realized; therefore, an evaluation effect equivalent to practical operation is achieved, users such as trainees and the like can realize virtual preview experience and assessment of safe practical operation training, and online teaching experience and learning ability of the trainees are improved.

Description

Technical Field

[0001] The present invention relates to the field of safety management technology, and in particular to a safety training simulation rehearsal method based on digital twins, a safety training simulation rehearsal system based on digital twins, an electronic device, and a computer-readable storage medium. Background Art

[0002] Underground electrical work is an important part of mine production, which involves multiple links such as the installation, maintenance, operation and troubleshooting of electrical equipment. The underground environment is humid and dark, with a large amount of dust, gas and other harmful gases, and the space is narrow, which poses a serious threat to the safe operation of electrical equipment. Therefore, underground electrical work must strictly abide by safety operating procedures, use qualified electrical equipment, and strengthen maintenance and inspection to ensure the safe and efficient operation of the mine. Therefore, it is necessary to conduct safety training for underground electrical operations, and its necessity is mainly reflected in the following aspects: Improve safety awareness: The underground electrical working environment is complex and dangerous. Safety training can enhance the safety awareness of operators, enable them to be vigilant at all times, strictly abide by operating procedures, and reduce the occurrence of accidents.

[0003] Improve operating skills: There are many types of underground electrical equipment, and the operation is complicated. Safety training can enable operators to master the operation methods and maintenance skills of the equipment, and improve work efficiency and safety.

[0004] Standardize work processes: Through training, the work processes of operators can be standardized, the probability of operational errors can be reduced, and each step of the operation can be ensured to meet safety standards.

[0005] Reduce the accident rate: Underground electrical accidents often cause serious consequences, including casualties, equipment damage and mine shutdown. Safety training can effectively improve the safety quality of operators, thereby reducing the probability of accidents and ensuring safe production in mines.

[0006] Traditional safety training methods mainly rely on administrators to provide offline training to trainees. However, this method requires trainees to study intensively, which takes up safety production time and is not suitable for large-scale enterprises. With the emergence of technologies such as virtual simulation, online training preview modes for online virtual teaching, learning, and assessment have emerged, such as the attached Figure 1As shown in the diagram of a virtual system, the administrator can virtually build the corresponding safety training simulation rehearsal scene on the computer simulation system in advance. Trainees and other users can interact with the system through virtual interactive devices such as VR terminals (VR glasses or helmets, etc.) and operating handles. The system outputs the corresponding safety training plot tasks to the VR terminal for plot interactive display. During this period, the user performs data sensing and data collection through the device and inputs the data into the system. The system calculates and virtually maps the physical data, so as to perform corresponding data operations, task selection and other operations in the virtual scene.

[0007] Although the above-mentioned existing technologies can provide certain offline and online safety training and teaching services, they still have the following shortcomings in safety training applications: Existing virtual simulation training mainly requires trainees to complete virtual teaching (similar to game plots) by operating virtual characters provided by computer simulation systems and performing corresponding plot tasks. Therefore, it is impossible for trainees to conduct analog practical training, and they lack the experience of safe operation and skill learning. Existing computer simulation systems conduct virtual training assessments on trainees mainly by scoring the virtual characters' progress in completing storyline completion nodes (or plots, task nodes, etc.). Therefore, it is impossible to conduct a virtual evaluation of the trainees' own operating actions, that is, it is impossible to evaluate the actual action postures of the physical model in its virtual system. There is a lack of virtual evaluation of safe operational practicality, and trainees and other users are unable to achieve virtual rehearsal experience and assessment of safe practical training. Therefore, the training efficiency and significance are low. Summary of the invention

[0008] In order to solve the technical problems existing in the prior art, the present invention provides the following technical solutions: On the one hand, a safety training simulation rehearsal method based on digital twins is provided, which is implemented by an electronic device and includes: S1. Based on digital twin technology, construct a virtual scene of safe operation and several safety training scenario tasks under the virtual scene of safe operation; S2, collecting three-dimensional posture image data when the user performs the safety training scenario task, and uploading it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; S3, identifying the three-dimensional operation posture features in the three-dimensional virtual posture image through the posture recognition model preset on the background server, and judging whether the three-dimensional operation posture features meet the posture operation requirements corresponding to the safety training scenario task: If it is in compliance, the user's execution log of the safety training scenario task is recorded and saved; If not, a corresponding operation error notification is generated and sent to the VR device worn by the user; S4. Repeat the above steps until the three-dimensional operation posture characteristics of the user performing all the safety training scenario tasks meet the corresponding posture operation requirements.

[0009] Preferably, the S1, based on the digital twin technology, constructs a virtual scene of safe operation and several safety training scenario tasks in the virtual scene of safe operation, including: The 3D structure image data of the physical model of the safety place is collected by a 3D camera deployed on site, and uploaded to the backend server; After receiving and preprocessing the three-dimensional structure image data, the backend server imports the data into a preset digital twin platform, performs three-dimensional modeling based on the digital twin technology, and constructs a digital twin site model corresponding to the physical model of the safety site; Record and bind the data set mapping relationship between the physical model of the safe place and the digital twin place model to construct the safe operation virtual scene; According to the preset safety operation process, a plurality of safety training scenario tasks corresponding to the safety operation process are constructed based on the safety operation virtual scene, and corresponding task attributes and task levels are configured for each of the safety training scenario tasks.

[0010] Preferably, the step S2, collecting three-dimensional posture image data when the user performs the safety training scenario task, and uploading it to a backend server, wherein the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image, includes: The user receives and executes the safety training scenario task distributed by the backend server through the VR device worn by the user; Through the 3D camera deployed on site, the 3D posture image data of the user's target part operating action when performing the safety training scenario task is collected in real time and uploaded to the backend server; The backend server receives and pre-processes the three-dimensional posture image data, imports it into a preset digital twin platform, and performs three-dimensional modeling based on the digital twin technology to construct a digital twin posture model corresponding to the target part; Recording and binding a data set mapping relationship between the digital twin posture model and the digital twin place model; The digital twin posture model is projected into the corresponding task scene in the safe operation virtual scene, and a corresponding three-dimensional virtual posture image is generated and synchronized to the VR device worn by the user.

[0011] Preferably, the user receives and executes the safety training scenario task distributed by the background server through the VR device worn by the user, including: The user logs in to the backend server through the VR device he wears, and the backend server records and saves the user ID; The backend server generates a security operation question and sends it to the VR device; The user responds to the safety operation question through the VR device and provides corresponding answer information; The backend server identifies the degree of match between the safety operation question and the answer information, and assigns a safety training scenario task with a task level matching the degree of match to the user according to the degree of match, and sends the task to the VR device worn by the user; The user receives and executes the safety training scenario tasks through the VR device he wears, and makes corresponding feedback.

[0012] Preferably, the method for generating the posture recognition model comprises: The administrator executes the safety training simulation rehearsal process, executes each of the safety training scenario tasks in turn, collects the three-dimensional posture image data of the target part performing the operation action and uploads it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; Dividing each of the three-dimensional virtual posture images into image sets corresponding to the safety training scenario tasks and saving them; Performing feature engineering on the image set, and extracting three-dimensional operation posture features in each of the three-dimensional virtual posture images in the image set by using CNN; The three-dimensional operation posture feature is feature-labeled, and the labeling information includes: the task attribute and task level of the safety training scenario task and the corresponding posture operation requirements, wherein the posture operation requirements include the matching degree of the operation posture; Collect the three-dimensional operation posture features corresponding to the safety training scenario task to obtain a corresponding feature set; Dividing the feature set into a training set and a validation set according to a preset ratio; Inputting the training set into a preset CNN model to perform feature training and learning to generate the posture recognition model; The recognition performance of the gesture recognition model is verified using the verification set: If the verification is successful, the posture recognition model is deployed on the backend server; If the verification fails, the above steps are repeated to rebuild the gesture recognition model.

[0013] On the other hand, a safety training simulation rehearsal system based on digital twins is provided, and the safety training simulation rehearsal system based on digital twins is used to implement the above-mentioned safety training simulation rehearsal method based on digital twins, and the system includes: VR equipment is used to log in to the backend server and participate in the execution of safety training plot tasks; A 3D camera is used to collect three-dimensional posture image data of the user when performing the safety training scenario task and upload it to the backend server; The backend server is used to assign the safety training scenario task to the user; and map the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; identify the three-dimensional operation posture feature in the three-dimensional virtual posture image through a preset posture recognition model, and determine whether the three-dimensional operation posture feature meets the posture operation requirements corresponding to the safety training scenario task: If it is in compliance, the user's execution log of the safety training scenario task is recorded and saved; If not, a corresponding operation error notification is generated and sent to the VR device worn by the user; The VR device and the 3D camera are respectively connected to the backend server for communication.

[0014] On the other hand, an electronic device is provided, comprising: a processor; and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned digital twin-based safety training simulation rehearsal methods is implemented.

[0015] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned digital twin-based safety training simulation rehearsal methods.

[0016] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: The present invention constructs a virtual safety operation scene and several safety training scenario tasks under the virtual safety operation scene based on digital twin technology; collects three-dimensional posture image data when the user performs the safety training scenario task, and uploads it to a background server, and the background server maps the three-dimensional posture image data to the virtual safety operation scene to generate a corresponding three-dimensional virtual posture image; through a posture recognition model preset on the background server, identifies the three-dimensional operation posture features in the three-dimensional virtual posture image, and determines whether the three-dimensional operation posture features meet the posture operation requirements of the corresponding safety training scenario task: if they meet, record and save the user's execution log of the safety training scenario task; if they do not meet, generate a corresponding operation error notification and send it to the VR device worn by the user; repeat the above steps until the three-dimensional operation posture features of the user performing all the safety training scenario tasks meet the corresponding posture operation requirements. It can use digital twin technology to conduct simulated rehearsal learning of safety training for users, and can perform AI recognition of three-dimensional virtual postures in safety training, and judge whether the user's safety training operation actions meet the standards in the virtual environment. In this way, it can also evaluate the actual action postures of the physical model in its virtual system, achieve an evaluation effect equivalent to practicality, and allow trainees and other users to experience and assess virtual rehearsal of safety practical training, enhance the efficiency and significance of virtual training for safety operations, and improve trainees' online teaching experience and learning ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a schematic diagram of the application of the virtual system; Figure 2 It is a flow chart of a safety training simulation preview method based on digital twins provided by an embodiment of the present invention; Figure 3 It is a block diagram of a safety training simulation rehearsal system based on digital twins provided by an embodiment of the present invention; Figure 4 is a schematic diagram of a digital twin posture model of a hand provided by an embodiment of the present invention; Figure 5 is a flow chart of generating a posture recognition model provided by an embodiment of the present invention; Figure 6 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0020] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0021] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0022] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0023] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0024] The embodiment of the present invention provides a safety training simulation preview method based on digital twins, which can be implemented by an electronic device, which can be a terminal or a server. Figure 2 The flowchart of the safety training simulation preview method based on digital twins is shown. The processing flow of the method may include the following steps: S1. Based on digital twin technology, construct a virtual scene of safe operation and several safety training scenario tasks under the virtual scene of safe operation; S2, collecting three-dimensional posture image data when the user performs the safety training scenario task, and uploading it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; S3, identifying the three-dimensional operation posture features in the three-dimensional virtual posture image through the posture recognition model preset on the background server, and judging whether the three-dimensional operation posture features meet the posture operation requirements corresponding to the safety training scenario task: If it is in compliance, the user's execution log of the safety training scenario task is recorded and saved; If not, a corresponding operation error notification is generated and sent to the VR device worn by the user; S4. Repeat the above steps until the three-dimensional operation posture characteristics of the user performing all the safety training scenario tasks meet the corresponding posture operation requirements.

[0025] The present invention utilizes digital twin technology to conduct safety training simulation rehearsal learning for users, and can perform AI recognition of three-dimensional virtual postures in safety training, and judge whether the user's safety training operation actions meet the standards in the virtual environment. In this way, the function of evaluating the actual action posture of the physical model can be realized in its virtual system, achieving an evaluation effect equivalent to practicality, allowing trainees and other users to experience and assess the virtual rehearsal of safety practical training, and enhancing the efficiency and significance of virtual training for safety operations.

[0026] like Figure 3 As shown, it is an application system for implementing the present invention. Users can use VR devices, such as VR glasses and other action devices such as joysticks, etc., to participate in the corresponding virtual plot task operations, and users can log in to the background server through the VR device to perform corresponding training tasks and perform corresponding operations according to the plot. For example, they can perform training according to the safe operation actions demonstrated in the plot task. The 3D camera (or other visual camera) deployed nearby can collect the user's operation image, and upload the action image of the user when performing the task (specifically, the three-dimensional action image of the corresponding part, that is, the three-dimensional posture image data) to the background server, so as to map the three-dimensional posture action of the user's physical part based on the digital twin technology (physical The data mapping between data and virtual data can refer to the dynamic management principle of data set mapping of virtual reality described later) to the pre-built safe operation virtual scene, so as to generate the user's three-dimensional virtual posture image in the corresponding plot environment in the corresponding virtual scene, that is, the user's physical action is mapped to the virtual scene through digital twin technology to generate a three-dimensional virtual posture image, and then the posture recognition model constructed by the present invention is used in the image to capture the user's posture action in the virtual scene (it can be a frame image or extract a posture image of a certain operation node / time point. Note that the target object is a three-dimensional image, which can be collected from multiple spatial angles. Combined with the attached Figure 4 The virtual action image shown can be captured by the screenshot window of the server, or automatically captured by the image capturing component of the digital twin platform) and posture recognition can be performed to determine whether the user's virtual operation action meets the posture operation requirements specified in the current task. Therefore, during the virtual online training, students can have a practical virtual experience.

[0027] The principle of dynamic management of dataset mapping in virtual reality (VR) mainly involves real-time and dynamic mapping and interaction of real-world datasets with virtual environments. This principle can be broken down into the following key aspects: ‌Perception Data Collection‌: Use a variety of sensors (such as cameras, gyroscopes, accelerometers, etc.) to collect user dynamic information and environmental data in real time. These data include the user's location, posture, movement trajectory, and environmental lighting, sound, etc.

[0028] ‌Data preprocessing and mapping‌: The collected data needs to be preprocessed, such as denoising and calibration, to ensure the accuracy and reliability of the data.

[0029] The preprocessed data is mapped into the virtual environment and associated with the objects and scenes of the virtual world in real time. This mapping is dynamic and updated as the user and the environment change.

[0030] ‌Real-time rendering and feedback‌: Based on the mapping results, the virtual environment is rendered in real time to generate a virtual scene that matches the user's current state and environment.

[0031] The rendered virtual scene is presented to the user through a display device (such as a head-mounted display), and real-time feedback and adjustments are made based on the user's interactive behavior (such as head rotation, gestures, etc.).

[0032] Dynamic Management: Virtual reality systems need to constantly monitor changes in users and the environment and dynamically adjust dataset mapping and rendering strategies.

[0033] When the user or environment changes, the system can respond quickly and update the virtual scene to maintain the consistency of user immersion and interactive experience.

[0034] Through this principle, virtual reality technology can realize real-time and dynamic interaction between users and virtual environments, bringing users an immersive experience.

[0035] Preferably, the S1, based on the digital twin technology, constructs a virtual scene of safe operation and several safety training scenario tasks in the virtual scene of safe operation, including: The 3D structure image data of the physical model of the safety place is collected by a 3D camera deployed on site, and uploaded to the backend server; After receiving and preprocessing the three-dimensional structure image data, the backend server imports the data into a preset digital twin platform, performs three-dimensional modeling based on the digital twin technology, and constructs a digital twin site model corresponding to the physical model of the safety site; Record and bind the data set mapping relationship between the physical model of the safe place and the digital twin place model to construct the safe operation virtual scene; According to the preset safety operation process, a plurality of safety training scenario tasks corresponding to the safety operation process are constructed based on the safety operation virtual scene, and corresponding task attributes and task levels are configured for each of the safety training scenario tasks.

[0036] The architecture of the digital twin platform (or virtual system) generally used to build virtual scenes mainly includes: ‌Data collection layer‌: collects data of physical entities in real time through sensors, IoT devices, etc., such as device status, environmental parameters, etc.

[0037] ‌Digital Twin Layer‌: Use digital twin technology to create a virtual model of the physical entity and update it synchronously with the actual data.

[0038] ‌Simulation training layer‌: Based on the digital twin model, build simulation training scenarios to provide immersive training experience.

[0039] Analysis layer: record, analyze and evaluate trainees’ operational behaviors and generate assessment reports.

[0040] Specifically, the required virtual scene of safe operation can be built in combination with the existing digital twin platform or virtual reality system. The physical scene model of the virtual scene of safe operation can be determined by the enterprise, and the 3D camera in the scene can collect the three-dimensional structural image data (point cloud image) of the scene. The background platform processes the three-dimensional structural image data, performs three-dimensional modeling based on the digital twin technology, and constructs a digital twin site model corresponding to the physical model of the safe place. The administrator can further construct several safety training scenario tasks corresponding to the safety operation process based on the virtual scene of safe operation. The safety training scenario tasks under different training nodes can be constructed according to the safe production operation process. At the same time, the corresponding task attributes and task levels are configured for each of the safety training scenario tasks to facilitate task management and subsequent task matching and allocation.

[0041] For example, for underground electrical operations, you can set up the following safety training scenarios for each process node: Power outage operation sequence: first stop low voltage, then high voltage; first pull the load switch, then the power switch; first pull the oil switch, then the isolation switch; Power supply operation sequence: first supply high voltage, then low voltage; first close the power supply, then the load switch; first close the isolating switch, then the oil switch.

[0042] The plot tasks under each node specify the operating actions of the corresponding parts. The model recognizes and judges these action postures to determine whether the operation is in place.

[0043] Each safety training scenario task builds the corresponding scenario content according to the process management content of the corresponding process node. The corresponding scenario can be built in the form of audio, video, graphics, etc., and the existing virtual operation simulation teaching video can be referred to. The specific task content can be built by the administrator, and this office only provides the technical solution.

[0044] Based on this platform, it is possible to achieve: ‌Scene Construction‌: Build a variety of safety training scenarios based on actual needs, such as fire escape, equipment operation, etc. Use 3D modeling technology to highly restore the real environment and provide an immersive training experience.

[0045] ‌Operation Simulation‌: Through virtual reality (VR) or augmented reality (AR) technology, trainees can practice safe operations in a virtual environment. The system can simulate various security threats and emergency situations to test trainees' emergency response capabilities.

[0046] ‌Intelligent Guidance‌: The system can monitor the trainees’ operation behaviors in real time and provide instant feedback and guidance. When the trainees operate improperly or encounter difficulties, the system can provide the correct operation steps or solutions.

[0047] The dataset mapping relationship between the physical model of the safe place and the digital twin place model can be constructed by the platform based on the dataset generated by building the model. This will facilitate the platform to dynamically map and generate virtual scene models and manage datasets.

[0048] Preferably, the step S2, collecting three-dimensional posture image data when the user performs the safety training scenario task, and uploading it to a backend server, wherein the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image, includes: The user receives and executes the safety training scenario task distributed by the backend server through the VR device worn by the user; Through the 3D camera deployed on site, the 3D posture image data of the user's target part operating action when performing the safety training scenario task is collected in real time and uploaded to the backend server; The backend server receives and pre-processes the three-dimensional posture image data, imports it into a preset digital twin platform, and performs three-dimensional modeling based on the digital twin technology to construct a digital twin posture model corresponding to the target part; Recording and binding a data set mapping relationship between the digital twin posture model and the digital twin place model; The digital twin posture model is projected into the corresponding task scene in the safe operation virtual scene, and a corresponding three-dimensional virtual posture image is generated and synchronized to the VR device worn by the user.

[0049] Taking the "power supply operation sequence" operation as an example, the user operates the power supply switch with his hand, and his action can be collected by the 3D camera. The collected three-dimensional posture image data of the user's target part performing the operation action when performing the safety training scenario task will be uploaded to the background server.

[0050] When the user logs in to the background through the VR device and performs the corresponding safety training plot task, the target part will move, and the action posture will be displayed according to the operation required in the plot task. The on-site 3D camera can be used to collect the three-dimensional posture image data of the user's operation action, such as the hand, and upload it to the background server. In the background, the three-dimensional posture model of the hand action is constructed in real time based on the digital twin technology, and data mapping is performed. Therefore, the operation action of the user's physical part can be mapped to the corresponding plot scene, and the corresponding three-dimensional virtual posture image can be generated in the corresponding task scene. At the same time, the image is output to the VR device by the background, allowing the user to understand and view his current operation action through the VR device. At the same time, it can display its own safety operation action in the task scene with the plot task system. The system records the action at the same time, so as to achieve synchronization of virtual and real operations.

[0051] like Figure 4 As shown, when a trainee or other user performs a safe operation, a digital twin posture model corresponding to the target part is generated. The target part is mainly the hand. If there are operation requirements for other parts, other parts such as the head, shoulders, etc. can be continued to be constructed according to the present invention.

[0052] Preferably, the user receives and executes the safety training scenario task distributed by the background server through the VR device worn by the user, including: The user logs in to the backend server through the VR device he wears, and the backend server records and saves the user ID; The backend server generates a security operation question and sends it to the VR device; The user responds to the safety operation question through the VR device and provides corresponding answer information; The backend server identifies the degree of match between the safety operation question and the answer information, and assigns a safety training scenario task with a task level matching the degree of match to the user according to the degree of match, and sends the task to the VR device worn by the user; The user receives and executes the safety training scenario tasks through the VR device he wears, and makes corresponding feedback.

[0053] Before the user performs the corresponding plot task, the background can interact with the user and send the corresponding safety operation questions to the user from the pre-stored question-answer pairs in the database. The user answers and the corresponding answer information is fed back to the background through the VR device. The background determines the matching degree of the question-answer pair based on the question and the answer information fed back by the user (it can extract the keywords in the corresponding question-answer pair, generate the corresponding search instructions and search in the database, compare with the standard answer, calculate the correlation or similarity, and determine the matching degree), so as to identify and judge the current user's mastery of the corresponding safety operation knowledge. The background server assigns the user a safety training execution task of the corresponding task level by identifying the matching degree between the question and the answer information, and the corresponding relationship between the task level and the matching degree of the specific task can be configured by the administrator in advance on the background. For example, the medium task level corresponds to a matching degree between 70% and 80%. The correspondence between the difficulty of the specific task level and the user matching degree is set by the administrator.

[0054] In the present invention, a large language model API interface can also be deployed in the background, so that the background can call the large language model (LLM, such as calling Baidu's Wenxin Yiyan large model), automatically generate corresponding questions and calculate the matching degree of the answer information fed back by the user. The core of automatically matching the question and answer pairs with the user's answer information through the LLM large language model and judging the accuracy (matching degree) of the user's answer is to use the powerful natural language processing capabilities of LLM. The specific process can refer to the following steps: 1. Text understanding: LLM first understands the text content of questions and user answers, including semantics, context, and keywords.

[0055] 2. Feature extraction: Extract key features from questions and answers. These features may include entities, relationships, sentiment tendencies, etc., for subsequent matching and judgment.

[0056] 3. Question-answer pair matching: LLM’s text matching capability is used to match user answers with questions (this process can be done by searching the database, which has a large amount of question-answer pair knowledge pre-stored in it), and the relevance and accuracy of the answers are determined.

[0057] 4. Accuracy evaluation: Based on the matching results, LLM can give an accuracy evaluation of the user's answer, which may include multiple levels such as complete match, partial match, and mismatch.

[0058] Through this process, the LLM large language model can efficiently realize automatic matching of question and answer pairs and accurately judge the accuracy of user answers.

[0059] like Figure 5 As shown, preferably, the method for generating the posture recognition model includes: The administrator executes the safety training simulation rehearsal process, executes each of the safety training scenario tasks in turn, collects the three-dimensional posture image data of the target part performing the operation action and uploads it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; Dividing each of the three-dimensional virtual posture images into image sets corresponding to the safety training scenario tasks and saving them; Performing feature engineering on the image set, and extracting three-dimensional operation posture features in each of the three-dimensional virtual posture images in the image set by using CNN; The three-dimensional operation posture feature is feature-labeled, and the labeling information includes: the task attribute and task level of the safety training scenario task and the corresponding posture operation requirements, wherein the posture operation requirements include the matching degree of the operation posture; Collect the three-dimensional operation posture features corresponding to the safety training scenario task to obtain a corresponding feature set; Dividing the feature set into a training set and a validation set according to a preset ratio; Inputting the training set into a preset CNN model to perform feature training and learning to generate the posture recognition model; The recognition performance of the gesture recognition model is verified using the verification set: If the verification is successful, the posture recognition model is deployed on the backend server; If the verification fails, the above steps are repeated to rebuild the gesture recognition model.

[0060] The present invention utilizes an AI model to extract and identify the operation posture features of three-dimensional posture actions in virtual scenes, and utilizes a convolutional neural network to extract the three-dimensional operation posture features of the corresponding parts of the user, so as to identify and output the identification information of the corresponding features. The model is used to determine whether the current user's virtual operation action meets the required operation requirements (the requirements for action features are mainly based on the marked posture operation requirements to determine whether the trainee's action features are consistent with the features learned by the model).

[0061] The construction of the posture recognition model of the present invention requires the use of big data and three-dimensional virtual posture images under several corresponding plot tasks. When the user performs virtual operation interactions in safety training, it is necessary to collect three-dimensional images of the corresponding virtual operations as a standard posture action image data set. In the present invention, the administrator conducts a safety training simulation preview in advance, and executes each safety training plot task in sequence according to the above-mentioned process of the present invention. The 3D camera then collects the three-dimensional posture image data of the corresponding parts and uploads it to the background server, and generates a three-dimensional virtual posture image of the corresponding parts in the background. The three-dimensional virtual posture image, as an image data set of standard operation actions, will be used to train the model so that the model can recognize the standard operation posture features under each safety training task.

[0062] Therefore, the collected images are used for feature training, and the constructed posture recognition model can label, identify, and compare the features of the operating postures under each plot task.

[0063] When students and other users perform virtual operations later, the posture recognition model can identify the operation posture features of the students' three-dimensional virtual posture images to determine whether the operation posture of the corresponding parts of the students conforms to the dynamic operation posture features learned by the model. If it conforms, the student's operation process will continue to be monitored; if it does not conform, the corresponding operation error reminder will be generated (these processes will be synchronized from the background to the VR device, so as to remind students to pay attention to the corresponding operation error reminders in real time).

[0064] As for the network structure of the convolutional neural network and the model training and verification process of the convolutional neural network, you can refer to the application principles and implementation process of the existing CNN model, which will not be described in detail in this example.

[0065] The corresponding administrator rehearsal process can be performed by the administrator to collect and preprocess the corresponding standard action posture image data set. Feature annotation can be used to annotate the task attributes and task levels of the safety training scenario tasks to which each feature belongs. By annotating this information, the model can subsequently identify and output the corresponding feature information of the trainee's current virtual operation action posture.

[0066] When using the posture recognition model to identify and extract the three-dimensional operation posture features in the student's three-dimensional virtual posture image, the feature similarity between the feature and the feature learned by the model can be further calculated. For example, the cosine theorem can be used to calculate the feature similarity, so as to calculate whether the student's operation posture features meet the matching degree required in the marked posture operation requirements (that is, the similarity threshold, which needs to determine the minimum value that the similarity between the student's action posture features and the model learning features must reach, such as 70%).

[0067] The training and construction steps of the CNN (Convolutional Neural Network) model include the following key parts: ‌Data preparation and preprocessing‌: ‌Data Collection‌: Collect sufficient training data and validation / testing data from reliable sources.

[0068] ‌Data preprocessing‌: Perform preprocessing operations such as normalization, cropping, scaling, rotation, and flipping on image data to unify the data format and enhance the generalization ability of the model.

[0069] ‌Data partitioning‌: Divide the dataset into training set, validation set, and test set (if the data volume is large enough). Here, the feature set can be divided into training set and validation set in an 8:2 ratio.

[0070] ‌Model definition‌: ‌Select a framework‌: Choose a suitable deep learning framework, such as TensorFlow, PyTorch, etc.

[0071] ‌Build network structure‌: Define the hierarchical structure of CNN, including input layer, convolution layer, activation layer (such as ReLU), pooling layer (such as maximum pooling), fully connected layer, output layer, etc.

[0072] ‌Set parameters‌: Set parameters for each layer, such as convolution kernel size, step size, padding method, activation function type, pooling window size, number of neurons in the fully connected layer, etc.

[0073] ‌Loss function and optimizer‌: ‌Select loss function‌: Select an appropriate loss function according to the task type (such as classification, regression), such as cross entropy loss, mean square error, etc.

[0074] ‌Select Optimizer‌: Select an optimization algorithm, such as SGD, Adam, RMSprop, etc., and set hyperparameters such as learning rate and momentum.

[0075] ‌Model Training‌: ‌Data loading‌: Use a data loader (such as DataLoader) to load the training data (training set) into memory in batches (so as to load the training sets obtained under each task).

[0076] ‌Forward propagation‌: Calculate the model's prediction value on the current batch of data.

[0077] ‌Calculate loss‌: Use the loss function to calculate the difference between the predicted value and the true value.

[0078] ‌Backward Propagation‌: Calculate the gradient based on the loss value and update the model parameters through the optimizer.

[0079] ‌Iterative training‌: Repeat the above steps until the predetermined training rounds are reached or other stopping conditions are met.

[0080] ‌Monitor the training process‌: Record the loss value, accuracy and other indicators during the training process, and use the validation set to evaluate the model performance.

[0081] ‌Model Validation and Evaluation‌: Validation model: Evaluate the performance of the model on the validation set to check whether the model is overfitting or underfitting. Specifically, the F1 score can be used for performance validation. The F1 score is widely used to evaluate the performance of segmentation models, especially for tasks such as tumor detection. The F1 score can comprehensively consider the accuracy and recall rate, helping doctors to have a more comprehensive understanding of the performance of the model.

[0082] ‌Adjust hyperparameters‌: Adjust hyperparameters such as model structure, learning rate, batch size, etc. based on the validation results, and retrain the model.

[0083] Model deployment and optimization: ‌Model Export‌: Export the trained model to a deployable format, such as TensorFlow SavedModel, PyTorch model file, etc.

[0084] ‌Model Optimization‌: Perform optimization operations such as quantization and pruning on the model to reduce model size and inference time.

[0085] ‌Deploy model‌: Deploy the model to the target platform (such as server, mobile device, embedded device, etc.) and apply it in practice.

[0086] The above are the general steps for building a CNN model. The specific implementation may vary depending on task requirements, data characteristics, and hardware conditions.

[0087] Therefore, during the user learning process, the system can monitor the trainees' operation behavior in real time and provide immediate feedback and guidance. When the trainees operate improperly or encounter difficulties, the system can provide correct operation steps or solutions. The system will output corresponding operation error reminders and generate corresponding records. Later, trainees can view the records and re-learn the recorded plot positions to improve their impressions. After the learning is completed, the system can generate an analysis report of this training (recording the trainees' learning status and record information under each task, etc.).

[0088] For the assessment tasks, you can also perform the corresponding safety assessment plot tasks according to the above plan.

[0089] On the other hand, a safety training simulation rehearsal system based on digital twins is provided, and the safety training simulation rehearsal system based on digital twins is used to implement the above-mentioned safety training simulation rehearsal method based on digital twins, and the system includes: VR equipment is used to log in to the backend server and participate in the execution of safety training plot tasks; A 3D camera is used to collect three-dimensional posture image data of the user when performing the safety training scenario task and upload it to the backend server; The backend server is used to assign the safety training scenario task to the user; and map the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; identify the three-dimensional operation posture feature in the three-dimensional virtual posture image through a preset posture recognition model, and determine whether the three-dimensional operation posture feature meets the posture operation requirements corresponding to the safety training scenario task: If it is in compliance, the user's execution log of the safety training scenario task is recorded and saved; If not, a corresponding operation error notification is generated and sent to the VR device worn by the user; The VR device and the 3D camera are respectively connected to the backend server for communication.

[0090] The specific composition and interaction of the above system should be understood in conjunction with the principles and interactions described in the above method, and will not be described in detail here.

[0091] Figure 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device 410 may include a first processor 2001 .

[0092] Optionally, the electronic device 410 may further include a memory 2002 and a transceiver 2003 .

[0093] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0094] Combine the following Figure 6 The components of the electronic device 410 are described in detail: The first processor 2001 is the control center of the electronic device 410, and may be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be application specific integrated circuits (ASICs), or may be one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0095] Optionally, the first processor 2001 can execute various functions of the electronic device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0096] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 CPU0 and CPU1 are shown in FIG.

[0097] In a specific implementation, as an embodiment, the electronic device 410 may also include multiple processors, such as Figure 6 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0098] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0099] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently and access the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0100] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0101] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0102] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and communicate with the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0103] It should be noted that Figure 6 The structure of the electronic device 410 shown in the figure does not constitute a limitation on the router, and the actual knowledge structure recognition device may include more or fewer components than those shown in the figure, or combine certain components, or arrange the components differently.

[0104] In addition, the technical effects of the electronic device 410 can refer to the technical effects of the digital twin-based safety training simulation rehearsal method described in the above method embodiment, which will not be repeated here.

[0105] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0107] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable system. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0108] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0109] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0110] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0111] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0112] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0113] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the system or unit, which can be electrical, mechanical or other forms.

[0114] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0115] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0116] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0117] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A safety training simulation rehearsal method based on digital twins, characterized in that: The method comprises: S1. Based on digital twin technology, construct a virtual scene of safe operation and several safety training plot tasks under the virtual scene of safe operation; S2, collecting three-dimensional posture image data when the user performs the safety training scenario task, and uploading it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; S3, identifying the three-dimensional operation posture features in the three-dimensional virtual posture image through the posture recognition model preset on the background server, and judging whether the three-dimensional operation posture features meet the posture operation requirements corresponding to the safety training scenario task: If it is in compliance, the execution log of the user on the safety training scenario task is recorded and saved; If not, a corresponding operation error notification is generated and sent to the VR device worn by the user; S4. Repeat the above steps until the three-dimensional operation posture characteristics of the user performing all the safety training scenario tasks meet the corresponding posture operation requirements.

2. The digital twin-based safety training simulation rehearsal method according to claim 1 is characterized in that: S1, based on digital twin technology, constructs a virtual scene of safe operation and several safety training scenario tasks under the virtual scene of safe operation, including: The 3D structure image data of the physical model of the safety place is collected by a 3D camera deployed on site, and uploaded to the backend server; The backend server receives and pre-processes the three-dimensional structure image data, imports it into a preset digital twin platform, and performs three-dimensional modeling based on the digital twin technology to construct a digital twin site model corresponding to the physical model of the safety site; Record and bind the data set mapping relationship between the physical model of the safe place and the digital twin place model to construct the safe operation virtual scene; According to the preset safety operation process, a plurality of safety training scenario tasks corresponding to the safety operation process are constructed based on the safety operation virtual scene, and corresponding task attributes and task levels are configured for each of the safety training scenario tasks.

3. The digital twin-based safety training simulation preview method according to claim 1 is characterized in that: S2, collecting three-dimensional posture image data when the user performs the safety training scenario task, and uploading it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image, including: The user receives and executes the safety training scenario task distributed by the backend server through the VR device worn by the user; Through the 3D camera deployed on site, the 3D posture image data of the user's target part operating action when performing the safety training scenario task is collected in real time and uploaded to the backend server; The backend server receives and pre-processes the three-dimensional posture image data, imports it into a preset digital twin platform, and performs three-dimensional modeling based on the digital twin technology to construct a digital twin posture model corresponding to the target part; Recording and binding a data set mapping relationship between the digital twin posture model and the digital twin place model; The digital twin posture model is projected into the corresponding task scene in the safe operation virtual scene, and a corresponding three-dimensional virtual posture image is generated and synchronized to the VR device worn by the user.

4. The digital twin-based safety training simulation preview method according to claim 3 is characterized in that: The user receives and executes the safety training scenario task distributed by the background server through the VR device worn by the user, including: The user logs in to the backend server through the VR device he wears, and the backend server records and saves the user ID; The backend server generates a security operation question and sends it to the VR device; The user responds to the safety operation question through the VR device and provides corresponding answer information; The backend server identifies the degree of match between the safety operation question and the answer information, and assigns a safety training scenario task with a task level matching the degree of match to the user according to the degree of match, and sends the task to the VR device worn by the user; The user receives and executes the safety training scenario tasks through the VR device he wears, and makes corresponding feedback.

5. The safety training simulation rehearsal method based on digital twin according to claim 1 is characterized in that: The method for generating the posture recognition model comprises: The administrator executes the safety training simulation rehearsal process, executes each of the safety training scenario tasks in turn, collects the three-dimensional posture image data of the target part performing the operation action and uploads it to the backend server, and the backend server maps the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; Dividing each of the three-dimensional virtual posture images into image sets corresponding to the safety training scenario tasks and saving them; Performing feature engineering on the image set, and extracting three-dimensional operation posture features in each of the three-dimensional virtual posture images in the image set by using CNN; The three-dimensional operation posture feature is marked, and the marking information includes: the task attribute and task level of the safety training scenario task and the corresponding posture operation requirements, wherein the posture operation requirements include the matching degree of the operation posture; Collect the three-dimensional operation posture features corresponding to the safety training scenario task to obtain a corresponding feature set; Dividing the feature set into a training set and a validation set according to a preset ratio; Inputting the training set into a preset CNN model to perform feature training and learning to generate the posture recognition model; The recognition performance of the gesture recognition model is verified using the verification set: If the verification is successful, the posture recognition model is deployed on the backend server; If the verification fails, the above steps are repeated to rebuild the gesture recognition model.

6. A safety training simulation rehearsal system based on digital twins, wherein the safety training simulation rehearsal system based on digital twins is used to implement the safety training simulation rehearsal method based on digital twins as described in any one of claims 1 to 5, characterized in that: The system comprises: VR equipment is used to log in to the backend server and participate in the execution of safety training plot tasks; A 3D camera is used to collect three-dimensional posture image data of the user when performing the safety training scenario task and upload it to the backend server; The backend server is used to assign the safety training scenario task to the user; and map the three-dimensional posture image data to the safety operation virtual scene to generate a corresponding three-dimensional virtual posture image; identify the three-dimensional operation posture feature in the three-dimensional virtual posture image through a preset posture recognition model, and determine whether the three-dimensional operation posture feature meets the posture operation requirements corresponding to the safety training scenario task: If it is in compliance, the execution log of the user on the safety training scenario task is recorded and saved; If not, a corresponding operation error notification is generated and sent to the VR device worn by the user; The VR device and the 3D camera are respectively connected to the background server for communication.

7. An electronic device, characterized in that: The electronic device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 5.

Images