Power industry safety training method and system based on virtual digital human technology

Abstract

The invention discloses a power industry safety training method and system based on a virtual digital human technology. The method comprises the steps of obtaining safety training requirements of the power industry, and collecting safety training related text data of the power industry; performing arrangement and data cleaning on the text data, and constructing a structure, a field and a relationship of a training content database table; creating a virtual digital human model, recording user actions and mapping the user actions to the virtual digital human model; establishing a power industry safety training speech synthesis library, and adopting an integrated natural language processing model to enable a virtual digital person to understand user input and generate feedback; modeling a training scene, designing an operation interaction logic consistent with an actual operation, and displaying an electric power industry safety operation training course to trainees in a virtual reality environment through real-time interaction design of a virtual digital human; and asking safety knowledge points to the trainee through the virtual digital person, and evaluating an answer result of the trainee for scoring. According to the invention, the training pertinence and effect are improved.

Description

Technical Field

[0001] The present invention belongs to the field of virtual digital human, and in particular relates to a safety training method and system for the electric power industry based on virtual digital human technology. Background Art

[0002] Virtual Digital Humans (VDH) are the product of the rapid development of artificial intelligence technology in recent years. They are created and used through computer means such as computer graphics, graphics rendering, motion capture, deep learning and speech synthesis, and have multiple human characteristics (such as appearance, performance and interaction capabilities). The core characteristics of virtual digital humans are highly realistic appearance and behavior, as well as natural and smooth interaction with human users. The emergence of virtual digital humans has brought new development opportunities for the digital transformation of the power industry.

[0003] With the rapid development of information technology, the power industry is facing an important stage of digital transformation. This process not only includes the intelligent upgrade of power grid infrastructure, but also includes the informatization and intelligent transformation of the entire process of power production, transmission, distribution, and consumption. There are some shortcomings in the existing technology: on the one hand, the training method is single and lacks interactivity. At present, some power industry safety training still adopts the traditional classroom teaching method, which lacks interactivity and fun, and it is difficult to stimulate the learning interest and enthusiasm of trainees. Although some training systems have introduced virtual simulation technology, the degree of simulation is not high, and it is impossible to fully simulate the real power operation environment, resulting in limited training effects. On the other hand, the degree of intelligence of the training system is not high. Some training systems lack personalized learning support and cannot provide customized training content and learning paths according to the actual situation and needs of trainees. The training system lacks an intelligent evaluation and feedback mechanism, and cannot accurately evaluate the learning effect of trainees, nor can it provide targeted improvement suggestions. In this context, virtual digital people, as one of the important applications of artificial intelligence technology, have begun to play an important role in the power industry. Summary of the invention

[0004] In order to solve the deficiencies in the prior art, the present invention provides a power industry safety training method and system based on virtual digital human technology to solve the technical problem of improving the effect of safety training in the power industry.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions.

[0006] The present invention first discloses a safety training method for the electric power industry based on virtual digital human technology, which comprises the following steps:

[0007] Step 1: Obtain the safety training needs of the power industry, determine the key areas and goals of safety training, and collect text data related to safety training in the power industry;

[0008] Step 2: Classify and clean the text data related to safety training in the power industry, and build the structure, fields and relationships of the training content database table;

[0009] Step 3: Create a 3D model of a virtual digital human, use a full-body motion capture system to record user movements and map them to the 3D model of the virtual digital human, and construct facial expressions of the virtual digital human; make detailed adjustments and optimize the performance of the virtual digital human's movements and expression animations, use a speech synthesis model to establish a speech synthesis library for power industry safety training, and use an integrated natural language processing model to enable the virtual digital human to understand user input and generate corresponding feedback;

[0010] Step 4: Model the training scenario using 3D modeling software, design the operation interaction logic consistent with the actual operation, and present the power industry safety operation training course to the trainees in a virtual reality environment through the real-time interaction design of the virtual digital human;

[0011] Step 5: During the training process, a virtual digital human is used to ask trainers questions about safety knowledge points, and the trainers’ answers are evaluated and scored. The training progress is adjusted based on the trainers’ scores.

[0012] The present invention further includes the following preferred embodiments:

[0013] The step of creating a virtual digital human 3D model further comprises:

[0014] Use 3D modeling software Blender to create a 3D model of the virtual digital human, including its body, facial features, hairstyle, and clothing; use Blender's advanced modeling tools to adjust the shape, proportions, and details of the virtual digital human.

[0015] The step of constructing the facial expression of a virtual digital human further comprises:

[0016] Use the facial expression capture technology Faceware to build the facial expressions and lip shapes of virtual digital humans, clean and organize the captured motion data, remove unnatural jitters and erroneous data, and bind the captured motion and expression data to the 3D model of the virtual digital human. Use the keyframe animation and dynamic simulation technology in the 3D animation software Maya to adjust the movements and expressions of the virtual digital human.

[0017] The method of using the speech synthesis model to establish a speech synthesis library for power industry safety training further includes:

[0018] The Tacotron 2 model is used for speech synthesis. First, a speech database for training is established by collecting public Chinese speech data sets and recording relevant power industry training speech and text transcription. The speech data sets are preprocessed by segmenting audio, extracting Mel spectrograms, and creating acoustic features.

[0019] Select WaveNet vocoder, set the learning rate, training rounds, and validation interval training parameters, use the mean square error (MSE) loss function and optimizer, and the MSE loss function calculates the average of the squares of the differences between all predicted values ​​and the true values;

[0020] The speech database is divided into test data and training data at a ratio of 2:8. The training data is input into speech model training to obtain the power industry safety training speech synthesis library.

[0021] The integrated natural language processing model is used to enable the virtual digital human to understand the user input and generate corresponding feedback, further comprising:

[0022] Collect text data of conversations in different languages ​​and contexts; establish a natural language text library, use the recurrent neural network (RNN) model to learn the statistical characteristics of the language, adopt the attention mechanism to make the model focus on the key parts of the input text, and train the model to recognize and understand the implicit meaning, puns, and slang in the context; use the accuracy index, recall rate index, and F1 score index to evaluate the performance of the model and further optimize the model.

[0023] The present invention also discloses a power industry safety training system based on virtual digital human technology using the aforementioned power industry safety training method based on virtual digital human technology, comprising:

[0024] The training content acquisition module is used to obtain the safety training needs of the power industry, determine the key areas and goals of safety training, and collect text data related to safety training in the power industry;

[0025] The database construction module is used to classify and clean the text data related to safety training in the power industry, and to build the structure, fields and relationships of the training content database table;

[0026] The virtual digital human creation module is used to create a virtual digital human 3D model, use a full-body motion capture system to record user movements and map them to the virtual digital human 3D model, and construct the virtual digital human's facial expressions; make detailed adjustments and performance optimizations to the virtual digital human's movements and expression animations, use a speech synthesis model to establish a speech synthesis library for power industry safety training, and use an integrated natural language processing model to enable the virtual digital human to understand user input and generate corresponding feedback;

[0027] The training course display module is used to model the training scene using 3D modeling software, design the operation interaction logic consistent with the actual operation, and display the power industry safety operation training course to the trainees in a virtual reality environment through the real-time interaction design of the virtual digital human;

[0028] The evaluation and feedback module is used to ask trainers questions about safety knowledge points through virtual digital humans during the training process, evaluate the trainers' answers and give them scores, and adjust the training progress based on the trainers' scores.

[0029] Accordingly, the present application also discloses a terminal, including a processor and a storage medium;

[0030] The storage medium is used to store instructions;

[0031] The processor is used to operate according to the instructions to execute the steps of the aforementioned power industry safety training method based on virtual digital human technology.

[0032] Correspondingly, the present application also discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the aforementioned power industry safety training method based on virtual digital human technology are implemented.

[0033] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method and system for safety training in the power industry based on virtual digital human technology. Based on the virtual digital human technology, a computer is used to create a virtual character with human characteristics, which is combined with virtual reality (VR), augmented reality (AR) and other technologies to provide trainers with a risk-free virtual environment to simulate various operations and emergency handling processes. Through interaction with virtual digital humans, trainers can have a deeper understanding of knowledge such as power system operation, equipment operation and accident handling. In addition, virtual digital humans can also provide personalized training content and methods according to the learning progress and characteristics of trainers, which not only improves the pertinence and effect of training, but also reduces the cost and risk of safety training in the power industry. With the continuous development and improvement of technology, it is expected to play a greater role in the field of safety training in the power industry in the future.

[0034] In short, with the continuous development and maturity of virtual digital human technology, the role of virtual digital human technology in safety training in the power industry is becoming increasingly important. It can provide highly realistic virtual environments and roles, enhance the learning experience, reduce training risks, and provide personalized teaching based on the characteristics of trainees, effectively improving training results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a flow chart of the power industry safety training method based on virtual digital human technology in the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0037] The embodiments described in this application are only some embodiments of the present invention, not all embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without creative work are all within the protection scope of the present invention.

[0038] In view of the shortcomings of the prior art, the present invention proposes a safety training method and system for the power industry based on virtual digital human technology. Figure 1 As shown, the power industry safety training method based on virtual digital human technology disclosed in the present invention includes the following steps:

[0039] Step 1: Obtain the safety training needs of the power industry, determine the key areas and objectives of safety training, and collect text data related to safety training in the power industry.

[0040] In this embodiment, the safety training needs of the power industry are first obtained, and the key areas and goals of safety training are determined. Specifically:

[0041] Obtain safety management methods for the power industry, analyze safety regulations, standards and requirements for the power industry, and ensure that all training content complies with the latest regulatory updates and industry standards;

[0042] Implement on-site risk assessment and management, assess specific risks in the workplace, including physical risks, health and safety risks, etc., show trainees risk assessment and management methods, and how to implement risk reduction measures.

[0043] Obtain emergency plans, show trainees the response procedures and emergency measures in emergency situations, and ensure that trainees understand how to use emergency equipment such as fire extinguishers, evacuation routes, etc.

[0044] Demonstrate a safety culture learning atmosphere to trainers and improve their safety awareness and behavior.

[0045] Show the trainees cases of enterprise production safety accidents, analyze the causes, processes and consequences of the accidents, summarize the lessons learned, and enhance the trainees' understanding and awareness of prevention of safety accidents.

[0046] Among them, the collection of text data related to safety training in the power industry further includes the collection of "Power Industry Safety Training Specifications", "Fire Protection Specifications for Thermal Power Plant and Substation Design", "Power Safety Production Supervision and Management Measures", "Power Industry Hazardous Chemicals Safety Management Regulations", safety operating procedures, emergency plans, safety training materials, etc. formulated by power companies based on their actual conditions, case analysis of safety accidents in the power industry, accident investigation reports, etc.

[0047] Step 2: Classify and clean the text data related to safety training in the power industry, and build the structure, fields and relationships of the training content database table.

[0048] First, the text data related to safety training in the power industry collected in step 1 is classified and sorted, and manual classification and data cleaning are performed according to training topics, training objects, training content, etc. to ensure the accuracy and timeliness of the data. Specifically:

[0049]

[0050]

[0051] Then, design the database structure and determine the database table structure, fields and relationships. Specifically:

[0052] In this embodiment, the open source database MySQL is used, and the specific table structure, fields and the relationship between them are as follows:

[0053] 1. User table (users)

[0054] user_id (INT, primary key)

[0055] username(VARCHAR)

[0056] password(VARCHAR)

[0057] full_name(VARCHAR)

[0058] role(ENUM('admin','trainer','trainee'))

[0059] created_at(DATETIME)

[0060] updated_at(DATETIME)

[0061] 2. Training topic table (training_topics)

[0062] topic_id (INT, primary key)

[0063] topic_name(VARCHAR)

[0064] description(TEXT)

[0065] created_at(DATETIME)

[0066] updated_at(DATETIME)

[0067] 3. Training content table (training_content)

[0068] content_id (INT, primary key)

[0069] topic_id (INT, foreign key, associated to the training topic table)

[0070] content_title(VARCHAR)

[0071] content_description(TEXT)

[0072] content_type(ENUM('text','video','audio','document'))

[0073] created_at(DATETIME)

[0074] updated_at(DATETIME)

[0075] 4. Training records table (training_records)

[0076] record_id (INT, primary key)

[0077] user_id (INT, foreign key, associated to the user table)

[0078] content_id (INT, foreign key, associated to the training content table)

[0079] completion_status(ENUM('completed','in_progress','not_started'))

[0080] completion_date(DATETIME)

[0081] feedback(TEXT)

[0082] created_at(DATETIME)

[0083] 5. Regulations and standards table (regulations_standards)

[0084] reg_std_id(INT,primary key)

[0085] title(VARCHAR)

[0086] description(TEXT)

[0087] created_at(DATETIME)

[0088] updated_at(DATETIME)

[0089] 6. Training materials table (training_materials)

[0090] material_id (INT, primary key)

[0091] reg_std_id (INT, foreign key, associated to the regulatory standards table)

[0092] material_title(VARCHAR)

[0093] material_description(TEXT)

[0094] material_type(ENUM('document','video','audio'))

[0095] created_at(DATETIME)

[0096] updated_at(DATETIME)

[0097] relation:

[0098] There is a many-to-many relationship between the training topic table and the training content table, which can be associated through an intermediate table.

[0099] There is a one-to-many relationship between the user table and the training record table. Each user can have multiple training records.

[0100] There is a one-to-many relationship between the training content table and the training record table. Each training content can correspond to multiple training records.

[0101] There is a one-to-many relationship between the regulatory standards table and the training materials table. Each regulatory standard can correspond to multiple training materials.

[0102] Finally, the query and retrieval functions of the database are designed to facilitate the rapid search and application of training content. In this embodiment, indexes are created for fields that are frequently used for query, such as training subject name, training content title, regulatory standard title, etc., and SQL query statements are used to write efficient SQL query statements, using WHERE clauses, JOIN operations, LIKE operations, etc. to accurately or fuzzily retrieve data, and using LIMIT clauses to limit the number of records returned to improve performance.

[0103] Step 3: Create a 3D model of a virtual digital human, use a full-body motion capture system to record user movements and map them to the 3D model of the virtual digital human, and construct facial expressions of the virtual digital human; make detailed adjustments and optimize the performance of the virtual digital human's movements and expression animations, use a speech synthesis model to establish a speech synthesis library for power industry safety training, and use an integrated natural language processing model to enable the virtual digital human to understand user input and generate corresponding feedback.

[0104] In this embodiment, the 3D modeling software Blender is used to create a 3D model of a virtual digital human. Blender is an open source 3D creation suite that includes tools such as modeling, sculpting, rendering, animation, materials, mapping, binding, particles, dynamics, post-processing, and audio processing. In Blender, a detailed 3D model of a virtual digital human can be created, including its body, facial features, hairstyle, clothing, etc. By using Blender's advanced modeling tools, the shape, proportion, and details of the virtual digital human are precisely adjusted to ensure that its appearance conforms to the image of a safety training instructor. The full-body motion capture system OptiTrack is used to record the movements of real actors. OptiTrack is a high-precision optical motion capture system that can capture the actor's body movements in real time and accurately map these motion data to the 3D model of the virtual digital human. Through OptiTrack, every subtle movement of the actor, including gestures, gaits, jumps, etc., can be captured, making the movements of the virtual digital human more smooth and natural. The facial expression capture technology Faceware is used to build the facial expressions of the virtual digital human. Faceware is an advanced facial capture solution that can capture the facial movements and expressions of the actor and convert these data into facial animations of the virtual digital human. Through Faceware, we can capture the actor's smile, frown, blink and other expressions, and accurately simulate the virtual digital human's mouth shape, making the virtual digital human's facial expression more vivid and realistic. Clean and organize the captured action data to remove unnatural jitters and erroneous data. At the same time, bind the captured action and expression data to the virtual digital human's 3D model to ensure that it can perform these actions and expressions correctly. By using the keyframe animation technology and dynamic simulation in the 3D animation software Maya, the virtual digital human's actions and expressions can be carefully adjusted to ensure that they are natural and smooth.

[0105] Finally, the virtual human’s motion and expression animations are adjusted in detail and performance optimized. This includes careful adjustments to the virtual human’s facial expressions to ensure that it can accurately express a variety of emotions. At the same time, it is necessary to ensure that the motion and expression animations can run smoothly on the target hardware platform.

[0106] After the appearance and movements of the virtual digital human are completed, speech synthesis technology is integrated to enable the virtual digital human to have natural and fluent voice communication with users, thereby increasing the participation of trainees and helping them better understand and master knowledge.

[0107] In this embodiment, the Tacotron 2 model is used for speech synthesis. Tacotron 2 is an end-to-end neural network for converting text into high-quality audio waveforms. It is an improved version of the Tacotron model, providing better sound quality and faster training speed. First, by collecting public Chinese speech data sets and recording relevant power industry training speech and text transcription, a speech database for training is established, and the speech data set is subjected to data preprocessing operations such as audio segmentation, Mel spectrum extraction, and acoustic feature creation.

[0108] Select WaveNet vocoder, set training parameters such as learning rate, training rounds, and validation interval, and use the mean square error (MSE) loss function and optimizer. The MSE loss function calculates the average of the squares of the differences between all predicted values ​​and the true values. The MSE loss function can help the model learn the key parameters for generating high-quality audio in speech synthesis training, improving the accuracy and robustness of speech synthesis.

[0109] Finally, the speech database is divided into test data and training data in a ratio of 2:8, and the training data is input into speech model training to obtain the power industry safety training speech synthesis library.

[0110] Among them, integrated natural language processing (NLP) technology is used to enable the virtual digital human to understand user input and generate corresponding feedback. The specific steps are:

[0111] Collect a large amount of text data, including conversations in different languages ​​and contexts. Build a natural language text library and use the recurrent neural network (RNN) model to learn the statistical characteristics of the language. Use the attention mechanism to let the model focus on the key parts of the input text, and train the model to recognize and understand the implicit meaning, puns, slang, etc. in the context. Use the accuracy index, recall rate index, and F1 score index to evaluate the performance of the model and further optimize the model.

[0112] Step 4: Use 3D modeling software to model the training scenario, design an operation interaction logic that is consistent with actual operations, and present the power industry safety operation training course to trainers in a virtual reality environment through the real-time interaction design of the virtual digital human.

[0113] In this embodiment, virtual reality (VR) technology is used for power industry safety training. Virtual reality (VR) technology can provide an immersive learning experience to help trainers learn power safety knowledge in a safe environment.

[0114] Taking the fire emergency response course as an example, 3dsMax 3D modeling software is used to model the fire scene and determine the elements that need to be interactive in the scene, such as fire hydrants, fire extinguishers, fire alarms, emergency exit signs, gas masks, etc.

[0115] Design the operation interaction logic to ensure consistency with actual operations, so that trainers can learn the correct operation procedures in a virtual environment.

[0116] Taking fire extinguishers as an example, trainers first need to find the right fire extinguisher in the virtual environment, including checking the type of fire extinguisher (such as dry powder, carbon dioxide, etc.) and the pressure indicator. For different types of fires, trainers need to select the right fire extinguisher for fire extinguishing operations.

[0117] Trainers need to simulate the correct way to hold a fire extinguisher, usually by holding the handle with both hands to keep the extinguisher steady.

[0118] Trainees need to find the safety pin on the fire extinguisher and simulate the action of pulling it out.

[0119] Trainees need to aim the nozzle of the fire extinguisher at the fire source and maintain an appropriate distance, usually 3-5 meters.

[0120] The trainees need to simulate the action of pressing down the fire extinguisher handle to activate the fire extinguisher.

[0121] The system simulates a fire extinguisher spraying dry powder or gas to put out the fire.

[0122] When using virtual reality (VR) technology for safety training in the power industry, virtual digital humans are demonstrated in a fire environment, making it easier for trainees to understand and operate.

[0123] Trainers can interact with virtual digital humans in real time in a virtual environment to obtain corresponding guidance and help, such as asking how to determine the cause of a fire and how to extinguish it.

[0124] By designing a multi-person collaborative VR scenario, trainees need to work with other virtual digital human characters to deal with fires, simulating communication and collaboration in actual work, training personnel's teamwork and communication skills when facing fires, and improving their emergency response capabilities in fire situations.

[0125] In this embodiment, typical safety accident cases in the power industry are selected, and the virtual digital human explains the process, consequences and cause analysis of the accident, and guides the trainees on how to avoid the same safety accidents in similar situations, thereby improving the safety awareness of the trainees.

[0126] Step 5: During the training process, a virtual digital human is used to ask trainers questions about safety knowledge points, and the trainers’ answers are evaluated and scored. The training progress is adjusted based on the trainers’ scores.

[0127] In this embodiment, a training scoring mechanism is established. During the training process, the virtual digital human asks the trainees questions about safety knowledge points. After judging whether the questions are correct or not, the training effect is scored to ensure the training effect. The training progress is adjusted in real time according to the training effect. For trainees with higher scores, the training progress can be accelerated, the difficulty and depth can be increased. For trainees with lower scores, the training progress can be slowed down to strengthen the learning and consolidation of basic knowledge. For interactive training, points are added or subtracted according to the trainees' emergency response methods, and improper handling is recorded to provide feedback and improvement suggestions for the trainees. After the training course, the training results are fed back based on the trainees' comprehensive performance.

[0128] The beneficial effect of the present invention is that, compared with the prior art, the present invention provides a method and system for safety training in the power industry based on virtual digital human technology. Based on the virtual digital human technology, a computer is used to create a virtual character with human characteristics, which is combined with virtual reality (VR), augmented reality (AR) and other technologies to provide trainers with a risk-free virtual environment to simulate various operations and emergency handling processes. Through interaction with virtual digital humans, trainers can have a deeper understanding of knowledge such as power system operation, equipment operation and accident handling. In addition, virtual digital humans can also provide personalized training content and methods according to the learning progress and characteristics of trainers, which not only improves the pertinence and effect of training, but also reduces the cost and risk of safety training in the power industry. With the continuous development and improvement of technology, it is expected to play a greater role in the field of safety training in the power industry in the future.

[0129] The present invention may be a system, a method and / or a computer program product. The present invention also discloses a power industry safety training system based on virtual digital human technology based on the aforementioned power industry safety training method based on virtual digital human technology, comprising:

[0130] The training content acquisition module is used to obtain the safety training needs of the power industry, determine the key areas and goals of safety training, and collect text data related to safety training in the power industry;

[0131] The database construction module is used to classify and clean the text data related to safety training in the power industry, and to build the structure, fields and relationships of the training content database table;

[0132] The virtual digital human creation module is used to create a virtual digital human 3D model, use a full-body motion capture system to record user movements and map them to the virtual digital human 3D model, and construct the virtual digital human's facial expressions; make detailed adjustments and performance optimizations to the virtual digital human's movements and expression animations, use a speech synthesis model to establish a speech synthesis library for power industry safety training, and use an integrated natural language processing model to enable the virtual digital human to understand user input and generate corresponding feedback;

[0133] The training course display module is used to model the training scene using 3D modeling software, design the operation interaction logic consistent with the actual operation, and display the power industry safety operation training course to the trainees in a virtual reality environment through the real-time interaction design of the virtual digital human;

[0134] The evaluation and feedback module is used to ask trainers questions about safety knowledge points through virtual digital humans during the training process, evaluate the trainers' answers and give them scores, and adjust the training progress based on the trainers' scores.

[0135] Based on the spirit of the present invention, those skilled in the art can easily think that a computer program product can be obtained based on the aforementioned power industry safety training method based on virtual digital human technology. The computer program product may include a computer-readable storage medium, which carries computer-readable program instructions for enabling a processor to implement various aspects of the present disclosure. That is, the present application also includes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the aforementioned power industry safety training method based on virtual digital human technology.

[0136] Computer readable storage medium can be a tangible device that can keep and store the instructions used by the instruction execution device. Computer readable storage medium can be, for example, - but not limited to - electrical storage device, magnetic storage device, optical storage device, electromagnetic storage device, semiconductor storage device or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, punch card or groove protrusion structure with instructions stored thereon and any suitable combination of the above. Computer readable storage medium used here is not interpreted as instantaneous signal itself, such as radio wave or other free propagating electromagnetic wave, electromagnetic wave propagated by waveguide or other transmission medium (for example, light pulse by optical fiber cable) or electrical signal transmitted by wire.

[0137] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0138] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages-such as Smalltalk, C++, etc., and conventional procedural programming languages-such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network-including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may be personalized by utilizing the state information of a computer-readable program instruction, and the electronic circuit may execute a computer-readable program instruction, thereby realizing various aspects of the present disclosure.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A safety training method for the power industry based on virtual digital human technology, characterized in that: The following steps are involved: Step 1: Obtain the safety training needs of the power industry, determine the key areas and goals of safety training, and collect text data related to safety training in the power industry; Step 2: Classify and clean the text data related to safety training in the power industry, and build the structure, fields and relationships of the training content database table; Step 3: Create a 3D model of a virtual digital human, use a full-body motion capture system to record user movements and map them to the 3D model of the virtual digital human, and construct facial expressions of the virtual digital human; make detailed adjustments and optimize the performance of the virtual digital human's movements and expression animations, use a speech synthesis model to establish a speech synthesis library for power industry safety training, and use an integrated natural language processing model to enable the virtual digital human to understand user input and generate corresponding feedback; Step 4: Model the training scenario using 3D modeling software, design the operation interaction logic consistent with the actual operation, and present the power industry safety operation training course to the trainees in a virtual reality environment through the real-time interaction design of the virtual digital human; Step 5: During the training process, a virtual digital human is used to ask trainers questions about safety knowledge points, and the trainers’ answers are evaluated and scored. The training progress is adjusted based on the trainers’ scores.

2. The power industry safety training method based on virtual digital human technology according to claim 1 is characterized in that: The step of creating a virtual digital human 3D model further comprises: Use 3D modeling software Blender to create a 3D model of the virtual digital human, including its body, facial features, hairstyle, and clothing; use Blender's advanced modeling tools to adjust the shape, proportions, and details of the virtual digital human.

3. The power industry safety training method based on virtual digital human technology according to claim 2 is characterized in that: Constructing virtual digital human facial expressions further includes: Use the facial expression capture technology Faceware to build the facial expressions and lip shapes of virtual digital humans, clean and organize the captured motion data, remove unnatural jitters and erroneous data, and bind the captured motion and expression data to the 3D model of the virtual digital human. Use the keyframe animation and dynamic simulation technology in the 3D animation software Maya to adjust the movements and expressions of the virtual digital human.

4. The power industry safety training method based on virtual digital human technology according to claim 3 is characterized in that: The method of using the speech synthesis model to establish a speech synthesis library for power industry safety training further includes: The Tacotron 2 model is used for speech synthesis. First, a speech database for training is established by collecting public Chinese speech data sets and recording relevant power industry training speech and text transcription. The speech data sets are preprocessed by segmenting audio, extracting Mel spectrograms, and creating acoustic features. Select WaveNet vocoder, set the learning rate, training rounds, and validation interval training parameters, use the mean square error (MSE) loss function and optimizer, and the MSE loss function calculates the average of the squares of the differences between all predicted values ​​and the true values; The speech database is divided into test data and training data at a ratio of 2:

8. The training data is input into speech model training to obtain the power industry safety training speech synthesis library.

5. The power industry safety training method based on virtual digital human technology according to claim 4 is characterized in that: An integrated natural language processing model is used to enable the virtual digital human to understand user input and generate corresponding feedback, further including: Collect text data of conversations in different languages ​​and contexts; establish a natural language text library, use the recurrent neural network (RNN) model to learn the statistical characteristics of the language, adopt the attention mechanism to make the model focus on the key parts of the input text, and train the model to recognize and understand the implicit meaning, puns, and slang in the context; use the accuracy index, recall rate index, and F1 score index to evaluate the performance of the model and further optimize the model.

6. A safety training system for the power industry based on virtual digital human technology, characterized in that: include: The training content acquisition module is used to obtain the safety training needs of the power industry, determine the key areas and goals of safety training, and collect text data related to safety training in the power industry; The database construction module is used to classify and clean the text data related to safety training in the power industry, and to build the structure, fields and relationships of the training content database table; The virtual digital human creation module is used to create a virtual digital human 3D model, use a full-body motion capture system to record user movements and map them to the virtual digital human 3D model, and construct the virtual digital human's facial expressions; make detailed adjustments and performance optimizations to the virtual digital human's movements and expression animations, use a speech synthesis model to establish a speech synthesis library for power industry safety training, and use an integrated natural language processing model to enable the virtual digital human to understand user input and generate corresponding feedback; The training course display module is used to model the training scene using 3D modeling software, design the operation interaction logic consistent with the actual operation, and display the power industry safety operation training course to the trainees in a virtual reality environment through the real-time interaction design of the virtual digital human; The evaluation and feedback module is used to ask trainers questions about safety knowledge points through virtual digital humans during the training process, evaluate the trainers' answers and give them scores, and adjust the training progress based on the trainers' scores.

7. The power industry safety training system based on virtual digital human technology according to claim 6 is characterized in that: The virtual digital human creation module is further used to: Use 3D modeling software Blender to create a 3D model of the virtual digital human, including its body, facial features, hairstyle, and clothing; use Blender's advanced modeling tools to adjust the shape, proportions, and details of the virtual digital human.

8. The power industry safety training system based on virtual digital human technology according to claim 7 is characterized in that: The virtual digital human creation module is further used to: Use the facial expression capture technology Faceware to build the facial expressions and lip shapes of virtual digital humans, clean and organize the captured motion data, remove unnatural jitters and erroneous data, and bind the captured motion and expression data to the 3D model of the virtual digital human. Use the keyframe animation and dynamic simulation technology in the 3D animation software Maya to adjust the movements and expressions of the virtual digital human.

9. The power industry safety training system based on virtual digital human technology according to claim 8 is characterized in that: The virtual digital human creation module is further used to: The Tacotron 2 model is used for speech synthesis. First, a speech database for training is established by collecting public Chinese speech data sets and recording relevant power industry training speech and text transcription. The speech data sets are preprocessed by segmenting audio, extracting Mel spectrograms, and creating acoustic features. Select WaveNet vocoder, set the learning rate, training rounds, and validation interval training parameters, use the mean square error (MSE) loss function and optimizer, and the MSE loss function calculates the average of the squares of the differences between all predicted values ​​and the true values; The speech database is divided into test data and training data at a ratio of 2:

8. The training data is input into speech model training to obtain the power industry safety training speech synthesis library.

10. The power industry safety training system based on virtual digital human technology according to claim 9 is characterized in that: The virtual digital human creation module is further used to: Collect text data of conversations in different languages ​​and contexts; establish a natural language text library, use the recurrent neural network (RNN) model to learn the statistical characteristics of the language, adopt the attention mechanism to make the model focus on the key parts of the input text, and train the model to recognize and understand the implicit meaning, puns, and slang in the context; use the accuracy index, recall rate index, and F1 score index to evaluate the performance of the model and further optimize the model.

11. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the power industry safety training method based on virtual digital human technology according to any one of claims 1-5.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the power industry safety training method based on virtual digital human technology described in any one of claims 1 to 5 are implemented.

Images