Communication engineering construction operation simulation training method and device based on VR technology

Abstract

The invention belongs to the technical field of construction operation simulation training, and relates to a communication engineering construction operation simulation training method and device based on a VR technology. The weight of the construction action recognition model is updated according to the size relation between the error value and the preset error threshold value, so that the weight updating process is closely related to the training process, the weight updating self-adaption degree is improved, and then the model training speed and the construction operation simulation training efficiency are improved; meanwhile, when the training error is large, the updating mode of the weight is adjusted according to the number of times of training and the size of the error, so that the updating of the weight is more scientific; meanwhile, through the work operation information identification model, fine actions of the user, namely work operation information of the user, can be identified, and after the work operation information of the user is clarified, the work operation information is projected to the VR scene model used for simulating a communication engineering construction work scene, so that the effect of accurate work simulation training is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of construction operation simulation training, and more specifically, to a communication engineering construction operation simulation training method and device based on VR technology. Background Art

[0002] Virtual reality (VR) technology is an emerging form of human-computer interaction that can create a virtual world, imitate the real world, and provide participants with an immersive experience. Virtual reality technology has been widely used in the field of training simulation. Communication engineering construction operation virtual reality training is a simulation training method based on virtual reality technology. By simulating real construction scenes, participants can receive various practical operation guidance and learning in a virtual environment. Compared with traditional training methods, virtual reality training has the following advantages: Low cost: Virtual reality simulation training can use virtual scenes to simulate training subjects, thereby saving material loss in real scene training. At the same time, virtual reality simulation training can also provide participants with an immersive experience, allowing participants to feel various scenarios and pressures in a real environment, so that participants can master skills more quickly.

[0003] In the prior art, there are simulation control training schemes based on VR technology. For example, a Chinese invention patent (CN113001569A) discloses a live working robot arm remote control system based on VR technology, including a robot arm working platform embedded control system and an augmented reality monitoring and operating system. The robot arm working platform embedded control system includes an embedded control unit, a sensor input unit, an actuator control unit, a power management unit and a wireless communication unit; the augmented reality monitoring and operating system includes a high-performance computing host, a human-computer interaction device and a wireless communication device, and the human-computer interaction device includes a head-mounted display, left and right robot arm joint detection equipment, left and right robot gripper somatosensory gloves and a keyboard and mouse input device; however, the above scheme has the problem of low simulation accuracy and effect during the simulation control training of construction operations. Summary of the invention

[0004] The purpose of the present invention is to provide a communication engineering construction operation simulation training method and device based on VR technology, which is used to improve the efficiency and accuracy of operation simulation training.

[0005] In order to solve the above technical problems, the present invention specifically provides the following technical solutions: a communication engineering construction operation simulation training method based on VR technology, specifically comprising the steps of:

[0006] S1: Establish a VR scene model for simulating communication engineering construction operation scenes;

[0007] S2: Identify the user's operation information through the operation action recognition model;

[0008] The operation information of the user identified by the operation action recognition model is as follows:

[0009] S2.1: Collect user action data;

[0010] S2.2: performing a preprocessing operation on the action data;

[0011] S2.3: Establish action recognition model;

[0012] The action recognition model is a BP neural network model, and the weight update method of the BP neural network model is:

[0013] Sa: Train the BP neural network model several times; calculate and obtain multiple error values ​​e k ;

[0014] Sb: According to the multiple error values ​​e k Determine the preset error threshold e t ;

[0015] Sc: Continue to train the BP neural network model; calculate the error value e n ;

[0016] Sd: If the error value e n Less than the preset error threshold e t , then enter Se, otherwise enter Sf;

[0017] Se: Use the first method to update the weight;

[0018] The formula for weight update in the first method is:

[0019] Δω=η×e n ×b

[0020] Where Δω is the updated value of the weight, η is the learning rate of the first weight update, and b is the bias;

[0021] Sf: Use the second method to update the weight;

[0022] The formula for weight updating in the second method is:

[0023]

[0024]

[0025] In the formula, c is the training frequency adjustment coefficient, k t To preset the maximum number of training times;

[0026] S2.4: Inputting the motion data pre-processed in S2.2 into the motion recognition model to obtain the user's operation information;

[0027] S3: Inputting the user's operation information into the VR scene model for simulating the communication engineering construction operation scene to realize simulation training of the communication engineering construction operation.

[0028] Preferably, in the Sa, the multiple error values ​​e k The calculation process is:

[0029] e k =(yy k )×f'(y)

[0030] In the formula, e is the error of the model training process, y is the actual value in a certain training process, and y k is the output value calculated for the kth training, and f′(y) is the derivative of the activation function.

[0031] Preferably, the several times are one of 10, 20, and 30.

[0032] Preferably, in the Sb, the preset error threshold e t The calculation formula is:

[0033] e t =a×e m

[0034] In the formula, a is the adjustment coefficient, e is m is the error value e during several training processes k The maximum value in .

[0035] Preferably, a is 0.6.

[0036] Preferably, in S2.1, the motion data includes acceleration data, angular velocity data, and displacement data.

[0037] Preferably, in S2.2, the preprocessing includes: data missing value processing and data denoising.

[0038] Preferably, in S2.3, the BP neural network model includes an input layer, several hidden layers and an output layer; wherein the input layer is used to receive preprocessed action data and pass the action data to the hidden layer, the hidden layer contains neurons for extracting and converting the action data, and the output layer is used to output the final processed result.

[0039] Preferably, interpolation is used to handle missing values.

[0040] Preferably, a Butterworth filter is used to pre-process the motion data.

[0041] According to another aspect of the present invention, a communication engineering construction operation simulation training device based on VR technology is provided, wherein the device adopts the communication engineering construction operation simulation training method based on VR technology, and the device further comprises:

[0042] A VR scene model building module is used to build a VR scene model for simulating the construction operation scene of a communication engineering project;

[0043] A job operation information identification module, used to identify the user's job operation information;

[0044] The projection module is used to input the user's operation information into the VR scene model used to simulate the communication engineering construction operation scene, so as to realize the simulation training of the communication engineering construction operation.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The present invention is based on the error value e n The weights are updated in relation to the size of the preset error threshold, so that the weight update process is closely related to the training process, which improves the adaptability of the weight update, thereby improving the training speed of the model and the efficiency of the construction operation simulation training; at the same time, when the training error is large, the weight update method is adjusted according to the number of training times and the error size, making the weight update more scientific;

[0047] At the same time, the present invention can identify the user's subtle movements, that is, the user's operation information, through the operation operation information recognition model. After clarifying the user's operation information, the operation operation information is projected into the VR scene model used to simulate the communication engineering construction operation scene, thereby achieving the effect of accurate operation simulation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0049] Figure 1 A flowchart of a communication engineering construction operation simulation training method based on VR technology provided by an embodiment of the present invention;

[0050] Figure 2A flow chart of recognizing a user's operation information through an operation action recognition model provided by an embodiment of the present invention;

[0051] Figure 3 The present invention provides a flowchart of a weight updating method for an operation action recognition model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The following first describes the concepts involved in the present application in conjunction with the accompanying drawings. It should be noted that the following description of each concept is only to make the content of the present application easier to understand, and does not limit the scope of protection of the present application; at the same time, the embodiments and features in the embodiments of the present application can be combined with each other in the absence of conflict. The present application will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

[0054] In the process of communication engineering construction, it is generally necessary to drive special vehicles to complete the construction work, such as cranes, forklifts, etc., and communication engineering construction is a sub-field of the construction field that has strict requirements on the operation of machinery. Generally, skilled special vehicle drivers are required to meet the construction requirements. Therefore, general drivers need to undergo simulation training of construction operations before they can take up their posts to avoid possible operational errors as much as possible. Therefore, this embodiment proposes a user (driver) simulation training method for communication engineering construction operations based on VR technology, which uses virtual reality technology to realize user construction operation simulation training.

[0055] like Figure 1 As shown, the present invention provides a communication engineering construction operation simulation training method based on VR technology, which specifically includes the steps of:

[0056] S1: Establish a VR scene model for simulating communication engineering construction operation scenes;

[0057] Among them, as the basis of simulation training, the established VR scene model not only has realistic requirements, but also needs to have an appropriate degree of simplicity and complexity, and must meet the requirements that the VR scene model can respond in real time. Therefore, when establishing the VR scene model, the following principles are required:

[0058] (1) Clear scene model elements;

[0059] According to the construction operation scene, the VR scene model includes communication cables, electric poles, construction vehicles, buildings, roads, etc.;

[0060] (2) Near real-time response;

[0061] In order to display the VR scene model more smoothly according to the operator's actions, the VR scene model cannot be too complicated. You can limit the running of unnecessary programs, adjust the number of elements, the number of light sources, and the number of unnecessary animation programs, and enhance the realism of the scene model by adding textures and other operations;

[0062] In this step, a VR scene model for simulating the construction scene of a communication engineering project is established using 3D MAX modeling software; specifically, the VR scene model is established according to the size of the actual training site and the scene elements of the training site, and then each scene element in the scene model is optimized, and the relative position of each scene element in the scene is scheduled and managed;

[0063] In fact, when establishing a scene model, in order to improve the efficiency of VR scene model establishment and real-time responsiveness requirements, it is generally required that the number of scene elements in the VR model or the number of faces and edges in the scene elements be as small as possible. However, in the established VR scene model, it is inevitable that the faces or edges of some scene elements will hardly appear in the simulation training. Therefore, it is necessary to optimize the scene elements in the VR scene model to reduce the complexity of the VR scene model and improve the real-time responsiveness of the VR scene model.

[0064] Specifically, the need to optimize each scene element in the VR scene model includes: deleting overlapping contact surfaces between the scene elements; and deleting bottom surfaces of the scene elements that are not visible to the naked eye.

[0065] S2: Identify the user's job operation information;

[0066] In this step, the user's operation information is identified through the operation action recognition model;

[0067] Motion recognition is widely used in the field of virtual reality. People cannot rely solely on their eyes to observe virtual images in the three-dimensional world. They also need to use various parts of the body to interact with the virtual environment. This is to achieve better simulation training. For example, by monitoring the user's various actions, the operation simulation training system can track user behavior more accurately and give timely feedback. It can also capture various user gestures to achieve more natural interaction with virtual objects.

[0068] Operation action recognition based on wearable sensors can meet the above needs very well. Through the sensor equipment worn by the user, the timing signals generated by various human actions, such as acceleration, angular velocity and other signals, can be obtained, and then data preprocessing operations are performed. The operation actions are recognized through the operation action recognition model, and finally the combination of visual feedback and human motion perception is realized, making the user's interactive experience in the virtual scene more realistic, thereby improving the effect of simulation training.

[0069] Specifically, attached Figure 2 The specific flow chart of identifying the user's operation information through the operation action recognition model is shown in the attached figure. Figure 2 As shown, the operation information of the user identified by the operation action recognition model is specifically:

[0070] S2.1: Collect user action data;

[0071] Wherein, the motion data includes acceleration data, angular velocity data, and displacement data;

[0072] The acceleration data is collected by an acceleration sensor worn by the user, the angular velocity data is collected by an angular velocity sensor worn by the user, and the displacement data is collected by a position sensor worn by the user. In addition, the acceleration sensor, angular velocity sensor and displacement sensor include multiple ones, which are located at different positions of the user's body to collect the above data of different parts of the user;

[0073] In addition, the acceleration sensor, angular velocity sensor and displacement sensor may be integrated together or independently collect data, which is not limited in this embodiment.

[0074] S2.2: performing a preprocessing operation on the action data;

[0075] Data preprocessing is important for subsequent model recognition because the quality of data determines the upper limit of the accuracy of model recognition. In this step, the preprocessing includes: data missing value processing and data denoising;

[0076] In the actual data collection process, there may be missing data, such as NaN values, which will have a certain impact on the subsequent action recognition. Therefore, the missing data needs to be processed;

[0077] In this step, interpolation method is used to handle missing values;

[0078] In the process of data collection, it is inevitable that the data will be interfered by noise from the external environment, resulting in the data collected containing certain noise components. The noise will reduce the quality of the data and affect the accuracy and reliability of data analysis and models. Therefore, it is necessary to denoise the noise.

[0079] In this step, a Butterworth filter is used to pre-process the motion data;

[0080] S2.3: Establish action recognition model;

[0081] In this step, the action recognition model is a BP neural network model; BP neural network (BackPropagation Neural Network) is a multi-layer feedforward neural network, which is trained by the error back propagation algorithm and is one of the most widely used neural network models; the core idea of ​​BP neural network is to train the network through two stages: forward propagation and back propagation. In the forward propagation stage, the input signal is transmitted from the input layer to the output layer. If the expected output is not obtained in the output layer, the back propagation stage is entered. In the back propagation stage, the error of the output layer is calculated, and the weights and biases of each neuron are adjusted layer by layer until the output error of the network reaches an acceptable range or the preset number of learning times is reached;

[0082] In this step, the BP neural network model includes an input layer, several hidden layers and an output layer; wherein the input layer is used to receive the pre-processed action data and pass the action data to the hidden layer, the hidden layer contains a large number of neurons for extracting and converting the action data, and the output layer is used to output the final processed result;

[0083] Among them, the neurons in the hidden layer are connected by weights, and the weights are used to adjust the importance of the action data; the process of BP neural network model training is to adjust the size of the weights to achieve the optimal effect; in the prior art, the error between the prediction result of the output layer and the actual target output is calculated by back propagation of the BP neural network model, and the error is the error under the current weight. The gradient is calculated according to the error by the chain rule, and the weight is updated according to the gradient and the learning rate, so that the error is reduced;

[0084] The error is minimized by continuously updating the updated value Δw of the weight value to achieve the purpose of model training; however, the above scheme requires a large amount of data sets and computing resources to achieve the above process, resulting in poor model training efficiency;

[0085] Based on the above defects, this embodiment proposes a new weight updating method to improve the training efficiency of the model; Figure 3 As shown, specifically, the weight updating method is:

[0086] Sa: Train the BP neural network model several times; calculate and obtain multiple error values ​​e k ;

[0087] In this step, the multiple error values ​​e k The calculation process is:

[0088] e k =(yy k )×f'(y);

[0089] In the formula, e k is the error of the model in the kth training process, y is the actual value in a certain training process, and y k is the output value calculated for the kth training, and f′(y) is the derivative of the activation function;

[0090] Wherein, in this step, the number of times is one of 10, 20, and 30;

[0091] Sb: According to the multiple error values ​​e k Determine the preset error threshold e t ;

[0092] In this step, the preset error threshold e t The calculation formula is:

[0093] e t =a×e m ;

[0094] In the formula, a is the adjustment coefficient, e is m is the error value e during several training processes k The maximum value in ;

[0095] In this step, a is 0.6;

[0096] Sc: Continue to train the BP neural network model; calculate the error value e n ;

[0097] Among them, the error value e n The calculation process of is the same as that of Sa. In this step, firstly, through steps Sa-Sb, the preset error threshold is determined, and then the model is trained to obtain an error value e n , according to the error value e n Proceed to next steps.

[0098] Sd: If the error value e n Less than the preset error threshold e t , then enter Se, otherwise enter Sf;

[0099] In this step, after calculating the error value e of a training process k After that, by taking the error value e k And the preset error threshold is used to judge, so that different weight update methods are adopted, so that the weights can converge as quickly as possible;

[0100] Se: Use the first method to update the weight;

[0101] The formula for weight update in the first method is:

[0102] Δω=η×e n ×b,

[0103] Where Δω is the updated value of the weight, η is the learning rate of the first weight update, and b is the bias;

[0104] Sf: Use the second method to update the weight;

[0105] The formula for weight updating in the second method is:

[0106]

[0107]

[0108] In the formula, c is the training frequency adjustment coefficient, k t To preset the maximum number of training times;

[0109] In this step, according to the error value e n The weights are updated in relation to the size of the preset error threshold, so that the weight update process is closely related to the training process, which improves the adaptability of the weight update, thereby improving the training speed of the model and the efficiency of the construction operation simulation training; at the same time, when the training error is large, the weight update method is adjusted according to the number of training times and the error size, making the weight update more scientific;

[0110] S2.4: Inputting the action data pre-processed in S2.2 into the action recognition model to obtain an action recognition result;

[0111] Through the above-mentioned model, the user's subtle movements, that is, the user's work operation information, can be identified. After the user's work operation information is clarified, the work operation information is projected into the VR scene model used to simulate the communication engineering construction work scene, thereby achieving the effect of accurate work simulation training.

[0112] S3: Inputting the user's operation information into the VR scene model for simulating the communication engineering construction operation scene to realize simulation training of the communication engineering construction operation.

[0113] Embodiment 2, this embodiment also includes a communication engineering construction operation simulation training system based on VR technology, the device adopts a communication engineering construction operation simulation training method based on VR technology in embodiment 1, and the system also includes:

[0114] A VR scene model building module is used to build a VR scene model for simulating the construction operation scene of a communication engineering project;

[0115] A job operation information identification module, used to identify the user's job operation information;

[0116] The projection module is used to input the user's operation information into the VR scene model used to simulate the communication engineering construction operation scene, so as to realize the simulation training of the communication engineering construction operation.

[0117] Embodiment three, this embodiment includes an electronic device, including a processor and a memory, the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, it implements each step of a communication engineering construction operation simulation training method based on VR technology in embodiment one, and can achieve the same technical effect.

[0118] Embodiment 4, this embodiment includes a computer-readable storage medium, on which a data processing program is stored, and the data processing program is executed by a processor to perform various steps of a communication engineering construction operation simulation training method based on VR technology in embodiment 1.

[0119] It will be appreciated by those skilled in the art that the embodiments herein may be provided as methods, devices (equipment), or computer program products. Therefore, this article may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Including but not limited to RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, magnetic tape, disk storage or other magnetic storage device, or any other medium that can be used to store desired information and can be accessed by a computer, etc. In addition, it is well known to those of ordinary skill in the art that communication media generally contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0120] This document is described with reference to flowcharts and / or block diagrams of methods, apparatuses (devices) and computer program products according to the embodiments of this document. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0121] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0122] The embodiments and / or implementation methods described above are only used to illustrate the preferred embodiments and / or implementation methods of the technology of the present invention, and are not intended to impose any formal restrictions on the implementation methods of the technology of the present invention. Any person skilled in the art may make some changes or modifications to other equivalent embodiments without departing from the scope of the technical means disclosed in the content of the present invention, but they should still be regarded as the technology or embodiments that are essentially the same as the present invention. Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described here, and that various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention is described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may also include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A communication engineering construction operation simulation training method based on VR technology, characterized in that: The specific steps include: S1: Establish a VR scene model for simulating communication engineering construction operation scenes; S2: Identify the user's operation information through the operation action recognition model; The operation information of the user identified by the operation action recognition model is as follows: S2.1: Collect user action data; S2.2: performing a preprocessing operation on the action data; S2.3: Establish action recognition model; The action recognition model is a BP neural network model, and the weight update method of the BP neural network model is: Sa: Train the BP neural network model several times; calculate and obtain multiple error values ​​e k ; Sb: According to the multiple error values ​​e k Determine the preset error threshold e t ; Sc: Continue to train the BP neural network model; calculate the error value e n ; Sd: If the error value e n Less than the preset error threshold e t , then enter Se, otherwise enter Sf; Se: Use the first method to update the weight; The formula for weight update in the first method is: Give = I × e n ×b; Where Δω is the updated value of the weight, η is the learning rate of the first weight update, and b is the bias; Sf: Use the second method to update the weight; The formula for weight updating in the second method is: In the formula, c is the training frequency adjustment coefficient, k t is the preset maximum number of training times, k is the current number of training times; S2.4: Inputting the motion data pre-processed in S2.2 into the motion recognition model to obtain the user's operation information; S3: Inputting the user's operation information into the VR scene model for simulating the communication engineering construction operation scene to realize simulation training of the communication engineering construction operation.

2. The communication engineering construction operation simulation training method based on VR technology according to claim 1 is characterized in that: In the Sa, the multiple error values ​​e k The calculation process is: and k =(yy k )×f′(y); In the formula, e k is the error of the model in the kth training process, y is the actual value in a certain training process, yk is the output value calculated in the kth training, and f′(y) is the derivative of the activation function.

3. The communication engineering construction operation simulation training method based on VR technology according to claim 1 is characterized in that: The number of times is one of 10, 20, and 30.

4. The communication engineering construction operation simulation training method based on VR technology according to claim 1 is characterized in that: In the Sb, the preset error threshold e t The calculation formula is: and t =a×e m ; In the formula, a is the adjustment coefficient, e is m is the error value e during several training processes k The maximum value in .

5. The communication engineering construction operation simulation training method based on VR technology according to claim 4 is characterized in that: a is 0.

6.

6. The communication engineering construction operation simulation training method based on VR technology according to claim 1 is characterized in that: In S2.1, the motion data includes acceleration data, angular velocity data, and displacement data.

7. The communication engineering construction operation simulation training method based on VR technology according to claim 6 is characterized in that: In S2.2, the preprocessing includes: data missing value processing and data denoising.

8. The communication engineering construction operation simulation training method based on VR technology according to claim 6 is characterized in that: In S2.3, the BP neural network model includes an input layer, several hidden layers and an output layer; wherein the input layer is used to receive preprocessed action data and pass the action data to the hidden layer, the hidden layer contains neurons for extracting and converting the action data, and the output layer is used to output the final processed result.

9. The communication engineering construction operation simulation training method based on VR technology according to claim 7 is characterized in that: An interpolation method is used to process missing values, and a Butterworth filter is used to preprocess the action data.

10. A communication engineering construction operation simulation training device based on VR technology, characterized in that: The device adopts the communication engineering construction operation simulation training method based on VR technology according to any one of claims 1 to 9, and the device also includes: A VR scene model building module is used to build a VR scene model for simulating the construction operation scene of a communication engineering project; A job operation information identification module, used to identify the user's job operation information; The projection module is used to input the user's operation information into the VR scene model used to simulate the communication engineering construction operation scene, so as to realize the simulation training of the communication engineering construction operation.

Images