Personnel training evaluation method and training evaluation device based on virtual reality simulation environment, computer storage medium and electronic equipment

Abstract

The invention discloses a person training evaluation method and device based on a virtual reality simulation environment, a computer storage medium and electronic equipment, and the method comprises the steps: obtaining the physiological information of a tested person in a target virtual reality simulation training process, and determining a physiological state evaluation result of the tested person according to the physiological information; after the target virtual reality simulation training is finished, obtaining test feedback information of the tested person, and determining a task load evaluation result of the tested person according to the test feedback information; and determining a training evaluation result of the tested person based on the physiological state evaluation result and the task load evaluation result. According to the method, the simulation training scene is constructed based on the virtual reality technology, the simulation training cost is reduced, the training evaluation result is determined based on the physiological information in the training process and the test feedback information after training is finished, the training evaluation precision is improved, and the training evaluation efficiency is improved. The imbalance problem that the state evaluation cost and the professional degree of simulation training are difficult to obtain at the same time in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of fire training technology, and in particular to a personnel training evaluation method based on a virtual reality simulation environment, a personnel training evaluation device based on a virtual reality simulation environment, a computer-readable storage medium, and an electronic device. Background Art

[0002] Through simulation training, real scenarios can be simulated, allowing participants to perform simulated operations, which is of great significance for improving skill levels, enhancing adaptability, and improving actual combat capabilities.

[0003] In the related art, when conducting fire simulation training, firstly, a real scene is built according to the training purpose to construct a simulation training scene, and the operator performs the target task in the simulation training scene to achieve the training purpose. However, the simulation training is costly and inflexible, and the evaluation of the training effect is based on the subjective judgment of the test personnel or evaluation experts, and the evaluation accuracy is low. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first purpose of the present application is to propose a personnel training and evaluation method based on a virtual reality simulation environment, to construct a simulation training scene based on virtual reality technology for simulation training, so as to reduce the cost of simulation training, and to determine the training evaluation results based on the physiological information during the training process and the test feedback information after the training, so as to improve the accuracy of training evaluation, and solve the imbalance problem in the related art that the state evaluation cost and professional level of simulation training are difficult to achieve at the same time.

[0005] The second purpose of this application is to propose a personnel training and evaluation device based on a virtual reality simulation environment.

[0006] The third object of the present application is to provide a computer-readable storage medium.

[0007] The fourth objective of the present application is to provide an electronic device.

[0008] To achieve the above-mentioned purpose, the first aspect of the present application proposes a personnel training and evaluation method based on a virtual reality simulation environment, the method comprising: during the target virtual reality simulation training process, obtaining the physiological information of the person under test, and determining the physiological state evaluation result of the person under test based on the physiological information; after the target virtual reality simulation training is completed, obtaining the test feedback information of the person under test, and determining the task load evaluation result of the person under test based on the test feedback information; determining the training evaluation result of the person under test based on the physiological state evaluation result and the task load evaluation result.

[0009] According to the personnel training and evaluation method based on the virtual reality simulation environment of the embodiment of the present application, first, during the target virtual reality simulation training process, the physiological information of the person under test is obtained, and the physiological state evaluation result of the person under test is determined based on the physiological information, and after the target virtual reality simulation training is completed, the test feedback information of the person under test is obtained, and the task load evaluation result of the person under test is determined based on the test feedback information, and then the training evaluation result of the person under test is determined based on the physiological state evaluation result and the task load evaluation result. Thus, the method constructs a simulation training scene based on virtual reality technology to carry out simulation training, so as to reduce the cost of simulation training, and determines the training evaluation result based on the physiological information during the training process and the test feedback information after the training is completed, so as to improve the accuracy of training evaluation, and solves the imbalance problem in the related technology that the state evaluation cost and professional level of simulation training are difficult to achieve at the same time.

[0010] In addition, the personnel training and evaluation method based on the virtual reality simulation environment according to the above embodiment of the present application may also have the following additional technical features:

[0011] According to one embodiment of the present application, determining the physiological state evaluation result of the measured person according to the physiological signal includes: determining the psychological state of the measured person during the target virtual reality simulation training process and the duration of each psychological state according to the physiological information; based on the psychological state of the measured person during the target virtual reality simulation training process and the duration of each psychological state, determining the total duration and time proportion of each psychological state in the target virtual reality simulation training process, so as to obtain the physiological state evaluation result.

[0012] According to one embodiment of the present application, the test feedback information includes the NASA-TLX (National Aeronauticsand Space Administration-Task Load Index) scale, and the task load evaluation result is determined based on the test feedback information, including: obtaining the scoring result of each subscale in the NASA-TLX scale; calculating the total workload based on the scoring result of each subscale in the NASA-TLX scale and the weight coefficient of each subscale to obtain the task load evaluation result.

[0013] According to one embodiment of the present application, the training evaluation results also include technical action evaluation results. The personnel training evaluation method based on a virtual reality simulation environment also includes: obtaining interaction information of the person being tested during the target virtual reality simulation training process; and determining the technical action evaluation results of the person being tested based on the physiological information and the interaction information.

[0014] According to one embodiment of the present application, the physiological signal includes eye movement information, the interactive information includes gait information and hand movement information, and the technical action evaluation result of the person under test is determined based on the physiological information and the interactive information, including: obtaining the interactive eye movement operation scoring standard corresponding to the target virtual reality simulation training; scoring the technical actions of the person under test in the target virtual reality simulation training according to the interactive eye movement operation scoring standard, gait information, hand movement information and eye movement information to obtain the technical action evaluation result of the person under test.

[0015] According to an embodiment of the present application, the interaction information includes gait information and hand movement information, the training evaluation result also includes a spatiotemporal behavior evaluation result, and the personnel training evaluation method based on a virtual reality simulation environment also includes: determining a SOI (Space of Interest) trajectory map and a first heat map of the tested person during a target virtual reality simulation training process according to the gait information; determining a degree of hand movement standardization of the tested person according to the hand movement information and a preset hand movement standard; determining a spatiotemporal behavior evaluation result of the tested person according to the SOI trajectory map, the first heat map, and the degree of hand movement standardization; and / or the physiological information includes eye movement information, the training evaluation result also includes an evaluation result of attention to a target area of ​​interest, and the personnel training evaluation method based on a virtual reality simulation environment also includes: determining an AOI (Areas of Interest, area of ​​interest) trajectory map and second heat map; determine the attention evaluation result of the subject on the target area of ​​interest according to the AOI trajectory map and the second heat map; and / or the physiological information includes EEG information, the training evaluation result also includes the attention evaluation result, and the personnel training evaluation method based on the virtual reality simulation environment also includes: performing time-frequency analysis on the EEG signal to obtain the attention evaluation result of the subject.

[0016] According to one embodiment of the present application, during the target virtual reality simulation training process, after obtaining the physiological information of the person being tested, the personnel training assessment method based on the virtual reality simulation environment also includes: determining the current physiological state of the person being tested based on the physiological information; and displaying the current physiological state based on virtual reality technology to remind the person being tested of the status.

[0017] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a personnel training and evaluation device based on a virtual reality simulation environment, the device comprising: a first acquisition module, used to obtain physiological information of a person under test during a target virtual reality simulation training process; a first determination module, used to determine a physiological state evaluation result of the person under test based on the physiological information; a second acquisition module, used to obtain test feedback information of the person under test after the target virtual reality simulation training is completed; a second determination module, used to determine a task load evaluation result of the person under test based on the test feedback information; a training evaluation module, used to determine the training evaluation result of the person under test based on the physiological state evaluation result and the task load evaluation result.

[0018] According to the personnel training and evaluation device based on the virtual reality simulation environment of the embodiment of the present application, the physiological information of the person under test is obtained by the first acquisition module during the target virtual reality simulation training process, and the physiological state evaluation result of the person under test is determined by the first determination module according to the physiological information, and the test feedback information of the person under test is obtained by the second acquisition module after the target virtual reality simulation training is completed, and the task load evaluation result of the person under test is determined by the second determination module according to the test feedback information, and the training evaluation module determines the training evaluation result of the person under test based on the physiological state evaluation result and the task load evaluation result. Thus, the device constructs a simulation training scene based on virtual reality technology to carry out simulation training, so as to reduce the cost of simulation training, and determines the training evaluation result based on the physiological information during the training process and the test feedback information after the training is completed, so as to improve the accuracy of training evaluation, and solves the imbalance problem that the state evaluation cost and professional level of simulation training in the related technology are difficult to achieve at the same time.

[0019] To achieve the above-mentioned objectives, the third aspect embodiment of the present application proposes a computer-readable storage medium, on which a personnel training and evaluation program based on a virtual reality simulation environment is stored. When the personnel training and evaluation program based on a virtual reality simulation environment is executed by a processor, the above-mentioned personnel training and evaluation method based on a virtual reality simulation environment is implemented.

[0020] According to the computer-readable storage medium of the embodiment of the present application, when the personnel training and evaluation program based on the virtual reality simulation environment stored in the computer-readable storage medium is executed by the processor, the above-mentioned method for personnel training and evaluation based on the virtual reality simulation environment is implemented. Based on the above-mentioned method for personnel training and evaluation based on the virtual reality simulation environment, the cost of simulation training is reduced, and the accuracy of training evaluation is improved, which solves the imbalance problem in the related technology that it is difficult to achieve both the state assessment cost and the professional level of simulation training.

[0021] To achieve the above-mentioned objectives, the fourth aspect of the present application proposes an electronic device, comprising: a memory, a processor, and a personnel training and evaluation program based on a virtual reality simulation environment stored in the memory and executable on the processor. When the processor executes the personnel training and evaluation program based on a virtual reality simulation environment, the above-mentioned personnel training and evaluation method based on a virtual reality simulation environment is implemented.

[0022] According to the electronic device of the embodiment of the present application, when the processor executes the personnel training and evaluation program based on the virtual reality simulation environment, the above-mentioned personnel training and evaluation method based on the virtual reality simulation environment is implemented. Based on the above-mentioned personnel training and evaluation method based on the virtual reality simulation environment, the cost of simulation training is reduced, and the accuracy of training evaluation is improved, which solves the imbalance problem in the related technology that it is difficult to achieve both the state assessment cost and the professional level of simulation training.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of a personnel training and evaluation method based on a virtual reality simulation environment according to an embodiment of the present application;

[0025] Figure 2 It is a flowchart of a personnel training and evaluation method based on a virtual reality simulation environment according to a specific embodiment of the present application;

[0026] Figure 3 Schematic diagram of the connection of a personnel training and evaluation device based on a virtual reality simulation environment according to an embodiment of the present application;

[0027] Figure 4 Schematic diagram of a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes, with reference to the accompanying drawings, a personnel training and evaluation method based on a virtual reality simulation environment, a personnel training and evaluation device based on a virtual reality simulation environment, a computer-readable storage medium, and an electronic device proposed in an embodiment of the present application.

[0030] Figure 1It is a flow chart of a personnel training and evaluation method based on a virtual reality simulation environment according to an embodiment of the present application.

[0031] The personnel training and evaluation method based on the virtual reality simulation environment can be used in the fields of medicine, military, aerospace, firefighting, etc. Exemplarily, when the training and evaluation method is applied to the medical field, it can be applied in scenarios such as postoperative recovery and personalized rehabilitation plans. For example, during the postoperative recovery period, the training and evaluation method is used to detect the patient's training and evaluation results. The medical team can timely identify the patient's postoperative recovery degree based on the training and evaluation results and take corresponding intervention measures; in the personalized rehabilitation plan scenario, the training and evaluation results of the patient can be used to assist in the formulation of a personalized rehabilitation plan.

[0032] The following is a detailed explanation using the application of the training evaluation method in a fire training scenario as an example.

[0033] like Figure 1 As shown, the personnel training and evaluation method based on the virtual reality simulation environment of the embodiment of the present application may include:

[0034] S1, during the target virtual reality simulation training, obtaining physiological information of the person under test, and determining the physiological state evaluation result of the person under test according to the physiological information;

[0035] S2, after the target virtual reality simulation training is completed, obtaining test feedback information of the tested person, and determining the task load evaluation result of the tested person according to the test feedback information;

[0036] S3, determining the training assessment results of the person being tested based on the physiological status assessment results and the task load assessment results.

[0037] Specifically, firstly, a simulation scene library is built through virtual reality technology based on the purpose of fire training. Then the test subjects select the required training scenes from the virtual scene library and conduct targeted virtual reality simulation training in the selected virtual scenes, such as fire rescue simulation training and fire extinguisher use training.

[0038] During the target virtual reality simulation training, the user wears a physiological signal acquisition bracelet and a VR (Virtual Reality) head display, collects physiological signals such as pulse through the physiological signal acquisition bracelet, and collects eye movement signals, EEG signals, etc. through the VR head display, so as to obtain the physiological information of the person being tested during the target virtual reality simulation training, and determine the physiological state evaluation results of the person being tested based on the physiological information. For example, the evaluation results of the user's physiological states such as fatigue and stress are output based on the acquired pulse data, and the evaluation results of the user's stress state are output based on the EEG data information. Specifically, the physiological state evaluation output can be based on a preset evaluation model.

[0039] After the target virtual reality simulation training is completed, a subjective questionnaire evaluation can be conducted on the testee to obtain the test feedback information of the testee, and the task load test result of the testee can be determined according to the subjective questionnaire evaluation result. The subjective questionnaire evaluation can be in the form of the NASA-TLX scale, and the task load evaluation result can be obtained by the user filling in each subscale. At this time, the task load evaluation result can be directly obtained by using the NASA-TLX scale filled in by the user, or it can be obtained based on further data processing of the NASA-TLX scale, and there is no specific limitation.

[0040] The training evaluation results of the tested persons include the physiological state evaluation results corresponding to the physiological information and the task load evaluation results corresponding to the subjective questionnaire. These two parts together constitute a complete evaluation report, which is generated after the training is completed and displayed through the terminal interface.

[0041] This embodiment is based on virtual reality scenes, real-time physiological monitoring and subjective evaluation technology. It supports firefighters to simulate training in virtual scenes, and monitor relevant physiological indicators such as fatigue and stress during their training in real time. Finally, it is combined with the results of professional subjective evaluation questionnaires to provide training feedback reports, which greatly improves the accuracy and professionalism of training evaluation.

[0042] In one embodiment of the present application, determining the physiological state evaluation result of the measured person according to the physiological signal includes: determining the psychological state of the measured person during the target virtual reality simulation training process and the duration of each psychological state according to the physiological information; determining the total duration and time proportion of each psychological state in the target virtual reality simulation training process based on the psychological state of the measured person during the target virtual reality simulation training process and the duration of each psychological state, so as to obtain the physiological state evaluation result.

[0043] Specifically, physiological information can include pulse, EEG, heart rate and other information. For example, the collected physiological information is first preprocessed, including filtering, removing baseline drift, etc., and then the features of the signal are extracted. These features can include time domain, frequency domain and nonlinear features, and then algorithms such as recursive feature elimination (RFE) and correlation bias reduction (CBR) are used to select the features that have the greatest impact on emotion recognition. Then, based on the construction of an emotion recognition model, the user's emotional state is classified according to the extracted features to obtain the psychological state of the person being tested during the target virtual reality simulation training process and the duration of each psychological state.

[0044] Then, according to the psychological state of the tested person during the target virtual reality simulation training and the duration of each psychological state, the total duration of each psychological state during the target virtual reality simulation training is calculated, and according to the ratio of the total duration of each psychological state to the target virtual reality simulation training duration, the time proportion of each psychological state is obtained, thereby obtaining the physiological state evaluation result. The physiological state evaluation result can be displayed in the form of a timeline, a pie chart, etc.

[0045] In one embodiment of the present application, the test feedback information includes the NASA-TLX scale, and the task load evaluation result is determined based on the test feedback information, including: obtaining the scoring result of each subscale in the NASA-TLX scale; calculating the total workload based on the scoring result of each subscale in the NASA-TLX scale and the weight coefficient of each subscale to obtain the task load evaluation result.

[0046] Specifically, at the end of the target virtual reality simulation training, the subjects fill in the corresponding NASA-TLX scale and make a subjective score for each subscale in the NASA-TLX scale. For example, the NASA-TLX scale includes subscales such as mental strength, physical strength, and time. Each subscale corresponds to a score of 1-10. The subjects can make a subjective score for the corresponding subscale by selecting a score between 1-10.

[0047] The total workload is calculated based on the weight coefficient of each subscale and the score of each subscale to obtain the task load evaluation result. Assuming that the NASA-TLX scale includes subscale A, subscale B, subscale C and subscale D, where the score of subscale A is a1 and the weight is a2; the score of subscale B is b1 and the weight is b2; the score of subscale C is c1 and the weight is c2; the score of subscale D is d1 and the weight is d2, then the total workload is = a1*a2+b1*b2+c1*c2+d1+d2.

[0048] In one embodiment of the present application, the training evaluation results also include technical action evaluation results, and the personnel training evaluation method based on the virtual reality simulation environment also includes: obtaining the interaction information of the person being tested during the target virtual reality simulation training process; and determining the technical action evaluation results of the person being tested based on the physiological information and the interaction information.

[0049] Specifically, interaction refers to the process of exchanging data, information and commands between users and the system or different applications in a computer system. During the target virtual reality simulation training process, the interaction information of the person under test obtained may include the action feedback data made by the person under test based on the instruction information given by the virtual reality scene, and may also include the operation data such as scene switching of the virtual reality scene based on the action command of the person under test. For example, the interaction information includes the posture information shown by the person under test during the operation.

[0050] Physiological information can include eye movement information, which is used to feedback the user's attention to the target area of ​​interest, while interactive information can feedback the user's training posture and feedback action. The technical actions of the person being tested during the test can be evaluated based on the physiological information, gait, and hand movement information. For example, in the application of a fire extinguisher, the standard posture information and target area of ​​interest information can be determined based on the action instructions given by the current virtual reality scene, and the feedback posture information given by the person being tested based on the action instructions can be compared with the standard posture information, and the feedback attention information determined based on the physiological information can be compared with the target area of ​​interest information to obtain the technical action evaluation results based on the action instructions. By analogy, the technical action evaluation of the entire simulation training is completed.

[0051] In one embodiment of the present application, the physiological signal includes eye movement information, the interactive information includes gait information and hand movement information (such as handle grabbing, touching and other actions), and the technical action evaluation result of the person being tested is determined based on the physiological information and the interactive information, including: obtaining the interactive eye movement operation scoring standard corresponding to the target virtual reality simulation training; scoring the technical actions of the person being tested in the target virtual reality simulation training according to the interactive eye movement operation scoring standard, gait information, hand movement information and eye movement information, so as to obtain the technical action evaluation result of the person being tested.

[0052] Specifically, eye movement information refers to data obtained by tracking the movement trajectory of human eyes when viewing objects or scenes, including information such as the location of the gaze point, dwell time, and movement speed, which can be used to feedback the user's attention to the target area of ​​interest. Gait information is the spatial position information of the person being tested during operation, which can be combined with posture information such as hands, head, legs, etc. (which can be obtained by connecting external sensors), such as users wearing motion sensors on their wrists, legs, etc. Hand movement information can also be obtained based on sensors, such as handle grabbing, touching, and other actions.

[0053] The virtual reality simulation training corresponds to the interactive eye movement operation scoring standard one by one. The corresponding interactive eye movement operation scoring standard is determined according to the target virtual reality simulation training selected by the testee (this standard can also be customized by the user). Based on the interactive eye movement operation scoring standard, the acquired gait information, hand movement information and eye movement information are scored for the technical movements to obtain the technical movement evaluation results of the testee.

[0054] Taking the firefighters' wall-searching ability training as an example, the interactive eye movement operation scoring criteria are as follows:

[0055] The total score is 100 points, starting from 100 points, and the score is deducted. Wrong operation will result in deduction of points (users can customize the deduction weight).

[0056] i. The user needs to hold a fire extinguisher or water gun and put out the fire in the fire area in the corridor first. The fire extinguishing time is used as the evaluation standard (the fire extinguishing must be completed within 10 seconds after the start. If it is not completed, 10 points will be deducted; the scenario can also be customized to search and rescue work after fire extinguishing, that is, the user only needs to search and rescue without extinguishing the fire);

[0057] ii. The user plays the role of wall searcher No. 1. He needs to lean against a load-bearing wall on one side, hold a thermal imager and a fire axe to explore the way, and grope forward with his head in a squatting position (the height of the head can be judged. If the head is higher than 1.2m for more than 5s, 5 points will be deducted, and 2 points will be deducted for each additional 1s);

[0058] iii. The search area (including the sight range) must reach 95% of the room area (excluding obstacles). If it does not reach this, or if it only reaches 85%, which is 10% less than 95%, 5 points will be deducted (the search area standard and deduction weight can be customized by the user);

[0059] iv. Inspect the user's handling of obstacles (whether the fire axe touches the obstacle and bypasses it). If there is an obstacle but the user does not find it, points will be deducted in the final calculation based on the proportion of the total area of ​​the ignored obstacle. For example, 5 points will be deducted for a 10% room ratio (customizable);

[0060] v. Examine the total time from the user entering the room to searching for the trapped person. The user can customize the standard time according to the area of ​​the room. If it exceeds the standard time, 2 points will be deducted for every 5 seconds exceeding the standard time. If the time in the user's eye movement data of continuously looking at the trapped person is less than 2 seconds, 10 points will be deducted (the time and the proportion of deductions can be customized by the user).

[0061] In addition, the final technical action evaluation result can be obtained by combining the technical action scores obtained by expert evaluation. For example, weight coefficients are set between the above technical action evaluation results and the expert evaluation results respectively, and the total score is calculated based on the preset weight coefficients, which is used as the final technical action evaluation result.

[0062] In one embodiment of the present application, the interaction information includes gait information and hand movement information (such as handle grabbing, touching and other movements), the training evaluation results also include spatiotemporal behavior evaluation results, and the personnel training evaluation method based on the virtual reality simulation environment also includes: determining the SOI trajectory map and the first heat map of the person under test during the target virtual reality simulation training process according to the gait information; determining the degree of hand movement standardization of the person under test according to the hand movement information and the preset hand movement standard; determining the spatiotemporal behavior evaluation results of the person under test according to the SOI trajectory map, the first heat map and the degree of hand movement standardization; and / or physiological information includes The training evaluation results also include the evaluation results of attention to the target area of ​​interest. The personnel training evaluation method based on the virtual reality simulation environment also includes: determining the AOI trajectory map and the second heat map of the person under test during the target virtual reality simulation training according to the eye movement information; determining the evaluation results of the person under test's attention to the target area of ​​interest according to the AOI trajectory map and the second heat map; and / or the physiological information includes EEG information, the training evaluation results also include the attention evaluation results. The personnel training evaluation method based on the virtual reality simulation environment also includes: performing time-frequency analysis on the EEG signal to obtain the attention evaluation result of the person under test.

[0063] Specifically, the evaluation dimensions of the training evaluation results can be added, deleted, or other calculation methods can be used to calculate and evaluate multi-dimensional data, without specific limitation.

[0064] For example, when the training evaluation results also include the spatiotemporal behavior evaluation results, an SOI trajectory map and a first heat map are generated based on the gait information, wherein the SOI trajectory map can display the position information of the person being tested that changes over time, and the first heat map can display the activity intensity or frequency of the person being tested in the simulated scene, thereby evaluating the spatiotemporal behavior of the person being tested in combination with the degree of standardization of hand movements, wherein the degree of standardization of hand movements can be determined by comparing the acquired hand movement information with the preset hand movement standards. When the training evaluation results also include the evaluation results of attention to the target area of ​​interest, the AOI trajectory map and the second heat map of the person being tested are determined based on the eye movement information, thereby determining the attention of the person being tested to the target area of ​​interest based on the AOI trajectory map and the second heat map. When the training evaluation results also include the attention evaluation results, the EEG signals are analyzed in the time domain and frequency domain to analyze the attention state of the person being tested.

[0065] In one embodiment of the present application, during the target virtual reality simulation training process, after obtaining the physiological information of the person being tested, the personnel training assessment method based on the virtual reality simulation environment also includes: determining the current physiological state of the person being tested based on the physiological information; and displaying the current physiological state based on virtual reality technology to remind the person being tested of the status.

[0066] Specifically, during the target virtual reality simulation training process, the current physiological state of the person being tested, such as fatigue, stress, etc., is analyzed based on the physiological information of the person being tested obtained in real time, and is displayed in real time in the scene interface seen by the user based on virtual reality technology to remind the user of the status.

[0067] For example, during the target virtual reality simulation training, the current physiological state of the person being tested is evaluated only based on the pulse and other data collected by the physiological signal collection bracelet, and the evaluation results of the fatigue, stress and other physiological states of the person being tested are output. This result is displayed to the person being tested in real time in the scene interface, so that the person being tested can know his or her real-time state, such as knowing when he or she is more stressed, so that he or she can adjust his or her mentality and state by taking deep breaths, etc. At the same time, the eye movement signals and EEG signals collected by the VR headset are not processed and displayed in real time, and are only used for analysis after the training is completed.

[0068] As a specific embodiment of the present application, Figure 2 As shown, the personnel training and evaluation method based on the virtual reality simulation environment may include:

[0069] S101, obtaining physiological information of a person under test during a target virtual reality simulation training process.

[0070] S102, determining the psychological state of the person under test during the target virtual reality simulation training and the duration of each psychological state according to the physiological information.

[0071] S103, based on the psychological state of the person being tested during the target virtual reality simulation training and the duration of each psychological state, determine the total duration and time proportion of each psychological state during the target virtual reality simulation training process to obtain a physiological state assessment result.

[0072] S104, after the target virtual reality simulation training is completed, the NASA-TLX scale of the tested person is obtained.

[0073] S105, determining the task load evaluation result of the tested person according to the test feedback information.

[0074] S106, determining the training evaluation result of the person being tested based on the physiological status evaluation result and the task load evaluation result.

[0075] Furthermore, the training operations of the persons being tested during the training process may be scored based on the interaction information during the target virtual reality simulation training process.

[0076] The above-mentioned target virtual reality simulation training process records and stores data for the subsequent expert evaluation function. While taking into account the subjective and objective evaluation costs and professionalism, it retains detailed evaluation space for comprehensive training effect evaluation of users.

[0077] Therefore, firefighters use virtual reality scenes for simulation training, and monitor and record their heart rate, pulse, EEG and other physiological signals in real time, evaluate their fatigue, stress and other related physiological indicators in real time, and conduct professional subjective questionnaire evaluation after the training, combining physiological and psychological indicators to provide subjective and objective training effect evaluation reports. In addition, after the evaluation, the training process recording and subjective and objective data are saved, which can be used for playback detail evaluation and in-depth analysis.

[0078] In summary, according to the personnel training and evaluation method based on the virtual reality simulation environment of the embodiment of the present application, first, during the target virtual reality simulation training process, the physiological information of the person being tested is obtained, and the physiological state evaluation result of the person being tested is determined based on the physiological information, and after the target virtual reality simulation training is completed, the test feedback information of the person being tested is obtained, and the task load evaluation result of the person being tested is determined based on the test feedback information, and then the training evaluation result of the person being tested is determined based on the physiological state evaluation result and the task load evaluation result. Therefore, the method constructs a simulation training scene based on virtual reality technology to perform simulation training, so as to reduce the cost of simulation training, and determines the training evaluation result based on the physiological information during the training process and the test feedback information after the training, so as to improve the accuracy of training evaluation, and solves the imbalance problem in the related technology that the state evaluation cost and professional level of simulation training are difficult to achieve at the same time.

[0079] Corresponding to the above-mentioned embodiments, the present application also proposes a personnel training and evaluation device based on a virtual reality simulation environment.

[0080] like Figure 3 As shown, the personnel training and evaluation device based on the virtual reality simulation environment of the embodiment of the present application may include: a first acquisition module 10, a first determination module 20, a second acquisition module 30, a second determination module 40 and a training evaluation module 50.

[0081] Among them, the first acquisition module 10 is used to obtain the physiological information of the person under test during the target virtual reality simulation training. The first determination module 20 is used to determine the physiological state evaluation result of the person under test based on the physiological information. The second acquisition module 30 is used to obtain the test feedback information of the person under test after the target virtual reality simulation training is completed. The second determination module 40 is used to determine the task load evaluation result of the person under test based on the test feedback information. The training evaluation module 50 is used to determine the training evaluation result of the person under test based on the physiological state evaluation result and the task load evaluation result.

[0082] According to one embodiment of the present application, the first determination module 20 determines the physiological state evaluation result of the measured person according to the physiological signal, and is specifically used to: determine the psychological state of the measured person during the target virtual reality simulation training process and the duration of each psychological state according to the physiological information; determine the total duration and time proportion of each psychological state in the target virtual reality simulation training process based on the psychological state of the measured person during the target virtual reality simulation training process and the duration of each psychological state, so as to obtain the physiological state evaluation result.

[0083] According to one embodiment of the present application, the test feedback information includes the NASA-TLX scale, and the second determination module 40 determines the task load evaluation result based on the test feedback information, specifically for: obtaining the scoring result of each subscale in the NASA-TLX scale; calculating the total workload based on the scoring result of each subscale in the NASA-TLX scale and the weight coefficient of each subscale to obtain the task load evaluation result.

[0084] According to one embodiment of the present application, the training evaluation results also include technical action evaluation results. The personnel training evaluation device based on the virtual reality simulation environment also includes: a third acquisition module, used to obtain the interaction information of the person being tested during the target virtual reality simulation training process; the training evaluation module 50 is also used to determine the technical action evaluation results of the person being tested based on the physiological information and the interaction information.

[0085] According to one embodiment of the present application, the physiological signal includes eye movement information, the interactive information includes gait information and hand movement information, and the training evaluation module 50 is also used to determine the technical action evaluation results of the testee based on the physiological information and the interactive information, specifically for: obtaining the interactive eye movement operation scoring standard corresponding to the target virtual reality simulation training; scoring the technical actions of the testee in the target virtual reality simulation training according to the interactive eye movement operation scoring standard, gait information, hand movement information and eye movement information, so as to obtain the technical action evaluation results of the testee.

[0086] According to one embodiment of the present application, the interaction information includes gait information and hand movement information, the training evaluation results also include spatiotemporal behavior evaluation results, and the training evaluation module 50 is also used to: determine the SOI trajectory map and the first heat map of the subject during the target virtual reality simulation training according to the gait information; determine the degree of standardization of the hand movements of the subject according to the hand movement information and the preset hand movement standards; determine the spatiotemporal behavior evaluation results of the subject according to the SOI trajectory map, the first heat map and the degree of standardization of the hand movements; and / or the physiological information includes eye movement information, the training evaluation results also include attention evaluation results for the target area of ​​interest, and the training evaluation module 50 is also used to: determine the AOI trajectory map and the second heat map of the subject during the target virtual reality simulation training according to the eye movement information; determine the attention evaluation results of the subject to the target area of ​​interest according to the AOI trajectory map and the second heat map; and / or the physiological information includes EEG information, the training evaluation results also include attention evaluation results, and the training evaluation module 50 is also used to: perform time-frequency analysis on the EEG signal to obtain the attention evaluation result of the subject.

[0087] According to one embodiment of the present application, during the target virtual reality simulation training process, after obtaining the physiological information of the person being tested, the training evaluation module 50 is also used to: determine the current physiological state of the person being tested based on the physiological information, and display the current physiological state based on virtual reality technology to provide real-time status reminders to the person being tested.

[0088] It should be noted that for details not disclosed in the personnel training and evaluation device based on a virtual reality simulation environment in the embodiment of the present application, please refer to the details disclosed in the personnel training and evaluation method based on a virtual reality simulation environment in the above embodiment of the present application, and the details will not be repeated here.

[0089] According to the personnel training and evaluation device based on the virtual reality simulation environment of the embodiment of the present application, the physiological information of the person under test is obtained by the first acquisition module during the target virtual reality simulation training process, and the physiological state evaluation result of the person under test is determined by the first determination module according to the physiological information, and the test feedback information of the person under test is obtained by the second acquisition module after the target virtual reality simulation training is completed, and the task load evaluation result of the person under test is determined by the second determination module according to the test feedback information, and the training evaluation module determines the training evaluation result of the person under test based on the physiological state evaluation result and the task load evaluation result. Thus, the device constructs a simulation training scene based on virtual reality technology to carry out simulation training, so as to reduce the cost of simulation training, and determines the training evaluation result based on the physiological information during the training process and the test feedback information after the training is completed, so as to improve the accuracy of training evaluation, and solves the imbalance problem that the state evaluation cost and professional level of simulation training in the related technology are difficult to achieve at the same time.

[0090] Corresponding to the above embodiments, the present application also proposes a computer-readable storage medium.

[0091] The computer-readable storage medium of the embodiment of the present application stores a personnel training and evaluation program based on a virtual reality simulation environment. When the personnel training and evaluation program based on a virtual reality simulation environment is executed by a processor, the personnel training and evaluation method based on a virtual reality simulation environment is implemented.

[0092] According to the computer-readable storage medium of the embodiment of the present application, when the personnel training and evaluation program based on the virtual reality simulation environment stored in the computer-readable storage medium is executed by the processor, the above-mentioned method for personnel training and evaluation based on the virtual reality simulation environment is implemented. Based on the above-mentioned method for personnel training and evaluation based on the virtual reality simulation environment, the cost of simulation training is reduced, and the accuracy of training evaluation is improved, which solves the imbalance problem in the related technology that it is difficult to achieve both the state assessment cost and the professional level of simulation training.

[0093] Corresponding to the above embodiment, the present application also proposes an electronic device.

[0094] like Figure 4 As shown, the electronic device 100 of the embodiment of the present application includes: a memory 110, a processor 120, and a personnel training and evaluation program based on a virtual reality simulation environment stored in the memory 110 and executable on the processor 120. When the processor 120 executes the personnel training and evaluation program based on a virtual reality simulation environment, the above-mentioned personnel training and evaluation method based on a virtual reality simulation environment is implemented.

[0095] According to the electronic device of the embodiment of the present application, when the processor executes the personnel training and evaluation program based on the virtual reality simulation environment, the above-mentioned personnel training and evaluation method based on the virtual reality simulation environment is implemented. Based on the above-mentioned personnel training and evaluation method based on the virtual reality simulation environment, the cost of simulation training is reduced, and the accuracy of training evaluation is improved, which solves the imbalance problem in the related technology that it is difficult to achieve both the state assessment cost and the professional level of simulation training.

[0096] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0097] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0099] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0100] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0101] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A personnel training and evaluation method based on a virtual reality simulation environment, characterized in that: The method comprises: During the target virtual reality simulation training, the physiological information of the person under test is obtained, and the physiological state evaluation result of the person under test is determined according to the physiological information; After the target virtual reality simulation training is completed, obtaining test feedback information of the tested person, and determining the task load evaluation result of the tested person according to the test feedback information; The training assessment result of the person being measured is determined based on the physiological state assessment result and the task load assessment result.

2. The personnel training and evaluation method based on virtual reality simulation environment according to claim 1 is characterized in that: Determining the physiological status evaluation result of the measured person according to the physiological signal includes: Determine the psychological state of the tested person during the target virtual reality simulation training and the duration of each psychological state according to the physiological information; Based on the psychological state of the tested person during the target virtual reality simulation training process and the duration of each psychological state, the total duration and time proportion of each psychological state during the target virtual reality simulation training process are determined to obtain the physiological state evaluation result.

3. The personnel training and evaluation method based on virtual reality simulation environment according to claim 1 is characterized in that: The test feedback information includes the NASA-TLX scale, and determining the task load evaluation result according to the test feedback information includes: Obtain the scoring results of each subscale of the NASA-TLX scale; The total workload is calculated according to the scoring results of each subscale in the NASA-TLX scale and the weight coefficient of each subscale to obtain the task load evaluation result.

4. The personnel training and evaluation method based on virtual reality simulation environment according to claim 1 is characterized in that: The training evaluation results also include technical action evaluation results, and the method further includes: During the target virtual reality simulation training, obtaining interaction information of the person being tested; The technical action evaluation result of the person being tested is determined based on the physiological information and the interactive information.

5. The personnel training and evaluation method based on virtual reality simulation environment according to claim 4 is characterized in that: The physiological signal includes eye movement information, the interactive information includes gait information and hand movement information, and the determining of the technical movement evaluation result of the measured person based on the physiological information and the interactive information includes: Obtaining interactive eye movement operation scoring criteria corresponding to the target virtual reality simulation training; The technical movements of the testee in the target virtual reality simulation training are scored according to the interactive eye movement operation scoring standard, the gait information, the hand movement information and the eye movement information to obtain the technical movement evaluation result of the testee.

6. The personnel training and evaluation method based on virtual reality simulation environment according to claim 4 is characterized in that: The interaction information includes gait information and hand movement information, the training evaluation result also includes a spatiotemporal behavior evaluation result, and the method further includes: Determine, according to the gait information, a SOI trajectory map and a first heat map of the person under test during the target virtual reality simulation training process; Determining the degree of standardization of the hand movements of the person being measured according to the hand movement information and a preset hand movement standard; Determining the spatiotemporal behavior evaluation result of the measured person according to the SOI trajectory diagram, the first heat map, and the degree of standardization of the hand movements; and / or The physiological information includes eye movement information, the training evaluation result also includes an attention evaluation result on the target interest area, and the method further includes: Determine an AOI trajectory map and a second heat map of the person under test during the target virtual reality simulation training according to the eye movement information; Determine the attention evaluation result of the measured person to the target area of ​​interest according to the AOI trajectory map and the second heat map; and / or The physiological information includes electroencephalogram information, the training evaluation result also includes attention evaluation result, and the method further includes: Performing time-frequency analysis on the EEG signal to obtain the attention evaluation result of the person being tested.

7. The personnel training and evaluation method based on virtual reality simulation environment according to claim 1 is characterized in that: During the target virtual reality simulation training, after obtaining the physiological information of the person being tested, the method further includes: Determining the current physiological state of the measured person according to the physiological information; The current physiological state is displayed based on virtual reality technology to remind the person being tested of the state.

8. A personnel training and evaluation device based on a virtual reality simulation environment, characterized in that: The device comprises: The first acquisition module is used to obtain physiological information of the person being tested during the target virtual reality simulation training process; A first determination module, used to determine the physiological state evaluation result of the measured person according to the physiological information; A second acquisition module is used to obtain test feedback information of the tested person after the target virtual reality simulation training is completed; A second determination module is used to determine the task load evaluation result of the tested person according to the test feedback information; The training assessment module is used to determine the training assessment results of the person being tested based on the physiological state assessment results and the task load assessment results.

9. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, a method for personnel training and evaluation based on a virtual reality simulation environment according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: include: A memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the method for personnel training and assessment based on a virtual reality simulation environment according to any one of claims 1 to 7 is implemented.

Images