Demining simulation training method and system based on virtual-real combination

Abstract

The invention discloses a mine clearing simulation training method and system based on virtuality and reality combination, and relates to the technical field of simulation training. Behavior situations and physiological feedback of trainees are analyzed through real-time monitoring data, and a training data set is generated; and calculating the auditory pressure and threat perception ability of the trainee based on the environment information and behavior data in the data set so as to construct a comprehensive auditory intensity coefficient # imgabs0 # and a pressure coefficient Ylxs, inputting the data into a mine elimination prediction model, and calculating the threatening perception ability of the trainee according to the calculation result of the auditory immersion index Tczs. According to the method, parameters such as sound source intensity and sound effect complexity in a training environment are dynamically adjusted, so that a student is always in a proper immersion state, the nervous emotion of the student caused by too high pressure can be avoided, the influence of insufficient immersion on a training effect can be prevented, personalized environment adjustment is realized, and the training efficiency is improved. And the overall experience of students is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of simulation training, and in particular to a mine clearance simulation training method and system based on the combination of virtual and real. Background Art

[0002] The mine clearance simulation training method belongs to the application of virtual reality VR and augmented reality AR technology in the military and public security fields. With the development of virtual-reality technology, simulation training plays an increasingly important role in military training, crisis response and public security training, especially in areas with high danger and high practical difficulty. The virtual environment can provide a safe and controllable training method. Specifically in the field of virtual-reality mine clearance simulation training, it aims to provide a low-risk, high-simulation training environment through the interaction of virtual and reality technology, so as to simulate the actual mine clearance scene without endangering the safety of trainees.

[0003] Although enhanced immersion can improve the realism of the training experience and enhance trainees' task engagement, excessive immersion may have negative effects. When the immersion is too high, the details in the virtual environment may interfere with the trainees' attention too much, causing them to be seriously distracted and neglect the improvement of actual operational skills. This problem of balancing immersion and training effectiveness may cause the system to be more inclined towards "experience" rather than "skill", causing trainees to perform well in the virtual environment but make mistakes in actual mine clearance operations.

[0004] In the existing mine clearance simulation training system, although the improvement of immersion enhances the experience of training, it also brings some difficulties in improving practical operation skills. Most of the current systems enhance the trainees' sense of immersion by improving the realism of the virtual environment, but this may ignore the practicality and effectiveness of the training. For example, in a highly realistic virtual environment, complex visual, auditory and other detailed designs may attract too much attention from trainees, causing them to be distracted and ignore the core skills training during the operation. In addition, some existing systems lack real-time monitoring of trainees' psychological pressure, making it difficult to adjust the immersion intensity at the appropriate time, which may increase the psychological burden of trainees. These defects make the performance of trainees in virtual training and in actual operation very different, leading to the phenomenon of "paper talk", which limits the practical value of simulation training. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a mine clearance simulation training method and system based on a combination of virtual and real, which solves the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a mine clearance simulation training method based on a combination of virtual and real, comprising the following steps: S1. Use virtual reality and augmented reality technologies to create a three-dimensional mine clearance virtual environment in advance, and use wearable devices to allow trainees to enter the three-dimensional mine clearance virtual environment; S2. When trainees are in the process of demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior situation data information, and relevant environmental data information under corresponding environmental conditions is collected, and after data preprocessing, a training data set is generated; S3. Based on the relevant environmental data information in the training data set, analyze the auditory pressure that the trainees are subjected to under different test environmental conditions to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; S4. Use deep learning technology to build a mine clearance prediction model by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions in the mine clearance simulation training process are adjusted.

[0007] Preferably, the specific steps of S1 include: S11. In the mine clearance simulation training, virtual reality technology and augmented reality technology are used in advance to create a three-dimensional mine clearance virtual environment, and terrain features are reconstructed through three-dimensional modeling technology. The terrain features include mine distribution areas, vegetation, surface texture and obstacle distribution; S12. Based on the three-dimensional mine-clearing virtual environment simulated in S11, the trainees enter the three-dimensional mine-clearing virtual environment through wearable devices, wherein the wearable devices include VR head-mounted displays, AR smart glasses, sensor gloves, bracelets, wearable microphones and wearable headphones.

[0008] Preferably, the specific steps of S2 include: S21. When the trainees are in the process of demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior status data information, wherein the relevant behavior status data information includes heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, wrong judgment frequency and jitter frequency ; S22, setting different environmental conditions and collecting relevant environmental data information under the corresponding environmental conditions, wherein the relevant environmental data information includes the number of sound sources n, the sound pressure level emitted by the corresponding sound source , the direction angle of the corresponding sound source relative to the student , the distance between the corresponding sound source and the student's head , the position coordinates of the corresponding sound source and the student's head orientation vector ; S23. Preprocess the relevant behavioral situation data information and the relevant environmental data information. The preprocessing includes noise removal, missing value filling and data smoothing operations. The missing value filling methods include mean filling, median filling, interpolation filling and regression filling. The relevant behavioral situation data information and the relevant environmental data information are scaled using dimensionless processing technology.

[0009] Preferably, the specific steps of S3 include: S31. Based on the relevant environmental data information in the training data set, analyze the auditory pressure that the trainees are subjected to under different test environmental conditions, and after linear normalization, construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , which can be obtained by: ; Where n represents the number of sound sources, i=1, 2, 3, ..., n, It is represented by the sound pressure level emitted by the i-th sound source, It is represented as the direction angle of the i-th sound source relative to the student, It is represented as the distance between the i-th sound source and the student’s head.

[0010] Preferably, the specific step S3 also includes: S311: The distance between the i-th sound source and the student's head Obtained by the following formula: ; S312: Direction angle of the i-th sound source relative to the student Obtained by the following formula: ; In the formula, represents the angle between the direction of the ith sound source and the orientation of the student’s head; Represented as the position coordinates of the i-th sound source; Indicates the coordinates of the student's head position; is the orientation vector of the student’s head; is the dot product of the i-th sound source propagation vector and the head direction vector; Expressed as the magnitude of the head orientation vector; The vector representing the position of the student's head pointing to the position of the i-th sound source; is the modulus of the propagation vector of the ith sound source; is the cosine value of the angle between the head direction and the propagation vector of the i-th sound source.

[0011] Preferably, the specific step S3 also includes: S32, according to the relevant behavior status data information in the training data set, by correlating the heart rate change value Xbz, skin conductivity Pdz, respiratory frequency Hxp and respiratory depth Hsd, and after linear normalization processing, obtain the tension factor , the tension factor Obtained by the following formula: ; In the formula, , , and are all weight values, P represents the first correction constant, which is, , , and The specific value is set by the user according to the situation.

[0012] Preferably, the specific step S3 also includes: S33, based on the relevant behavior status data information in the training data set, combined with the tension factor obtained in S32 , analyze the trainees' threat perception ability under different environmental conditions, and after linear normalization, construct the pressure coefficient Ylxs, which is obtained by the following formula: ; In the formula, Expressed as the tension factor, It is expressed as the frequency of wrong judgment, Expressed as the jitter frequency, and are weight values, among which, and The specific value is set by the user according to the situation. is the second normal number.

[0013] Preferably, the specific steps of S4 include: S41. Use deep learning technology to build a mine clearance prediction model, and input the training data set after data preprocessing into the mine clearance prediction model. After training, fit and output the auditory immersion index Tczs, and the auditory immersion index Tczs is obtained by the following formula: ; In the formula, Expressed as the comprehensive hearing intensity coefficient, Expressed as the target sound source intensity, Expressed as the pressure coefficient, and are weight values, among which, and The specific value is set by the user according to the situation.

[0014] Preferably, the specific step S4 also includes: S42, pre-set an evaluation threshold Q, wherein the evaluation threshold Q includes a first threshold and a second threshold, and the first threshold is greater than the second threshold, and compare and analyze the auditory immersion index Tczs with the evaluation threshold Q to comprehensively judge whether the current environmental conditions enable the trainees to be in a normal state during the mine clearance training, and the specific judgment contents are as follows: If the auditory immersion index Tczs is within the second threshold, it is judged that the current environmental conditions make the trainees in an abnormal state during the mine clearance training, and the sound source intensity will be increased by one level; If the second threshold value is less than the auditory immersion index Tczs ≤ the first threshold value, it is judged that the current environmental conditions make the trainees in a normal state when conducting mine clearance training. At this time, the trainees will be formally trained in mine clearance under the current environmental conditions; If the auditory immersion index Tczs is greater than the first threshold, it is determined whether the current environmental conditions cause the trainees to be in an abnormal state during the mine clearance training. At this time, the sound source intensity will be lowered by one level.

[0015] A mine clearance simulation training system based on the combination of virtual and real, including an environment building module, a monitoring module, a training analysis module and a simulation optimization module; The environment building module is used to create a three-dimensional mine clearance virtual environment in advance using virtual reality technology and augmented reality technology, and to allow trainees to enter the three-dimensional mine clearance virtual environment through wearable devices; The monitoring module is used to monitor the trainees' coefficient behavior performance under different environmental conditions in real time during the mine clearance simulation training, so as to obtain relevant behavior situation data information, and collect relevant environmental data information under corresponding environmental conditions, and generate a training data set after data preprocessing; The training analysis module analyzes the auditory pressure experienced by trainees under different test environmental conditions based on the relevant environmental data information in the training data set, so as to construct a comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; The simulation optimization module is used to construct a mine clearance prediction model using deep learning technology by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions in the mine clearance simulation training process are adjusted.

[0016] The present invention provides a mine clearance simulation training method and system based on the combination of virtual and real, which has the following beneficial effects: (1) By reconstructing a real mine-clearing scene in a three-dimensional virtual environment and using wearable devices such as VR and AR, trainees can enter an immersive training environment. This combination of virtual and real can simulate various complex mine-clearing scenes, allowing trainees to practice mine-clearing skills in a simulated environment, improving their adaptability to the real environment and sense of task substitution. The present invention uses real-time monitoring data to analyze the trainees' behavioral status and physiological feedback, generates a training data set, and calculates the trainees' auditory pressure and threat perception ability based on the environmental information and behavioral data in the data set, thereby constructing a comprehensive auditory intensity coefficient. The data are input into the mine-clearing prediction model. The sound source intensity, sound effect complexity and other parameters in the training environment are dynamically adjusted through the calculation result of the auditory immersion index Tczs, so that the trainees are always in a suitable immersion state. This can not only avoid excessive pressure causing the trainees to feel nervous, but also prevent insufficient immersion from affecting the training effect, thereby realizing personalized environmental adjustment and improving the overall experience of the trainees. The mine-clearing prediction model is constructed by deep learning technology, and the auditory intensity coefficient and pressure coefficient are input into the model for training. The trainees' immersion index Tczs in different environments can be accurately predicted. The training environment is dynamically adjusted based on the prediction results to ensure that the trainees are always in the best immersion state. This intelligent adjustment mechanism can effectively improve the effectiveness of training, so that the trainees can always maintain a relatively good psychological and physiological state when facing different environmental conditions, thereby improving the training efficiency. Compared with the traditional static training environment, the intelligent environment adjustment of the present invention significantly enhances the flexibility and adaptability of training, further avoids the contradiction between immersion and operation skills, and enables the skills learned by the trainees in virtual training to be better applied to actual mine-clearing tasks.

[0017] (2) During the mine-clearing simulation, the sound pressure level, distance from the trainee's head and direction angle of each sound source are calculated in detail, so that the system can track complex sound scenes. By determining the direction of the sound in the environment, it can simulate the sound sources at different locations around the trainee, making their sense of tension closer to the threat perception in reality. The system can accurately adjust the auditory pressure during training, further ensuring that trainees maintain relatively realistic sound perception under interference from different sound sources. This precisely restored complex sound field environment greatly enhances the trainees' sense of immersion and trains their ability to respond quickly in high-noise, complex sound source environments, helping to improve their adaptability to real mine clearance scenarios. Through the comprehensive calculation and linear normalization of the sound pressure level, directional angle and distance of multiple sound sources, the comprehensive auditory intensity coefficient The auditory pressure data under different environments is standardized. This standardized auditory intensity coefficient ensures that the system can effectively control auditory pressure under various environmental conditions, and further avoids the psychological burden caused by excessively loud sounds or lack of immersion caused by too weak sounds during training.

[0018] (3) Under different environmental conditions, the trainees' threat perception ability is further analyzed in combination with the tension factor, and the stress coefficient Ylxs is generated. This coefficient can truly reflect the trainees' psychological load level in a specific environment and effectively ensure the safety of training. The auditory immersion index Tczs is predicted through a deep learning model. Based on this index, the system can automatically adjust the sound intensity and background sound effects in the training environment. When the auditory immersion is too low, the complexity of the sound effects is increased to enhance the sense of involvement; when it is too high, the sound intensity is reduced to further avoid excessive psychological pressure affecting the training effect. This method can provide personalized adjustments based on the actual status of the trainees to avoid the quality of training being affected by excessive pressure or insufficient immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a mine clearance simulation training method based on the combination of virtual and real in the present invention; Figure 2 The present invention is a block diagram of a mine clearance simulation training system based on the combination of virtual and real. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Example 1: Please refer to Figure 1 The present invention provides a mine clearance simulation training method based on the combination of virtual and real, comprising the following steps: S1. Use virtual reality and augmented reality technologies to create a three-dimensional mine clearance virtual environment in advance, and use wearable devices to allow trainees to enter the three-dimensional mine clearance virtual environment; S2. When trainees are in the process of demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior situation data information, and relevant environmental data information under corresponding environmental conditions is collected, and after data preprocessing, a training data set is generated; S3. Based on the relevant environmental data information in the training data set, analyze the auditory pressure that the trainees are subjected to under different test environmental conditions to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; S4. Use deep learning technology to build a mine clearance prediction model by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions in the mine clearance simulation training process are adjusted.

[0022] In this embodiment, by using virtual reality technology and augmented reality technology to construct a three-dimensional mine clearance virtual environment in step S1, the trainees can experience a realistic mine clearance situation in person. With the support of wearable devices, the trainees can obtain highly realistic visual, auditory and tactile feedback in the virtual environment, which helps to enhance the immersion of training, make it easier for them to enter the training state, and improve the quality of the training experience. In step S2, by real-time monitoring of the trainees' behavioral performance under different environmental conditions, relevant behavioral situation data and environmental data are obtained, and a training data set is generated through data preprocessing. This real-time data collection and monitoring method can not only help the system dynamically grasp the trainees' behavioral patterns and physiological states, but also effectively identify the trainees' stress states and adaptation conditions during the training process. By processing and analyzing the data, the training environment can be appropriately adjusted according to the trainees' performance to ensure that the trainees receive a realistic training experience within a safe range. By analyzing the training data in steps S3 and S4, the comprehensive auditory intensity coefficient is calculated. The system can intelligently adjust the sound intensity and complexity in the training environment according to the feedback of the auditory immersion index Tczs, thereby providing students with personalized training experience under different environmental conditions. This intelligent adjustment mechanism can not only avoid the psychological pressure and fatigue caused by excessive immersion, but also enhance the environmental intensity when students are undertrained, ensuring that students can obtain more effective skills training and achieve relatively better training results. In short, the mine clearance simulation training method provided by this method achieves a balance between immersion and training effectiveness through a virtual-real training environment and an intelligent adjustment mechanism, so that students can obtain a highly realistic immersive experience without excessively disturbing their attention, and can better master practical operation skills, thereby significantly improving the quality and safety of training.

[0023] Example 2: Please refer to Figure 1 , specifically: S1 specific steps include: S11. In the mine clearance simulation training, virtual reality technology and augmented reality technology are used in advance to create a three-dimensional mine clearance virtual environment, and terrain features are reconstructed through three-dimensional modeling technology. The terrain features include mine distribution areas, vegetation, surface texture and obstacle distribution, so that trainees can experience situations similar to real mine clearance tasks in person; S12. Based on the three-dimensional mine-clearing virtual environment simulated in S11, the trainees enter the three-dimensional mine-clearing virtual environment through wearable devices, wherein the wearable devices include VR head-mounted displays, AR smart glasses, sensor gloves, bracelets, wearable microphones and wearable headphones. The wearable devices capture the movements of the trainees' heads, hands, and even bodies, and synchronize them in real time in the virtual environment to achieve precise interaction.

[0024] The specific steps of S2 include: S21. When the trainees are in the process of demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior status data information, wherein the relevant behavior status data information includes heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, wrong judgment frequency and jitter frequency ; The above-mentioned heart rate change value Xbz is monitored and obtained by a photoelectric heart rate sensor; Skin conductivity Pdz is monitored and obtained through a galvanic skin response (GSR) sensor; The respiratory rate Hxp and respiratory depth Hsd are monitored and obtained through the respiratory belt sensor; Frequency of wrong judgment The training management software can be used to monitor and obtain the information. By recording the decision-making results of trainees in virtual training (determining whether there are mines), the number of wrong judgments can be automatically calculated, thereby obtaining the frequency of wrong judgments. S22, setting different environmental conditions and collecting relevant environmental data information under the corresponding environmental conditions, wherein the relevant environmental data information includes the number of sound sources n, the sound pressure level emitted by the corresponding sound source , the direction angle of the corresponding sound source relative to the student , the distance between the corresponding sound source and the student's head , the position coordinates of the corresponding sound source and the student's head orientation vector ; The above-mentioned relevant environmental data information can be used to track the movement and position of the trainees in the virtual environment through the inertial measurement unit (IMU). The sensors of the VR helmet and handle can be used to accurately track the position of the trainees' heads and hands to achieve real-time updating of spatial coordinates. Through environmental modeling technology, the coordinate system can be embedded in the virtual environment to record the trainees' position information in real time.

[0025] S23. Preprocess the relevant behavioral situation data information and the relevant environmental data information. The preprocessing includes noise removal, missing value filling and data smoothing operations. The missing value filling methods include mean filling, median filling, interpolation filling and regression filling. The relevant behavioral situation data information and the relevant environmental data information are scaled using dimensionless processing technology so that their data range falls between 0 and 1.

[0026] In this embodiment, by using virtual reality and augmented reality technology to construct a three-dimensional mine clearance environment in mine clearance simulation training, real terrain features such as minefield distribution, vegetation, surface texture and obstacles are simulated. With the help of wearable devices such as VR head-mounted displays and AR smart glasses, trainees can not only truly feel the details of the surrounding environment, but also interact with the virtual environment through feedback from devices such as sensor gloves and bracelets. This highly simulated training environment greatly improves the trainees' sense of task substitution and immersion experience, allowing them to adapt to and master various skills in actual mine clearance situations more quickly, and increase the effectiveness and authenticity of training. This method realizes multi-dimensional monitoring of trainees' psychological pressure, behavioral state and their response to the environment by real-time monitoring of trainees' heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, wrong judgment frequency and jitter frequency and other behavioral status data during training. At the same time, the system also records key parameters in the training environment, such as the number of sound sources, sound pressure level, direction angle, and distance from the trainee's head. This multi-dimensional data collection and analysis mechanism enables the system to obtain the trainee's physiological and psychological feedback in real time, adjust the training content based on these feedbacks, and provide personalized feedback guidance, thereby improving the adaptability of the training process and the accuracy of feedback. After data collection, the system performs comprehensive preprocessing on the behavioral and environmental data, including noise removal, missing value filling, and data smoothing to ensure the accuracy and consistency of the data. This preprocessing process not only improves the data quality, but also reduces the interference of environmental noise and data missing on the training effect. Through a variety of filling methods such as mean filling, median filling, and interpolation filling, reasonable completion is ensured in different data missing situations. At the same time, dimensionless processing technology standardizes the data to a unified range between 0 and 1, which is convenient for subsequent model training and predictive analysis. This systematic preprocessing method ensures the integrity and consistency of the data, improves the reliability of model prediction, and ensures the continuous optimization of the training environment. In summary, this method improves the authenticity, feedback accuracy and data quality of mine clearance simulation training through immersive virtual environment, intelligent real-time monitoring and data preprocessing, provides students with a more effective and safe training experience, and significantly enhances the intelligence level of the training model.

[0027] Example 3: Please refer to Figure 1 , specifically: S3 specific steps include: S31. Based on the relevant environmental data information in the training data set, analyze the auditory pressure that the trainees are subjected to under different test environmental conditions, and after linear normalization, construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , which can be obtained by: ; Where n represents the number of sound sources, i=1, 2, 3, ..., n, It is represented by the sound pressure level (intensity) emitted by the i-th sound source, It is represented as the direction angle of the i-th sound source relative to the student, It is represented as the distance between the i-th sound source and the student’s head.

[0028] The specific steps of S3 also include: S311: The distance between the i-th sound source and the student's head Obtained by the following formula: ; S312: Direction angle of the i-th sound source relative to the student Obtained by the following formula: ; In the formula, represents the angle between the direction of the ith sound source and the orientation of the student’s head; It is represented as the position coordinates of the i-th sound source, which is the specific location where the sound is emitted in the virtual environment; Indicates the coordinates of the student's head position, which is the student's actual current coordinate position in the virtual environment; is the student's head orientation vector, indicating the current direction of the student's head; is the dot product of the i-th sound source propagation vector and the head direction vector, indicating the relationship between these two vectors; It is expressed as the modulus of the head orientation vector, i.e., the length of the unit vector of the student’s head orientation; The vector that indicates the position of the student's head pointing to the position of the i-th sound source, also known as the sound propagation vector; is the modulus (length) of the propagation vector of the ith sound source; is the cosine value of the angle between the head direction and the propagation vector of the i-th sound source, which is calculated to angle.

[0029] Comprehensive hearing intensity coefficient It reflects the total sound pressure level of the comprehensive sound intensity transmitted to the students’ ears; In this embodiment, by collecting data from multiple sound sources in the training environment, the sound pressure level, direction angle and distance from the trainee's head of each sound source are calculated, and a comprehensive hearing intensity coefficient is constructed based on these parameters. , so that the system can accurately quantify the auditory pressure of trainees during training. This coefficient intuitively reflects the comprehensive impact of sound sources on trainees under specific environmental conditions. By adjusting the sound pressure level, direction angle and other factors in the environment, it can provide trainees with an immersive experience closer to the real scene, thereby improving the training effect and the trainees' situational adaptability. The distance and direction angle between each sound source and the trainee's head are dynamically calculated through formulas, so that the system can provide real-time feedback on the spatial information of the sound. Specifically, the system accurately calculates the relative position relationship between each sound source and the trainee in the virtual environment, including changes in distance and angle, so that the auditory impact of different sound source positions on trainees is more realistic. This dynamic feedback enables trainees to perceive the direction and distance changes of the sound source, thereby more realistically simulating the sound interference from all directions in the real mine clearance scene, and improving the trainees' judgment and anti-interference ability in high-noise environments. In summary, this method realizes intelligent auditory pressure regulation in mine clearance training through dynamic calculation and comprehensive control of the sound pressure level, distance and directional angle of multiple sound sources, improves the training immersion and situational realism, and helps trainees better adapt to the auditory challenges in real mine clearance tasks in a virtual environment.

[0030] Example 4: Please refer to Figure 1 , specifically: S3 specific steps also include: S32, according to the relevant behavior status data information in the training data set, by correlating the heart rate change value Xbz, skin conductivity Pdz, respiratory frequency Hxp and respiratory depth Hsd, and after linear normalization processing, obtain the tension factor , the tension factor Obtained by the following formula: ; In the formula, , , and are all weight values, P represents the first correction constant, which is, , , and The specific value is set by the user according to the situation.

[0031] The specific steps of S3 also include: S33, based on the relevant behavior status data information in the training data set, combined with the tension factor obtained in S32 , analyze the trainees' threat perception ability under different environmental conditions, and after linear normalization, construct the pressure coefficient Ylxs, which is obtained by the following formula: ; In the formula, Expressed as the tension factor, It is expressed as the frequency of wrong judgment, Expressed as the jitter frequency, and are weight values, among which, and The specific value is set by the user according to the situation. is the second normal number.

[0032] Jitter frequency It refers to the frequency of dodging and shaking of the trainees' hands and heads during training, which can be monitored and obtained through camera equipment; In this embodiment, by performing linear normalization on multiple groups of data, the tension factor of the trainee during the training process can be effectively obtained. This accurate assessment helps to identify the psychological state of the trainee in a timely manner and provide data support for subsequent training adjustments. On the basis of the tension factor, combined with other behavioral situation information in the training data, the threat perception ability of the trainee under different environmental conditions can be deeply analyzed. By dynamically adjusting the weight value and the correction constant, the system can optimize the personalized training plan according to individual differences and specific situations. By constructing the pressure coefficient, the tension factor, the frequency of misjudgment and the frequency of jitter are combined, which helps to comprehensively evaluate the performance of the trainee in training. Such a comprehensive indicator can provide instructors with a more comprehensive analysis of the trainee status and effectively improve the pertinence of the training. By analyzing the pressure coefficient in different environments, the decision-making ability of the trainee in a high-pressure environment can be understood, and then a basis for improving their reaction ability and judgment in real situations can be provided. Combining the pressure coefficient and related behavioral situation data can form a good feedback mechanism. Instructors and trainees can adjust training strategies and priorities in a timely manner based on data analysis results. This data analysis-based method can make mine clearance training more scientific and systematic, help build a training management platform based on actual data, and improve training efficiency and effectiveness. By systematically integrating physiological data and behavioral situation analysis, the S3 step can not only enhance trainees' psychological adaptability, but also provide strong support for the scientific and personalized training. The ultimate goal is to improve trainees' coping capabilities and overall performance in real environments.

[0033] Example 5: Please refer to Figure 1 , specifically: S4 specific steps include: S41. Use deep learning technology to build a mine clearance prediction model, and input the training data set after data preprocessing into the mine clearance prediction model. After training, fit and output the auditory immersion index Tczs, and the auditory immersion index Tczs is obtained by the following formula: ; In the formula, Expressed as the comprehensive hearing intensity coefficient, Expressed as the target sound source intensity, Expressed as the pressure coefficient, and are weight values, among which, and The specific value is set by the user according to the situation.

[0034] The target sound source intensity mentioned above The data smoothing optimization method can be used to obtain it, specifically: by calculating the incremental change of the pressure coefficient under the sound source intensity, the student's immersion balance point (i.e., the pressure coefficient The specific steps are as follows: 1. The pressure coefficient measured at different sound source intensities , and obtain several groups of comprehensive hearing intensity coefficients and 2. Calculate the data points; As the sound source intensity increases, the increment is plotted based on this. The increment and comprehensive hearing intensity coefficient as the sound source intensity increases The curve is used to find the position where the incremental change tends to balance. The comprehensive hearing intensity coefficient at this time is The target sound source intensity ; The specific steps of S4 also include: S42, pre-set an evaluation threshold Q, wherein the evaluation threshold Q includes a first threshold and a second threshold, and the first threshold is greater than the second threshold, and compare and analyze the auditory immersion index Tczs with the evaluation threshold Q to comprehensively judge whether the current environmental conditions enable the trainees to be in a normal state during the mine clearance training, and the specific judgment contents are as follows: If the auditory immersion index Tczs is less than or equal to the second threshold, it is judged that the current environmental conditions make the trainees in an abnormal state during the mine clearance training. At this time, the first-level sound source intensity will be increased to increase the volume of the sound effects, especially the loudness of key sound effects such as explosions and metal collisions, increase the complexity of environmental sounds, and make the trainees more immersed in the environment; If the second threshold value is less than the auditory immersion index Tczs ≤ the first threshold value, it is judged that the current environmental conditions make the trainees in a normal state when conducting mine clearance training. At this time, the trainees will be formally trained in mine clearance under the current environmental conditions; If the auditory immersion index Tczs is greater than the first threshold, it is determined whether the current environmental conditions put the trainees in an abnormal state during mine clearance training. In this case, the intensity of the first-level sound source will be lowered, the volume of background noise will be reduced, or the high-frequency components will be reduced to reduce the psychological burden on the trainees, reduce the complexity of environmental sounds, and avoid excessive stress affecting learning outcomes.

[0035] In this embodiment, in step S33, based on the tension factor, the frequency of misjudgment and the frequency of jitter, a pressure coefficient is generated, and the trainee's ability to perceive threats is quantified through linear normalization. This method allows the trainee to adjust the perception and judgment ability of threats under different environmental conditions, which helps to cultivate the trainee's ability to cope and judge under high pressure, and gradually improve their emergency response ability. The mine clearance prediction model constructed in step S4 can adjust the sound source intensity and background noise according to the auditory immersion index through deep learning, and set the evaluation threshold Q to include the first threshold and the second threshold. It is further allowed to maintain the psychological balance of the trainee under different auditory immersion conditions by adjusting the environmental sound effects such as increasing or decreasing the loudness of key sound effects and the intensity of background noise, so that the training process is both immersive and avoids excessive pressure from interfering with the training effect. This adaptive adjustment of sound not only effectively improves the training input, but also subtly improves the trainee's psychological stress resistance. By setting multiple judgment mechanisms such as the comparison of Tczs value and Q value, the system can adjust the sound effect feedback in real time according to the trainee's auditory immersion index, so that the system can monitor and optimize the training environment at any time during the training process. Especially when the auditory immersion index is too high or too low, the corresponding sound effects are enhanced or weakened to help trainees maintain a suitable psychological state. This intelligent feedback mechanism ensures the stability and effectiveness of the training effect. Through the collection and comprehensive analysis of multi-dimensional physiological data, combined with an intelligent deep learning prediction model, this method not only enhances the trainees' immersion experience in mine clearance simulation training, but also improves the trainees' psychological adaptability and threat perception in the simulated environment through refined pressure regulation, providing more reliable psychological and technical support for actual mine clearance tasks.

[0036] Example 6: Please refer to Figure 2 ,Specifically: A mine clearance simulation training system based on ,virtual and real combination, including an environment building module, a monitoring module, a training analysis module and a simulation ,optimization module; The environment building module is used to create a three-dimensional mine clearance virtual environment in advance using virtual reality technology and augmented reality technology, and to allow trainees to enter the three-dimensional mine clearance virtual environment through wearable devices; The monitoring module is used to monitor the trainees' coefficient behavior performance under different environmental conditions in real time during the mine clearance simulation training, so as to obtain relevant behavior situation data information, and collect relevant environmental data information under corresponding environmental conditions, and generate a training data set after data preprocessing; The training analysis module analyzes the auditory pressure experienced by trainees under different test environmental conditions based on the relevant environmental data information in the training data set, so as to construct a comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; The simulation optimization module is used to construct a mine clearance prediction model using deep learning technology by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions in the mine clearance simulation training process are adjusted.

[0037] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A mine clearance simulation training method based on the combination of virtual and real, Features: The following steps are included: S1. Use virtual reality and augmented reality technologies to create a three-dimensional mine clearance virtual environment in advance, and use wearable devices to allow trainees to enter the three-dimensional mine clearance virtual environment; S2. When trainees are in the process of demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior situation data information, and relevant environmental data information under corresponding environmental conditions is collected, and after data preprocessing, a training data set is generated; S3. Based on the relevant environmental data information in the training data set, analyze the auditory pressure that the trainees are subjected to under different test environmental conditions to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; S4. Use deep learning technology to build a mine clearance prediction model by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions in the mine clearance simulation training process are adjusted.

2. A mine clearance simulation training method based on a combination of virtual and real according to claim 1, Features: The specific steps of S1 include: S11. In the mine clearance simulation training, virtual reality technology and augmented reality technology are used in advance to create a three-dimensional mine clearance virtual environment, and terrain features are reconstructed through three-dimensional modeling technology. The terrain features include mine distribution areas, vegetation, surface texture and obstacle distribution; S12. Based on the three-dimensional mine-clearing virtual environment simulated in S11, the trainees enter the three-dimensional mine-clearing virtual environment through wearable devices, wherein the wearable devices include VR head-mounted displays, AR smart glasses, sensor gloves, bracelets, wearable microphones and wearable headphones.

3. A mine clearance simulation training method based on a combination of virtual and real according to claim 2, Features: The specific steps of S2 include: S21. When the trainees are in the process of demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior status data information, wherein the relevant behavior status data information includes heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, wrong judgment frequency and jitter frequency ; S22, setting different environmental conditions and collecting relevant environmental data information under the corresponding environmental conditions, wherein the relevant environmental data information includes the number of sound sources n, the sound pressure level emitted by the corresponding sound source , the direction angle of the corresponding sound source relative to the student , the distance between the corresponding sound source and the student's head , the position coordinates of the corresponding sound source and the student's head orientation vector ; S23. Preprocess the relevant behavioral situation data information and the relevant environmental data information. The preprocessing includes noise removal, missing value filling and data smoothing operations. The missing value filling methods include mean filling, median filling, interpolation filling and regression filling. The relevant behavioral situation data information and the relevant environmental data information are scaled using dimensionless processing technology.

4. A mine clearance simulation training method based on a combination of virtual and real according to claim 3, Features: The specific steps of S3 include: S31. Based on the relevant environmental data information in the training data set, analyze the auditory pressure that the trainees are subjected to under different test environmental conditions, and after linear normalization, construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , which can be obtained by: ; Where n represents the number of sound sources, i=1, 2, 3, ..., n, It is represented by the sound pressure level emitted by the ith sound source, It is represented as the direction angle of the i-th sound source relative to the student, It is represented as the distance between the i-th sound source and the student’s head.

5. A mine clearance simulation training method based on a combination of virtual and real according to claim 4, Features: The specific steps of S3 also include: S311: The distance between the i-th sound source and the student's head Obtained by the following formula: ; S312: Direction angle of the i-th sound source relative to the student Obtained by the following formula: ; In the formula, represents the angle between the direction of the ith sound source and the orientation of the student’s head; Represented as the position coordinates of the i-th sound source; Indicates the coordinates of the student's head position; is the orientation vector of the student’s head; is the dot product of the i-th sound source propagation vector and the head direction vector; Expressed as the magnitude of the head orientation vector; The vector representing the position of the student's head pointing to the position of the i-th sound source; is the modulus of the propagation vector of the ith sound source; is the cosine value of the angle between the head direction and the propagation vector of the i-th sound source.

6. A mine clearance simulation training method based on a combination of virtual and real according to claim 5, Features: The specific steps of S3 also include: S32, according to the relevant behavior status data information in the training data set, by correlating the heart rate change value Xbz, skin conductivity Pdz, respiratory frequency Hxp and respiratory depth Hsd, and after linear normalization processing, obtain the tension factor , the tension factor Obtained by the following formula: ; In the formula, , , and are all weight values, P represents the first correction constant, which is, , , and The specific value is set by the user according to the situation.

7. A mine clearance simulation training method based on a combination of virtual and real according to claim 6, Features: The specific steps of S3 also include: S33, based on the relevant behavior status data information in the training data set, combined with the tension factor obtained in S32 , analyze the trainees' threat perception ability under different environmental conditions, and after linear normalization, construct the pressure coefficient Ylxs, which is obtained by the following formula: ; In the formula, Expressed as the tension factor, It is expressed as the frequency of wrong judgment, Expressed as the jitter frequency, and are weight values, among which, and The specific value is set by the user according to the situation. is the second normal number.

8. A mine clearance simulation training method based on a combination of virtual and real according to claim 7, Features: The specific steps of S4 include: S41. Use deep learning technology to build a mine clearance prediction model, and input the training data set after data preprocessing into the mine clearance prediction model. After training, fit and output the auditory immersion index Tczs, and the auditory immersion index Tczs is obtained by the following formula: ; In the formula, Expressed as the comprehensive hearing intensity coefficient, Expressed as the target sound source intensity, Expressed as the pressure coefficient, and are weight values, among which, and The specific value is set by the user according to the situation.

9. A mine clearance simulation training method based on a combination of virtual and real according to claim 8, Features: The specific steps of S4 also include: S42, pre-set an evaluation threshold Q, wherein the evaluation threshold Q includes a first threshold and a second threshold, and the first threshold is greater than the second threshold, and compare and analyze the auditory immersion index Tczs with the evaluation threshold Q to comprehensively judge whether the current environmental conditions enable the trainees to be in a normal state during the mine clearance training, and the specific judgment contents are as follows: If the auditory immersion index Tczs is within the second threshold, it is judged that the current environmental conditions make the trainees in an abnormal state during the mine clearance training, and the sound source intensity will be increased by one level; If the second threshold value is less than the auditory immersion index Tczs ≤ the first threshold value, it is judged that the current environmental conditions make the trainees in a normal state when conducting mine clearance training. At this time, the trainees will be formally trained in mine clearance under the current environmental conditions; If the auditory immersion index Tczs is greater than the first threshold, it is determined whether the current environmental conditions cause the trainees to be in an abnormal state during the mine clearance training. At this time, the sound source intensity will be lowered by one level.

10. A mine clearance simulation training system based on a combination of virtual and real, used to implement a mine clearance simulation training method based on a combination of virtual and real as described in any one of claims 1 to 9 above, Features: Including environment construction module, monitoring module, training analysis module and simulation optimization module; The environment building module is used to create a three-dimensional mine clearance virtual environment in advance using virtual reality technology and augmented reality technology, and to allow trainees to enter the three-dimensional mine clearance virtual environment through wearable devices; The monitoring module is used to monitor the trainees' coefficient behavior performance under different environmental conditions in real time during the mine clearance simulation training, so as to obtain relevant behavior situation data information, and collect relevant environmental data information under corresponding environmental conditions, and generate a training data set after data preprocessing; The training analysis module analyzes the auditory pressure experienced by trainees under different test environmental conditions based on the relevant environmental data information in the training data set, so as to construct a comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; The simulation optimization module is used to construct a mine clearance prediction model using deep learning technology by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions in the mine clearance simulation training process are adjusted.

Images