A mine clearance simulation training method and system based on the combination of virtual and real

Abstract

The present invention discloses a mine clearance simulation training method and system based on the combination of virtual and real, and relates to the field of simulation training technology. The present invention uses real-time monitoring data to analyze the behavioral status and physiological feedback of trainees, generates a training data set, and calculates the trainees' auditory pressure and threat perception ability based on the environmental information and behavioral data in the data set, thereby constructing a comprehensive auditory intensity coefficient #imgabs0# and a pressure coefficient Ylxs. These data are input into a mine clearance prediction model. Through the calculation result of the auditory immersion index Tczs, the sound source intensity, sound effect complexity and other parameters in the training environment are dynamically adjusted, so that the trainees are always in an appropriate immersive state. This not only avoids excessive pressure from causing tension in the trainees, but also prevents insufficient immersion from affecting the training effect, thereby achieving personalized environmental adjustment and improving the overall experience of the trainees.

Description

Technical Field

[0001] The present invention relates to the technical field of simulation training, and in particular to a mine clearance simulation training method and system based on the combination of virtual and real. Background Art

[0002] Mine clearance simulation training methods are applications of virtual reality (VR) and augmented reality (AR) technologies in the military and public security fields. With the development of virtual-reality integration technologies, simulation training plays an increasingly important role in military training, crisis response, and public security training. Virtual environments can provide a safe and controllable training method, especially in high-risk and challenging fields. Specifically, in the field of virtual-reality integration mine clearance simulation training, it aims to provide a low-risk, highly realistic training environment through interactive virtual and real-world technologies, simulating actual mine clearance scenarios without endangering the safety of trainees.

[0003] Although enhancing immersion can improve the realism of the training experience and increase trainees' task engagement, excessive immersion may have negative effects. When the immersion is too high, the details in the virtual environment may interfere with the trainees' attention too much, causing them to be seriously distracted and neglect the improvement of their actual operational skills. This problem of balancing immersion and training effectiveness may cause the system to be more inclined towards "experience" rather than "skill", causing trainees to perform well in the virtual environment but make mistakes in actual mine clearance operations.

[0004] While enhanced immersion in existing mine-clearing simulation training systems enhances the training experience, it also presents challenges in improving practical operational skills. Most current systems enhance trainees' sense of immersion by increasing the realism of the virtual environment, but this can neglect the practicality and effectiveness of training. For example, in highly realistic virtual environments, complex visual and auditory details can distract trainees and lead to a lack of focus, neglecting the development of core skills during the operation. Furthermore, some existing systems lack real-time monitoring of trainees' psychological stress, making it difficult to adjust immersion intensity at the appropriate time, which can exacerbate trainees' psychological burden. These shortcomings lead to significant discrepancies between trainees' performance in virtual training and in actual operations, resulting in a phenomenon of "paper talk" and limiting the practical value of simulation training. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the present invention provides a mine clearance simulation training method and system based on a combination of virtual and real, which solves the problems in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a mine clearance simulation training method based on the combination of virtual and real, comprising the following steps:

[0007] S1. Create a 3D mine clearance virtual environment using virtual reality and augmented reality technologies in advance, and allow trainees to enter the 3D mine clearance virtual environment through wearable devices;

[0008] S2. During the demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavioral situation data information, and relevant environmental data information under the corresponding environmental conditions is collected. After data preprocessing, a training data set is generated;

[0009] S3. Based on the relevant environmental data information in the training data set, analyze the auditory pressure experienced by the trainees under different test conditions to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs;

[0010] S4. Use deep learning technology to build a mine clearance prediction model by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization processing, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions during the mine clearance simulation training are adjusted.

[0011] Preferably, the specific steps of S1 include:

[0012] S11. During the mine clearance simulation training, virtual reality and augmented reality technologies are used in advance to create a three-dimensional mine clearance virtual environment, and terrain features are reconstructed using three-dimensional modeling technology. The terrain features include mine distribution areas, vegetation, surface texture, and obstacle distribution;

[0013] S12. Based on the three-dimensional mine clearance virtual environment simulated in S11, the trainees enter the three-dimensional mine clearance virtual environment through wearable devices, wherein the wearable devices include VR head-mounted displays, AR smart glasses, sensor gloves, wristbands, wearable microphones and wearable headphones.

[0014] Preferably, the specific steps of S2 include:

[0015] S21. When the trainees are in the process of mine clearance simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior status data information, wherein the relevant behavior status data information includes heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, error judgment frequency and jitter frequency ;

[0016] S22, set different environmental conditions and collect relevant environmental data information under the corresponding environmental conditions, wherein the relevant environmental data information includes the number of sound sources n, the sound pressure level emitted by the corresponding sound source , the direction angle of the corresponding sound source relative to the student , the distance between the corresponding sound source and the student's head , the position coordinates of the corresponding sound source and the student's head direction vector ;

[0017] S23. Preprocess the relevant behavioral situation data information and the relevant environmental data information. The preprocessing includes noise removal, missing value filling and data smoothing operations. Among them, the missing value filling methods include mean filling, median filling, interpolation filling and regression filling, and use dimensionless processing technology to scale the relevant behavioral situation data information and the relevant environmental data information.

[0018] Preferably, the specific steps of S3 include:

[0019] S31. Based on the relevant environmental data information in the training data set, analyze the auditory pressure experienced by the trainees under different test environmental conditions, and after linear normalization, construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , specifically obtained as follows: ;

[0020] Where n represents the number of sound sources, i=1, 2, 3, ..., n, It is represented as the sound pressure level emitted by the i-th sound source, It is represented as the direction angle of the i-th sound source relative to the student, It is represented as the distance between the i-th sound source and the student's head.

[0021] Preferably, the specific step S3 further includes:

[0022] S311: The distance between the i-th sound source and the student's head Obtained by the following formula: ;

[0023] S312: Direction angle of the i-th sound source relative to the student Obtained by the following formula: ;

[0024] Where, represents the angle between the direction of the i-th sound source and the orientation of the student’s head; Represented as the position coordinates of the i-th sound source; Indicates the coordinates of the student's head position; is the student's head direction vector; is the dot product of the i-th sound source propagation vector and the head direction vector; Expressed as the modulus of the head-facing vector; The vector indicating the position of the student's head pointing to the position of the i-th sound source; is the modulus of the propagation vector of the i-th sound source; is the cosine value of the angle between the head direction and the propagation vector of the i-th sound source.

[0025] Preferably, the specific step S3 further includes:

[0026] S32. According to the relevant behavioral data information in the training data set, the stress level factor is obtained by correlating the heart rate change value Xbz, skin conductivity Pdz, respiratory frequency Hxp and respiratory depth Hsd, and performing linear normalization processing. , the stress factor Obtained by the following formula: ;

[0027] Where, 、 、 and are all weight values, P represents the first correction constant, which is, 、 、 and The specific value is set by the user according to the situation.

[0028] Preferably, the specific step S3 further includes:

[0029] S33, based on the relevant behavioral situation data information in the training data set, combined with the tension factor obtained in S32 , analyze the trainees' threat perception ability under different environmental conditions, and after linear normalization, construct the pressure coefficient Ylxs. The pressure coefficient Ylxs is obtained by the following formula:

[0030] ;

[0031] Where, Expressed as the stress factor, Expressed as the frequency of wrong judgment, Expressed as the jitter frequency, and are all weight values, among which, and The specific value is set by the user according to the situation. is the second normal number.

[0032] Preferably, the specific steps of S4 include:

[0033] S41. Build a mine clearance prediction model using deep learning technology, and input the pre-processed training data set into the mine clearance prediction model. After training, fit and output the auditory immersion index Tczs. The auditory immersion index Tczs is obtained by the following formula: ;

[0034] Where, Expressed as the comprehensive hearing intensity coefficient, Expressed as the target sound source intensity, Expressed as the pressure coefficient, and are all weight values, among which, and The specific value is set by the user according to the situation.

[0035] Preferably, the specific step S4 further includes:

[0036] S42. Pre-set an evaluation threshold Q, which includes a first threshold and a second threshold, with the first threshold being greater than the second threshold. Compare and analyze the auditory immersion index Tczs with the evaluation threshold Q to comprehensively determine whether the current environmental conditions allow the trainee to be in a normal state during mine clearance training. The specific determination is as follows:

[0037] If the auditory immersion index Tczs is less than or equal to the second threshold, it is judged that the current environmental conditions make the trainees in an abnormal state during mine clearance training, and the sound source intensity will be increased by one level;

[0038] If the second threshold value is less than the auditory immersion index Tczs and is less than the first threshold value, it is determined that the current environmental conditions allow the trainees to be in a normal state for mine clearance training. In this case, the trainees will be formally trained in the current environmental conditions.

[0039] If the auditory immersion index Tczs is greater than the first threshold, it is determined whether the current environmental conditions put the trainees in an abnormal state during mine clearance training. In this case, the sound source intensity will be lowered by one level.

[0040] A mine clearance simulation training system based on the combination of virtual and real, including an environment construction module, a monitoring module, a training analysis module and a simulation optimization module;

[0041] The environment building module is used to pre-create a three-dimensional mine clearance virtual environment using virtual reality technology and augmented reality technology, and to allow trainees to enter the three-dimensional mine clearance virtual environment through wearable devices;

[0042] The monitoring module is used to monitor the trainees' coefficient behavior performance under different environmental conditions in real time during the mine clearance simulation training to obtain relevant behavior situation data information, and collect relevant environmental data information under the corresponding environmental conditions, and generate a training data set after data preprocessing;

[0043] The training analysis module analyzes the auditory pressure experienced by trainees under different test conditions based on the relevant environmental data information in the training data set, so as to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs;

[0044] The simulation optimization module is used to build a mine clearance prediction model using deep learning technology by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization processing, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions during the mine clearance simulation training are adjusted.

[0045] The present invention provides a mine clearance simulation training method and system based on a combination of virtual and real, which has the following beneficial effects:

[0046] (1) By reconstructing a real mine-clearing scene in a three-dimensional virtual environment and using wearable devices such as VR and AR, trainees can enter an immersive training environment. This combination of virtual and real can simulate various complex mine-clearing scenes, allowing trainees to practice mine-clearing skills in a simulated environment, improving their adaptability to the real environment and sense of task substitution. The present invention uses real-time monitoring data to analyze the trainees' behavioral status and physiological feedback, generate a training data set, and calculate the trainees' auditory pressure and threat perception ability based on the environmental information and behavioral data in the data set, thereby constructing a comprehensive auditory intensity coefficient. The auditory immersion index (Tczs) is calculated to dynamically adjust parameters such as the sound source intensity and sound effect complexity in the training environment, ensuring that the trainee is always in an optimal state of immersion. This not only prevents excessive pressure from causing tension in the trainee, but also prevents insufficient immersion from affecting the training effect, thereby achieving personalized environmental adjustment and improving the trainee's overall experience. By constructing a mine clearance prediction model using deep learning technology and inputting the auditory intensity coefficient and pressure coefficient into the model for training, the model can accurately predict the trainee's immersion index (Tczs) in different environments. Based on the predicted results, the training environment is dynamically adjusted to ensure that the trainee is always in the optimal state of immersion. This intelligent adjustment mechanism can effectively improve the effectiveness of training, allowing trainees to maintain a relatively optimal psychological and physiological state when facing different environmental conditions, thereby improving training efficiency. Compared with traditional static training environments, the intelligent environmental adjustment of the present invention significantly enhances the flexibility and adaptability of training, further avoiding the conflict between immersion and operational skills, and enabling the skills learned by trainees in virtual training to be better applied to actual mine clearance tasks.

[0047] (2) During the mine clearance simulation, the sound pressure level, distance from the trainee's head, and direction angle of each sound source are calculated in detail, so that the system can track complex sound scenes. By determining the direction of the sound in the environment, it can simulate the sound sources at different locations around the trainee, making their sense of tension closer to the threat perception in reality. By combining the auditory intensity coefficient The system can accurately adjust the auditory pressure during training, further ensuring that students maintain relatively realistic sound perception under interference from different sound sources. This precisely restored complex sound field environment greatly enhances the students' sense of immersion and trains their ability to quickly respond in high-noise, complex sound source environments, helping to improve their adaptability to real mine clearance scenarios. Through the comprehensive calculation and linear normalization of the sound pressure level, direction angle and distance of multiple sound sources, the comprehensive auditory intensity coefficient The auditory pressure data under different environments is standardized. This standardized auditory intensity coefficient ensures that the system can effectively control auditory pressure under various environmental conditions, further avoiding the psychological burden caused by excessively loud sounds or lack of immersion caused by excessively weak sounds during training.

[0048] (3) Under different environmental conditions, the trainees’ threat perception ability is further analyzed in combination with the tension factor, and a stress coefficient Ylxs is generated. This coefficient can truly reflect the trainees’ psychological load level in a specific environment and effectively ensure the safety of training. The auditory immersion index Tczs is predicted through a deep learning model. Based on this index, the system can automatically adjust the sound intensity and background sound effects in the training environment. When the auditory immersion is too low, the complexity of the sound effects is increased to enhance the sense of involvement; when it is too high, the sound intensity is reduced to further avoid excessive psychological pressure affecting the training effect. This method can provide personalized adjustments based on the trainees’ actual status and avoid the training quality being affected by excessive pressure or insufficient immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of a mine clearance simulation training method based on a combination of virtual and real aspects of the present invention;

[0050] Figure 2 This is a block diagram of a mine clearance simulation training system based on the combination of virtual and real in the present invention. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] Example 1: Please refer to Figure 1 The present invention provides a mine clearance simulation training method based on the combination of virtual and real, comprising the following steps:

[0053] S1. Create a 3D mine clearance virtual environment using virtual reality and augmented reality technologies in advance, and allow trainees to enter the 3D mine clearance virtual environment through wearable devices;

[0054] S2. During the demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavioral situation data information, and relevant environmental data information under the corresponding environmental conditions is collected. After data preprocessing, a training data set is generated;

[0055] S3. Based on the relevant environmental data information in the training data set, analyze the auditory pressure experienced by the trainees under different test conditions to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs;

[0056] S4. Use deep learning technology to build a mine clearance prediction model by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization processing, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions during the mine clearance simulation training are adjusted.

[0057] In this embodiment, by using virtual reality technology and augmented reality technology to construct a three-dimensional mine clearance virtual environment in step S1, the trainees can experience a realistic mine clearance situation in person. With the support of wearable devices, the trainees can obtain highly realistic visual, auditory and tactile feedback in the virtual environment, which helps to enhance the immersion of training, makes it easier for them to enter the training state, and improves the quality of the training experience. In step S2, by real-time monitoring of the trainees' behavioral performance under different environmental conditions, relevant behavioral situation data and environmental data are obtained, and a training data set is generated through data preprocessing. This real-time data collection and monitoring method can not only help the system dynamically grasp the trainees' behavioral patterns and physiological states, but also effectively identify the trainees' stress states and adaptation conditions during the training process. By processing and analyzing the data, the training environment can be appropriately adjusted according to the trainees' performance to ensure that the trainees receive a realistic training experience within a safe range. By analyzing the training data in steps S3 and S4, the comprehensive auditory intensity coefficient is calculated. The system uses deep learning technology to construct a mine clearance prediction model and fits the output auditory immersion index Tczs. Based on the feedback from the auditory immersion index Tczs, the system can intelligently adjust the sound intensity and complexity of the training environment, thereby providing trainees with a personalized training experience under different environmental conditions. This intelligent adjustment mechanism not only avoids the psychological stress and fatigue caused by excessive immersion, but also enhances the environmental intensity when trainees are undertrained, ensuring that trainees receive more effective skills training and achieve relatively optimal training results. In summary, the mine clearance simulation training method provided by this method achieves a balance between immersion and training effectiveness through a combined virtual and real training environment and an intelligent adjustment mechanism. This allows trainees to obtain a highly realistic immersive experience without excessively distracting their attention, enabling them to better master practical operational skills, thereby significantly improving training quality and safety.

[0058] Example 2: Please refer to Figure 1 , specifically: S1 specific steps include:

[0059] S11. During the mine clearance simulation training, virtual reality and augmented reality technologies are used in advance to create a three-dimensional mine clearance virtual environment. Three-dimensional modeling technology is used to reconstruct terrain features, including mine distribution areas, vegetation, surface texture, and obstacle distribution. This allows trainees to experience a situation similar to that of a real mine clearance mission.

[0060] S12. Based on the three-dimensional mine clearance virtual environment simulated in S11, the trainees enter the three-dimensional mine clearance virtual environment through wearable devices, wherein the wearable devices include VR head-mounted displays, AR smart glasses, sensor gloves, wristbands, wearable microphones and wearable headphones. The wearable devices capture the trainees' head, hand, and even body movements, and synchronize them in real time in the virtual environment to achieve precise interaction.

[0061] The specific steps of S2 include:

[0062] S21. When the trainees are in the process of mine clearance simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior status data information, wherein the relevant behavior status data information includes heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, error judgment frequency and jitter frequency ;

[0063] The above-mentioned heart rate change value Xbz is monitored and obtained by a photoelectric heart rate sensor;

[0064] Skin conductivity Pdz is monitored and obtained by a galvanic skin response (GSR) sensor;

[0065] The respiratory rate Hxp and respiratory depth Hsd are monitored and obtained through the respiratory belt sensor;

[0066] Frequency of incorrect judgment This can be monitored and obtained through training management software. By recording the decision-making results of trainees in virtual training (determining whether there are mines), the number of wrong judgments can be automatically calculated to obtain the frequency of wrong judgments.

[0067] S22, set different environmental conditions and collect relevant environmental data information under the corresponding environmental conditions, wherein the relevant environmental data information includes the number of sound sources n, the sound pressure level emitted by the corresponding sound source , the direction angle of the corresponding sound source relative to the student , the distance between the corresponding sound source and the student's head , the position coordinates of the corresponding sound source and the student's head direction vector ;

[0068] The above-mentioned relevant environmental data information can be used to track the trainee's movement and position in the virtual environment through the inertial measurement unit (IMU). The sensors of the VR helmet and handle can accurately track the position of the trainee's head and hand, realize real-time update of spatial coordinates, and embed the coordinate system in the virtual environment through environmental modeling technology to record the trainee's position information in real time.

[0069] S23. Preprocess the relevant behavioral situation data information and the relevant environmental data information. The preprocessing includes noise removal, missing value filling and data smoothing operations. Among them, the missing value filling methods include mean filling, median filling, interpolation filling and regression filling. The relevant behavioral situation data information and the relevant environmental data information are scaled using dimensionless processing technology so that their data range falls between 0 and 1.

[0070] In this embodiment, a three-dimensional mine clearance environment is constructed using virtual reality and augmented reality technologies during mine clearance simulation training, simulating realistic terrain features such as minefield distribution, vegetation, surface texture, and obstacles. Trainees, using wearable devices such as VR head-mounted displays and AR smart glasses, can not only truly perceive the details of their surroundings but also interact with the virtual environment through feedback from devices such as sensor gloves and wristbands. This highly simulated training environment significantly enhances trainees' sense of task involvement and immersion, enabling them to more quickly adapt to and master various skills in actual mine clearance scenarios, thereby increasing the effectiveness and authenticity of the training. This method achieves multi-dimensional monitoring of trainees' psychological stress, behavioral state, and responses to the environment by monitoring their heart rate variability (Xbz), skin conductivity (Pdz), respiratory rate (Hxp), respiratory depth (Hsd), frequency of misjudgments, and jitter frequency in real time during training. The system also records key parameters of the training environment, such as the number of sound sources, sound pressure level, directional angle, and distance from the trainee's head. This multi-dimensional data collection and analysis mechanism enables the system to obtain real-time physiological and psychological feedback from trainees, adjust training content based on this feedback, and provide personalized feedback guidance, thereby improving the adaptability of the training process and the accuracy of feedback. After data collection, the system performs comprehensive preprocessing on behavioral and environmental data, including noise removal, missing value filling, and data smoothing, to ensure data accuracy and consistency. This preprocessing not only improves data quality but also reduces the impact of environmental noise and missing data on training effectiveness. Various imputation methods, such as mean imputation, median imputation, and interpolation, ensure appropriate data completion in various missing data scenarios. Furthermore, dimensionless processing technology normalizes the data to a uniform range of 0 to 1, facilitating subsequent model training and predictive analysis. This systematic preprocessing approach ensures data integrity and consistency, improves the reliability of model predictions, and ensures continuous optimization of the training environment. In summary, this method improves the authenticity, feedback accuracy, and data quality of mine clearance simulation training through an immersive virtual environment, intelligent real-time monitoring, and data preprocessing, providing trainees with a more effective and safe training experience and significantly enhancing the intelligence level of the training model.

[0071] Example 3: Please refer to Figure 1 , specifically: S3 specific steps include:

[0072] S31. Based on the relevant environmental data information in the training data set, analyze the auditory pressure experienced by the trainees under different test environmental conditions, and after linear normalization, construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , specifically obtained as follows:

[0073] ;

[0074] Where n represents the number of sound sources, i=1, 2, 3, ..., n, It is expressed as the sound pressure level (intensity) emitted by the i-th sound source, It is represented as the direction angle of the i-th sound source relative to the student, It is represented as the distance between the i-th sound source and the student's head.

[0075] The specific steps of S3 also include:

[0076] S311: The distance between the i-th sound source and the student's head Obtained by the following formula: ;

[0077] S312: Direction angle of the i-th sound source relative to the student Obtained by the following formula: ;

[0078] Where, represents the angle between the direction of the i-th sound source and the orientation of the student’s head; Represented as the position coordinates of the i-th sound source, it is the specific location where the sound is emitted in the virtual environment; Indicates the coordinates of the student's head position, which is the student's actual current coordinate position in the virtual environment; is the student's head heading vector, indicating the current direction of the student's head; is the dot product of the i-th sound source propagation vector and the head direction vector, indicating the relationship between the two vectors; It is expressed as the modulus of the head orientation vector, i.e., the length of the unit vector of the student’s head orientation; The vector that indicates the position of the student's head pointing to the position of the i-th sound source, also known as the sound propagation vector; is the modulus (length) of the propagation vector of the i-th sound source; is the cosine value of the angle between the head direction and the propagation vector of the i-th sound source, which is calculated to angle.

[0079] Comprehensive hearing intensity coefficient It reflects the total sound pressure level of the integrated sound intensity transmitted to the students' ears;

[0080] In this embodiment, by collecting data from multiple sound sources in the training environment, the sound pressure level, direction angle and distance from the trainee's head of each sound source are calculated, and a comprehensive hearing intensity coefficient is constructed based on these parameters. , which enables the system to accurately quantify the auditory pressure of trainees during training. This coefficient intuitively reflects the comprehensive impact of sound sources on trainees under specific environmental conditions. By adjusting factors such as the sound pressure level and direction angle in the environment, it can provide trainees with an immersive experience closer to the real scene, thereby improving the training effect and the trainees' situational adaptability. The distance and direction angle between each sound source and the trainee's head are dynamically calculated through formulas, allowing the system to provide real-time feedback on the spatial information of the sound. Specifically, the system accurately calculates the relative position relationship between each sound source and the trainee in the virtual environment, including changes in distance and angle, so that the auditory impact of different sound source positions on the trainees is more realistic. This dynamic feedback enables trainees to perceive changes in the direction and distance of the sound source, thereby more realistically simulating the sound interference from all directions in a real mine clearance scene, improving the trainees' judgment and anti-interference ability in high-noise environments. In summary, this method realizes intelligent auditory pressure regulation in mine clearance training through dynamic calculation and comprehensive control of the sound pressure levels, distances and directional angles of multiple sound sources, improves the training immersion and situational realism, and helps trainees better adapt to the auditory challenges of real mine clearance tasks in a virtual environment.

[0081] Example 4: Please refer to Figure 1 , specifically: S3 specific steps also include:

[0082] S32. According to the relevant behavioral data information in the training data set, the stress level factor is obtained by correlating the heart rate change value Xbz, skin conductivity Pdz, respiratory frequency Hxp and respiratory depth Hsd, and performing linear normalization processing. , the tension factor Obtained by the following formula: ;

[0083] Where, 、 、 and are all weight values, P represents the first correction constant, which is, 、 、 and The specific value is set by the user according to the situation.

[0084] The specific steps of S3 also include:

[0085] S33, based on the relevant behavioral situation data information in the training data set, combined with the tension factor obtained in S32 , analyze the trainees' threat perception ability under different environmental conditions, and after linear normalization, construct the pressure coefficient Ylxs. The pressure coefficient Ylxs is obtained by the following formula: ;

[0086] Where, Expressed as the stress factor, Expressed as the frequency of wrong judgment, Expressed as the jitter frequency, and are all weight values, among which, and The specific value is set by the user according to the situation. is the second normal number.

[0087] Jitter frequency It refers to the frequency of dodging and shaking of the trainees' hands and heads during training, which can be monitored and obtained through camera equipment;

[0088] In this embodiment, by linearly normalizing multiple sets of data, the student's stress factor during training can be effectively determined. This precise assessment helps to timely identify the student's psychological state and provide data support for subsequent training adjustments. Based on the stress factor and combined with other behavioral status information in the training data, a deep analysis of the student's threat perception ability under different environmental conditions can be achieved. By dynamically adjusting weights and correction constants, the system can optimize personalized training plans based on individual differences and specific situations. By constructing a stress coefficient that combines factors such as stress factor, misjudgment frequency, and jitter frequency, it helps to comprehensively evaluate the student's performance during training. This comprehensive indicator can provide instructors with a more comprehensive analysis of the student's status, effectively improving the targeted nature of training. By analyzing the stress coefficient in different environments, it is possible to understand the student's decision-making ability in high-pressure environments, thereby providing a basis for improving their reaction ability and judgment in real-world situations. Combining the stress coefficient and related behavioral status data can form a sound feedback mechanism. Instructors and trainees can adjust training strategies and priorities in a timely manner based on data analysis results. This data-analysis-based approach can make mine clearance training more scientific and systematic, help build a training management platform based on actual data, and improve training efficiency and effectiveness. By systematically integrating physiological data and behavioral situation analysis, the S3 step can not only enhance trainees' psychological adaptability, but also provide strong support for the scientific and personalized training. The ultimate goal is to improve trainees' coping capabilities and overall performance in real-world environments.

[0089] Example 5: Please refer to Figure 1 , specifically: S4 specific steps include:

[0090] S41. Build a mine clearance prediction model using deep learning technology, and input the pre-processed training data set into the mine clearance prediction model. After training, fit and output the auditory immersion index Tczs. The auditory immersion index Tczs is obtained by the following formula: ;

[0091] Where, Expressed as the comprehensive hearing intensity coefficient, Expressed as the target sound source intensity, Expressed as the pressure coefficient, and are all weight values, among which, and The specific value is set by the user according to the situation.

[0092] The target sound source intensity mentioned above The data smoothing optimization method can be used to obtain it, specifically: by calculating the incremental change of the pressure coefficient under the sound source intensity, the student's immersion balance point (i.e., the pressure coefficient The specific steps are: 1. The pressure coefficient measured at different sound source intensities , and obtain several groups of comprehensive hearing intensity coefficients and Data points; 2. Calculation As the sound source intensity increases, the increment is plotted based on this. The increment and comprehensive hearing intensity coefficient as the sound source intensity increases The curve is used to find the position where the incremental change tends to be balanced. The comprehensive hearing intensity coefficient at this time is The target sound source intensity ;

[0093] The specific steps of S4 also include:

[0094] S42. Pre-set an evaluation threshold Q, which includes a first threshold and a second threshold, with the first threshold being greater than the second threshold. Compare and analyze the auditory immersion index Tczs with the evaluation threshold Q to comprehensively determine whether the current environmental conditions allow the trainee to be in a normal state during mine clearance training. The specific determination is as follows:

[0095] If the auditory immersion index Tczs is less than or equal to the second threshold, it is judged that the current environmental conditions put the trainee in an abnormal state during mine clearance training. In this case, the intensity of the first-level sound source will be increased, and the volume of the sound effects will be increased. In particular, the loudness of key sound effects such as explosions and metal collisions will be increased, increasing the complexity of the environmental sounds and making the trainees more immersed in the environment.

[0096] If the second threshold value is less than the auditory immersion index Tczs and is less than the first threshold value, it is determined that the current environmental conditions allow the trainees to be in a normal state for mine clearance training. In this case, the trainees will be formally trained in the current environmental conditions.

[0097] If the auditory immersion index Tczs is greater than the first threshold, it is determined whether the current environmental conditions put the trainees in an abnormal state during mine clearance training. In this case, the intensity of the sound source will be lowered by one level, the volume of background noise will be reduced, or the high-frequency components will be reduced to relieve the psychological burden of the trainees, reduce the complexity of the environmental sounds, and avoid excessive stress affecting learning outcomes.

[0098] In this embodiment, in step S33, a stress coefficient is generated based on the stress factor, misjudgment frequency, and jitter frequency. This is then linearly normalized to quantify the trainee's threat perception. This method allows for adjustments to the trainee's threat perception and judgment abilities under different environmental conditions, helping to cultivate their coping and judgment abilities under high-pressure situations and gradually enhance their emergency response capabilities. The mine clearance prediction model constructed in step S4 uses deep learning to adjust the sound source intensity and background noise according to the auditory immersion index. The evaluation threshold Q is set to include a first threshold and a second threshold. This further allows for adjustments to environmental sound effects, such as increasing or decreasing the loudness of key sounds and the intensity of background noise, under different auditory immersion conditions to maintain the trainee's psychological balance. This ensures that the training process is both immersive and prevents excessive stress from interfering with training effectiveness. This adaptive sound adjustment not only effectively increases training engagement but also subtly enhances the trainee's psychological resilience. By establishing multiple judgment mechanisms, such as comparing the Tczs value with the Q value, the system can adjust sound effect feedback in real time based on the trainee's auditory immersion index, enabling the system to monitor and optimize the training environment at any time during training. In particular, when the auditory immersion index is too high or too low, the corresponding sound effects are amplified or weakened to help trainees maintain a suitable psychological state. This intelligent feedback mechanism ensures the stability and effectiveness of training results. By collecting and comprehensively analyzing multi-dimensional physiological data and combining it with an intelligent deep learning prediction model, this method not only enhances the trainees' immersive experience during mine clearance simulation training, but also improves their psychological adaptability and threat perception in the simulated environment through refined stress regulation, providing more reliable psychological and technical support for actual mine clearance tasks.

[0099] Example 6: Please refer to Figure 2 ,Specifically: A mine clearance simulation training system based on the combination of virtual and real,,including an environment construction module, a monitoring module, a training analysis module, and a simulation optimization module;

[0100] The environment building module is used to pre-create a three-dimensional mine clearance virtual environment using virtual reality technology and augmented reality technology, and to allow trainees to enter the three-dimensional mine clearance virtual environment through wearable devices;

[0101] The monitoring module is used to monitor the trainees' coefficient behavior performance under different environmental conditions in real time during the mine clearance simulation training to obtain relevant behavior situation data information, and collect relevant environmental data information under the corresponding environmental conditions, and generate a training data set after data preprocessing;

[0102] The training analysis module analyzes the auditory pressure experienced by trainees under different test conditions based on the relevant environmental data information in the training data set, so as to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs;

[0103] The simulation optimization module is used to build a mine clearance prediction model using deep learning technology by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization processing, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions during the mine clearance simulation training are adjusted.

[0104] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A mine clearance simulation training method based on a combination of virtual and real, characterized by: The following steps are included: S1. Create a 3D mine clearance virtual environment using virtual reality and augmented reality technologies in advance, and allow trainees to enter the 3D mine clearance virtual environment through wearable devices; S2. During the demining simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavioral situation data information, and relevant environmental data information under the corresponding environmental conditions is collected. After data preprocessing, a training data set is generated; S3. Based on the relevant environmental data information in the training data set, analyze the auditory pressure experienced by the trainees under different test conditions to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; S4. Use deep learning technology to build a mine clearance prediction model by combining the comprehensive hearing intensity coefficient The trainees' pressure coefficient Ylxs under the corresponding environmental conditions is input into the mine clearance prediction model. After linear normalization, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions during the mine clearance simulation training are adjusted. The specific steps of S3 include: S31. Based on the relevant environmental data information in the training data set, analyze the auditory pressure experienced by the trainees under different test environmental conditions, and after linear normalization, construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , specifically obtained as follows: ； Where n represents the number of sound sources, i=1, 2, 3, ..., n, It is represented as the sound pressure level emitted by the i-th sound source, It is represented as the direction angle of the i-th sound source relative to the student, It is represented as the distance between the i-th sound source and the student’s head; The specific steps of S3 also include: S32. According to the relevant behavioral data information in the training data set, the stress level factor is obtained by correlating the heart rate change value Xbz, skin conductivity Pdz, respiratory frequency Hxp and respiratory depth Hsd, and performing linear normalization processing. , the tension factor Obtained by the following formula: ； Where, 、 、 and are all weight values, P represents the first correction constant, which is, 、 、 and The specific value is set by the user according to the situation; The specific steps of S3 also include: S33, based on the relevant behavioral situation data information in the training data set, combined with the tension factor obtained in S32 , analyze the trainees' threat perception ability under different environmental conditions, and after linear normalization, construct the pressure coefficient Ylxs. The pressure coefficient Ylxs is obtained by the following formula: ; Where, Expressed as the stress factor, Expressed as the frequency of wrong judgment, Expressed as the jitter frequency, and are all weight values, among which, and The specific value is set by the user according to the situation. is the second positive constant; The specific steps of S4 include: S41. Build a mine clearance prediction model using deep learning technology, and input the pre-processed training data set into the mine clearance prediction model. After training, fit and output the auditory immersion index Tczs. The auditory immersion index Tczs is obtained by the following formula: ； Where, Expressed as the comprehensive hearing intensity coefficient, Expressed as the target sound source intensity, Expressed as the pressure coefficient, and are all weight values, among which, and The specific value is set by the user according to the situation.

2. The mine clearance simulation training method based on the combination of virtual and real according to claim 1 is characterized by: The specific steps of S1 include: S11. During the mine clearance simulation training, virtual reality and augmented reality technologies are used in advance to create a three-dimensional mine clearance virtual environment, and terrain features are reconstructed using three-dimensional modeling technology. The terrain features include mine distribution areas, vegetation, surface texture, and obstacle distribution; S12. Based on the three-dimensional mine clearance virtual environment simulated in S11, the trainees enter the three-dimensional mine clearance virtual environment through wearable devices, wherein the wearable devices include VR head-mounted displays, AR smart glasses, sensor gloves, wristbands, wearable microphones and wearable headphones.

3. The mine clearance simulation training method based on the combination of virtual and real according to claim 2 is characterized by: The specific steps of S2 include: S21. When the trainees are in the process of mine clearance simulation training, the trainees' coefficient behavior performance under different environmental conditions is monitored in real time to obtain relevant behavior status data information, wherein the relevant behavior status data information includes heart rate change value Xbz, skin conductivity Pdz, respiratory rate Hxp, respiratory depth Hsd, error judgment frequency and jitter frequency ; S22, set different environmental conditions and collect relevant environmental data information under the corresponding environmental conditions, wherein the relevant environmental data information includes the number of sound sources n, the sound pressure level emitted by the corresponding sound source , the direction angle of the corresponding sound source relative to the student , the distance between the corresponding sound source and the student's head , the position coordinates of the corresponding sound source and the student's head direction vector ; S23. Preprocess the relevant behavioral situation data information and the relevant environmental data information. The preprocessing includes noise removal, missing value filling and data smoothing operations. Among them, the missing value filling methods include mean filling, median filling, interpolation filling and regression filling, and use dimensionless processing technology to scale the relevant behavioral situation data information and the relevant environmental data information.

4. The mine clearance simulation training method based on the combination of virtual and real according to claim 3 is characterized by: The specific steps of S3 also include: S311: The distance between the i-th sound source and the student's head Obtained by the following formula: ; S312: Direction angle of the i-th sound source relative to the student Obtained by the following formula: ; Where, represents the angle between the direction of the i-th sound source and the orientation of the student’s head; Represented as the position coordinates of the i-th sound source; Indicates the coordinates of the student's head position; is the student's head direction vector; is the dot product of the i-th sound source propagation vector and the head direction vector; Expressed as the modulus of the head-facing vector; The vector indicating the position of the student's head pointing to the position of the i-th sound source; is the modulus of the propagation vector of the i-th sound source; is the cosine value of the angle between the head direction and the propagation vector of the i-th sound source.

5. The mine clearance simulation training method based on the combination of virtual and real according to claim 4 is characterized in that: The specific steps of S4 also include: S42. Pre-set an evaluation threshold Q, which includes a first threshold and a second threshold, with the first threshold being greater than the second threshold. Compare and analyze the auditory immersion index Tczs with the evaluation threshold Q to comprehensively determine whether the current environmental conditions allow the trainee to be in a normal state during mine clearance training. The specific determination is as follows: If the auditory immersion index Tczs is less than or equal to the second threshold, it is judged that the current environmental conditions make the trainees in an abnormal state during mine clearance training, and the sound source intensity will be increased by one level; If the second threshold value is less than the auditory immersion index Tczs and is less than the first threshold value, it is determined that the current environmental conditions allow the trainees to be in a normal state for mine clearance training. In this case, the trainees will be formally trained in the current environmental conditions. If the auditory immersion index Tczs is greater than the first threshold, it is determined whether the current environmental conditions put the trainees in an abnormal state during mine clearance training. In this case, the sound source intensity will be lowered by one level.

6. A mine clearance simulation training system based on a combination of virtual and real, used to implement the mine clearance simulation training method based on a combination of virtual and real as described in any one of claims 1 to 5, characterized in that: Including environment construction module, monitoring module, training analysis module and simulation optimization module; The environment building module is used to pre-create a three-dimensional mine clearance virtual environment using virtual reality technology and augmented reality technology, and to allow trainees to enter the three-dimensional mine clearance virtual environment through wearable devices; The monitoring module is used to monitor the trainees' coefficient behavior performance under different environmental conditions in real time during the mine clearance simulation training to obtain relevant behavior situation data information, and collect relevant environmental data information under the corresponding environmental conditions, and generate a training data set after data preprocessing; The training analysis module analyzes the auditory pressure experienced by trainees under different test conditions based on the relevant environmental data information in the training data set, so as to construct the comprehensive auditory intensity coefficient under the corresponding environmental conditions. , and combined with the relevant behavioral situation data information in the training data set, analyze the trainees' threat perception ability under different environmental conditions to construct the pressure coefficient Ylxs; The simulation optimization module is used to build a mine clearance prediction model using deep learning technology by combining the comprehensive hearing intensity coefficient The pressure coefficient Ylxs of the trainees under the corresponding environmental conditions is input into the mine clearance prediction model, and after linear normalization processing, the auditory immersion index Tczs is fitted and output. Based on the auditory immersion index Tczs, the environmental conditions during the mine clearance simulation training are adjusted.

Images