Immersive teaching system and method based on virtual reality

Abstract

The invention discloses an immersive teaching system and method based on virtual reality, particularly relates to the technical field of immersive teaching of virtual reality, and quantifies hidden dangers existing in a data fusion process by obtaining and calculating a weight adjustment delay coefficient, an error data interference propagation coefficient and a sensor data conflict coefficient. A data fusion hidden danger assessment model is constructed according to the data fusion hidden danger assessment model, real-time monitoring and quantitative analysis of fusion hidden dangers are achieved, an output data fusion hidden danger assessment index visually reflects the severity degree of potential problems in the multi-sensor fusion process, intelligent early warning is conducted on a teaching system, early warning prompts are triggered in time, and the teaching efficiency is improved. According to the method, a user or a system manager is reminded to take corresponding adjustment measures, so that negative effects caused by error propagation are reduced, the problem of dislocation between a virtual scene and actual actions of the user is effectively avoided, the interaction accuracy and immersion are improved, and the reliability of multi-sensor data fusion in a virtual reality immersion teaching system is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of immersive teaching in virtual reality. More specifically, the present invention relates to an immersive teaching system and method based on virtual reality. Background Art

[0002] As an emerging immersive interaction means, virtual reality technology has been widely used in fields such as education, training, and scientific research. Especially in immersive teaching systems, it can provide a highly realistic virtual learning environment, improving teaching efficiency and learning experience. Immersive teaching systems based on virtual reality usually rely on multiple sensors (such as gyroscopes, accelerometers, cameras, infrared sensors, lidar, etc.) to achieve precise spatial positioning and attitude tracking, enabling users to interact naturally in the virtual environment. However, the fusion accuracy of sensor data directly affects the immersion and interaction effects of the virtual reality system.

[0003] In a VR system, multi-sensor data fusion usually adopts information such as inertial measurement unit data, optical tracking data, and depth sensor data, and uses data fusion algorithms such as Kalman filtering, extended Kalman filtering, and particle filtering to improve the estimation accuracy of position and attitude. However, in complex environments or during long-term use, problems such as data drift, noise accumulation, and clock synchronization errors may occur, resulting in errors in the sensor fusion results.

[0004] Especially in teaching applications, since existing VR systems often rely on a black-boxed sensor data processing process, if there are potential hidden dangers in the multi-sensor fusion process, but these hidden dangers do not bring obvious manifestations in a short period of time, the existing teaching systems may not be able to perceive their existence in time, which may lead to a misalignment between the virtual scene and the user's actual actions, making the user's interaction behavior in the virtual world not match the actual actions. As the system runs for a longer time, the tracking error may gradually accumulate, which may affect the accuracy of spatial operations during teaching, reduce the user's immersion, and may cause physiological discomforts such as dizziness, balance disorder, and visual fatigue. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide an immersive teaching system and method based on virtual reality to solve the problems raised in the above background art.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An immersive teaching method based on virtual reality, comprising the following steps:

[0008] Step S1, obtaining the weight adjustment delay information in the multi-sensor data fusion process, and obtaining the weight adjustment delay coefficient according to the weight adjustment delay information;

[0009] Step S2, obtain the error data interference propagation information in the multi-sensor data fusion process, and obtain the error data interference propagation coefficient according to the error data interference propagation information;

[0010] Step S3, obtain the sensor data conflict information in the multi-sensor data fusion process, and obtain the sensor data conflict coefficient according to the sensor data conflict information;

[0011] Step S4, construct a data fusion hidden danger assessment model based on the weight adjustment delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, output the data fusion hidden danger assessment index, and evaluate the potential hidden danger degree of the multi-sensor data fusion process;

[0012] Step S5, give a warning prompt for the existing virtual reality immersive teaching system according to the potential hidden danger degree of the multi-sensor data fusion process.

[0013] In a preferred embodiment, by obtaining the weight adjustment delay information in the multi-sensor data fusion process, analyzing the response situation of the weight adjustment in the multi-sensor data fusion process, and obtaining the weight adjustment delay coefficient, to measure the delay degree of the weight adjustment in the multi-sensor data fusion process;

[0014] The acquisition logic of the weight adjustment delay coefficient is as follows:

[0015] Obtain the weight values of a single sensor at two consecutive time steps t and t-Δt, and calculate the weight adjustment rate: where ΔW represents the weight adjustment rate of the sensor, W(t) represents the weight value of the sensor at time step t, W(t-Δt) represents the weight value of the sensor at time step t-Δt, and Δt represents the time interval of weight update; calculate the average value of the weight adjustment rate: where Wdelay represents the average value of the weight adjustment rate, ΔW i represents the weight adjustment rate calculated when the i-th sensor makes a weight adjustment, i = {1, 2,..., I}, I is a positive integer; compare the weight adjustment rate calculated when the sensor makes a weight adjustment with the average value of the weight adjustment rate. If the weight adjustment rate calculated when the sensor makes a weight adjustment is less than the average value of the weight adjustment rate, mark the weight adjustment rate calculated at this time as the weight adjustment delay rate; calculate the weight adjustment delay coefficient, and the expression is as follows: where Qzt represents the weight adjustment delay coefficient, ΔWy mn represents the weight adjustment rate at which the m-th sensor is marked as the weight adjustment delay rate for the n-th time, n = {1, 2,..., N}, N is a positive integer, m = {1, 2,..., M}, M is a positive integer.

[0016] In a preferred embodiment, by obtaining the error data interference propagation information in the multi-sensor data fusion process, analyzing the propagation interference situation of the error data between different sensors, and obtaining the error data interference propagation coefficient to measure the degree of propagation interference of the error data between different sensors;

[0017] The acquisition logic of the error data interference propagation coefficient is as follows:

[0018] Obtain the measurement value Cl k (t) of the k-th sensor at time t, and calculate the error residual: En k (t) = |Cl k (t) - Ctrue k (t)|, where En k (t) represents the error residual of the k-th sensor at time t, and Ctrue k (t) represents the expected true value; calculate the error propagation correlation coefficient: where ρ kh represents the error propagation correlation coefficient between sensors k and h, represents the mean value of the error residuals of sensor k, En h (t) represents the error residual of the h-th sensor at time t, represents the mean value of the error residuals of sensor h, T is a positive integer; construct the error propagation influence matrix MZ, and its matrix elements are: where mz kh is an element in the error propagation influence matrix, and ∈ is a very small positive value used to avoid the denominator being zero; calculate the error data interference propagation coefficient, and the expression is as follows: where k = {1, 2,..., K}, h = {1, 2,..., K}, K is a positive integer and h ≠ k.

[0019] In a preferred embodiment, by obtaining the sensor data conflict information in the multi-sensor data fusion process, analyzing the data conflict situation of the sensors in the data fusion process, and obtaining the sensor data conflict coefficient to measure the degree of data conflict of the sensors in the data fusion process;

[0020] The acquisition logic of the sensor data conflict coefficient is as follows:

[0021] Obtain the multi-dimensional conflict distance of the sensor measurement values: where represents the multi-dimensional conflict distance of the sensor measurement values, represents the data vector of the measurement value of the x-th sensor, The data vector representing the measurement value of the y-th sensor, denotes the transpose of the vector, ∑ -1 represents the inverse matrix of the covariance matrix of the measurement data; calculate the mutual information of sensor data conflict: where represents the mutual information of sensor data conflict, p(a, b) represents and the joint probability distribution between, p(a) represents the marginal probability distribution of, p(b) represents the marginal probability distribution of, a takes values in the data vector and b takes values in the data vector ;

[0022] Calculate the conflict influence coefficient: where CT xy represents the conflict influence coefficient between sensor x and sensor y;

[0023] Calculate the information entropy of each sensor data: where Hs represents the information entropy of sensor data, p(q) represents the probability that the measurement value of the sensor falls into the interval q, q = {1, 2,..., Q}, and Q is a positive integer;

[0024] Calculate the sensor data conflict coefficient, and the expression is as follows: where CGC represents the sensor data conflict coefficient, Hs x represents the information entropy of the x-th sensor, Hs y represents the information entropy of the y-th sensor, x = {1, 2,..., X}, y = {1, 2,..., X}, X is a positive integer and x ≠ y.

[0025] In a preferred embodiment, a data fusion hidden danger assessment model is constructed according to the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, and the data fusion hidden danger assessment index SJR is output. The formula on which the model is based is as follows In the formula, d1, d2, and d3 respectively represent the preset proportionality coefficients of the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, and d1, d2, and d3 are all greater than 0.

[0026] In a preferred embodiment, the data fusion hidden danger assessment index is compared with a preset data fusion hidden danger assessment index threshold to determine the potential hidden danger degree of the multi-sensor data fusion process, as follows:

[0027] If the data fusion hidden danger assessment index is greater than the data fusion hidden danger assessment index threshold, a fusion hidden danger signal is generated;

[0028] If the data fusion hidden danger assessment index is less than or equal to the data fusion hidden danger assessment index threshold, there is no need to generate a fusion hidden danger signal.

[0029] In a preferred embodiment, when there is a fusion hidden danger signal, multiple data fusion hidden danger assessment indexes output by the data fusion hidden danger assessment model are obtained, a fusion hidden danger data set is established, and the fusion hidden danger data set is marked as rhy = {SJR β}, where β = {1, 2, 3,..., γ}, and γ is a positive integer;

[0030] Calculate the average value of the data fusion hidden danger assessment indexes in the fusion hidden danger data set. The expression is as follows: Where represents the average value of the data fusion hidden danger assessment indexes in the fusion hidden danger data set, and SJR β represents the β-th output data fusion hidden danger assessment index;

[0031] Calculate the standard deviation of the data fusion hidden danger assessment indexes in the fusion hidden danger data set. The expression is as follows: Where BZC represents the standard deviation of the data fusion hidden danger assessment index;

[0032] Compare the standard deviation of the data fusion hidden danger assessment index with a preset standard deviation threshold, and give an early warning prompt to the existing virtual reality immersive teaching system, specifically as follows:

[0033] If the standard deviation of the data fusion hidden danger assessment index is greater than the standard deviation threshold, an early warning signal is triggered;

[0034] If the standard deviation of the data fusion hidden danger assessment index is less than or equal to the standard deviation threshold, there is no need to trigger an early warning signal.

[0035] In a preferred embodiment, a virtual reality-based immersive teaching system includes a weight adjustment delay module, an error data interference propagation module, a data conflict module, a hidden danger degree assessment module, and an instant early warning module;

[0036] The weight adjustment delay module is used to obtain the weight adjustment delay information in the multi-sensor data fusion process and obtain the weight adjustment delay coefficient according to the weight adjustment delay information;

[0037] The error data interference propagation module is used to obtain the error data interference propagation information in the multi-sensor data fusion process and obtain the error data interference propagation coefficient according to the error data interference propagation information;

[0038] A data conflict module, which is used to obtain the sensor data conflict information in the multi-sensor data fusion process and obtain the sensor data conflict coefficient according to the sensor data conflict information;

[0039] A hidden danger degree evaluation module, which is used to construct a data fusion hidden danger evaluation model according to the weight adjustment delay coefficient, error data interference propagation coefficient and sensor data conflict coefficient, output the data fusion hidden danger evaluation index, and evaluate the potential hidden danger degree of the multi-sensor data fusion process;

[0040] An immediate warning module, which is used to give a warning prompt to the existing virtual reality immersive teaching system according to the potential hidden danger degree of the multi-sensor data fusion process.

[0041] The technical effects and advantages of the present invention:

[0042] 1. By obtaining and calculating the weight adjustment delay coefficient, error data interference propagation coefficient and sensor data conflict coefficient, the present invention can quantify the hidden dangers existing in the data fusion process, and accordingly construct a data fusion hidden danger evaluation model to realize real-time monitoring and quantitative analysis of the fusion hidden dangers. The data fusion hidden danger evaluation index output by this model can intuitively reflect the severity of potential problems in the multi-sensor fusion process, and then conduct intelligent warning on the teaching system. When the fusion hidden danger index exceeds the preset threshold, the system can timely trigger a warning prompt to remind users or system managers to take corresponding adjustment measures to reduce the negative impact brought by error propagation, so as to effectively avoid the misalignment problem between the virtual scene and the actual actions of users, improve the accuracy and immersion of interaction, effectively improve the reliability of multi-sensor data fusion in the virtual reality immersive teaching system, and enhance the adaptive perception ability of the virtual reality immersive teaching system to potential fusion hidden dangers. Even if there are no obvious error hidden dangers in a short period of time, the possible system deviations can be predicted in advance through subsequent evaluations, so as to prevent the tracking errors from gradually accumulating after long-term operation, affecting the accuracy of spatial operations in the teaching process, making the virtual teaching environment more in line with the real interaction needs, and enhancing the operability and practicability of teaching content. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] For the convenience of those skilled in the art to understand, the present invention will be further described below with reference to the accompanying drawings;

[0044] Figure 1 It is the flowchart of the method of Embodiment 1 of the present invention;

[0045] Figure 2 It is the flowchart of the system of Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1: Figure 1 The immersive teaching method based on virtual reality of the present invention is given, including the following steps:

[0048] Step S1, obtain the weight adjustment delay information in the multi-sensor data fusion process, and obtain the weight adjustment delay coefficient according to the weight adjustment delay information;

[0049] Step S2, obtain the error data interference propagation information in the multi-sensor data fusion process, and obtain the error data interference propagation coefficient according to the error data interference propagation information;

[0050] Step S3, obtain the sensor data conflict information in the multi-sensor data fusion process, and obtain the sensor data conflict coefficient according to the sensor data conflict information;

[0051] Step S4, construct a data fusion hidden danger assessment model according to the weight adjustment delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, output the data fusion hidden danger assessment index, and evaluate the potential hidden danger degree of the multi-sensor data fusion process;

[0052] Step S5, give a warning prompt to the existing virtual reality immersive teaching system according to the potential hidden danger degree of the multi-sensor data fusion process;

[0053] Step S1, obtain the weight adjustment delay information in the multi-sensor data fusion process, and obtain the weight adjustment delay coefficient according to the weight adjustment delay information;

[0054] In the process of multi-sensor data fusion, the data quality of each sensor may change dynamically over time, so it is necessary to dynamically adjust its fusion weight. If there is a large delay in weight adjustment, it may cause the system to be unable to adapt to changes in sensor data in a short time, thereby affecting the fusion accuracy; the weight adjustment delay coefficient in the present invention is used to measure the response speed of weight adjustment in the fusion process of multi-sensor data, and is used to evaluate the lag degree of weight adjustment, so as to judge the real-time and stability of data fusion; if the weight adjustment delay coefficient is too large, it means that the more delayed the weight adjustment is, the more obvious the potential hidden dangers in the multi-sensor data fusion process are, and the system may not be able to quickly adapt to changes in sensor data; if the weight adjustment delay coefficient is smaller, it means that the weight adjustment is more timely, the probability of potential hidden dangers in the multi-sensor data fusion process is smaller, and the system can quickly adapt to changes in sensor data; based on the weight adjustment delay coefficient, the potential hidden dangers of the multi-sensor data fusion process can effectively improve the real-time and stability of data fusion, and ensure the precise interaction and immersive experience of the virtual reality immersive teaching system. By calculating the weight adjustment delay coefficient, the lag degree of weight adjustment of each sensor data in the fusion process can be quantified, and problems such as fusion error accumulation, data inconsistency and system response delay caused by slow or drastic weight updates can be identified; through this evaluation method, data fusion hidden dangers can be graded and warned, and the weight update rate can be adjusted in real time to reduce lag and volatility, improve the accuracy and robustness of data fusion, thereby optimizing the stability of the virtual reality immersive teaching system, enhancing the user's immersion and interactive experience, and ensuring the accurate presentation of teaching content and the improvement of learning effects;

[0055] Therefore, by obtaining the weight adjustment delay information of the multi-sensor data fusion process, the response of the weight adjustment in the multi-sensor data fusion process is analyzed, and the weight adjustment delay coefficient is obtained to measure the delay degree of the weight adjustment in the multi-sensor data fusion process;

[0056] The logic for obtaining the weight adjustment delay coefficient is as follows:

[0057] Get the weight value of a single sensor at two consecutive time steps t and t-Δt, and calculate the weight adjustment rate: Where ΔW represents the weight adjustment rate of the sensor, W(t) represents the weight value of the sensor at time step t, W(t-Δt) represents the weight value of the sensor at time step t-Δt, and Δt represents the time interval for weight update; calculate the average value of the weight adjustment rate: Where Wdelay represents the average value of the weight adjustment rate, ΔW iRepresents the weight adjustment rate calculated when the i-th sensor performs weight adjustment, where i = {1, 2,..., I} and I is a positive integer; compare the weight adjustment rate calculated when the sensor performs weight adjustment with the average value of the weight adjustment rate. If the weight adjustment rate calculated when the sensor performs weight adjustment is less than the average value of the weight adjustment rate, it indicates that the data weight response speed of this sensor is slow, and mark the weight adjustment rate calculated during this weight adjustment as the weight adjustment delay rate; calculate the weight adjustment delay coefficient, and the expression is as follows: where Qzt represents the weight adjustment delay coefficient, and ΔWy mn represents the weight adjustment rate at which the m-th sensor is marked as the weight adjustment delay rate for the n-th time, where n = {1, 2,..., N}, N is a positive integer, m = {1, 2,..., M}, and M is a positive integer;

[0058] Step S2, obtain the error data interference propagation information in the multi-sensor data fusion process, and obtain the error data interference propagation coefficient according to the error data interference propagation information;

[0059] During the multi-sensor data fusion process, errors may propagate between different sensor data. For example, IMU drift will affect the pose estimation of the SLAM system, and ultimately affect the synchronization and rendering accuracy of the entire VR system. If the error propagation is not detected and compensated, it will lead to the accumulation of fusion errors, which will further affect the VR user experience and even cause dizziness. In the present invention, the error data interference propagation coefficient is an index used to measure the degree of mutual transfer, superposition and interference of the error data generated by each sensor during the data fusion process. The larger the error data interference propagation coefficient, the more serious the propagation interference degree of the error data during the data fusion process, and the more obvious the potential hidden danger in the multi-sensor data fusion process. If the error data interference propagation coefficient is smaller, it indicates that the propagation interference degree of the error data during the data fusion process is lower, the data fusion process is more stable, and the probability of potential hidden dangers in the multi-sensor data fusion process is smaller. By obtaining the error data interference propagation information of the multi-sensor data fusion process, analyzing the propagation interference situation of errors between different sensors, and calculating the error data interference propagation coefficient, the influence range and degree of errors between sensors can be quantified, ensuring the stability and reliability of data fusion. The calculation of the error data interference propagation coefficient combines error residual analysis, error correlation calculation and error propagation influence matrix, which can accurately measure how errors are transmitted between different sensors and the degree of interference on the fusion result, enabling the system to identify the error propagation path, predict potential error accumulation risks, and timely adjust the fusion strategy, trigger the warning mechanism, dynamically adjust the sensor data fusion weight, and even adopt optimization strategies such as adaptive Kalman filtering and error compensation mechanism to effectively reduce the impact of error propagation, improve the accuracy and stability of data fusion, and ultimately enhance the reliability and user experience of the virtual reality immersive teaching system.

[0060] Therefore, by obtaining the error data interference propagation information of the multi-sensor data fusion process, analyzing the propagation interference situation of the error data between different sensors, and obtaining the error data interference propagation coefficient, the propagation interference degree of the error data between different sensors can be measured;

[0061] The acquisition logic of the error data interference propagation coefficient is as follows:

[0062] Obtain the measurement value Cl k (t) of the k-th sensor at time t, and calculate the error residual: En k (t) = |Cl k (t) - Ctrue k (t)|, where En k (t) represents the error residual of the k-th sensor at time t, and Ctrue k (t) represents the expected true value; calculate the error propagation correlation coefficient: where ρkh represents the error propagation correlation coefficient between sensors k and h, represents the mean error residual of sensor k, En h (t) represents the error residual of the h-th sensor at time t, represents the mean error residual of sensor h, T is a positive integer; construct the error propagation influence matrix MZ, whose matrix elements are: Among them mz kh is the element in the error propagation impact matrix, ∈ is a minimum positive value to avoid the denominator being zero; the error data interference propagation coefficient is calculated, and the expression is as follows: Where k = {1, 2, ..., K}, h = {1, 2, ..., K}, K is a positive integer and h ≠ k;

[0063] Step S3, obtaining sensor data conflict information in the multi-sensor data fusion process, and obtaining a sensor data conflict coefficient according to the sensor data conflict information;

[0064] When multiple sensors provide conflicting data, the fusion algorithm may have difficulty determining a credible data source, leading to incorrect decisions. For example: the IMU detects high-speed rotation, but the camera does not detect the corresponding motion features. The lidar ranging does not match the depth camera data. When sensor conflicts are not resolved, virtual objects in the VR system may drift, be misplaced, and even affect the interaction logic; the sensor data conflict coefficient in the present invention is used to measure the inconsistency between the measurement results of different sensor data at the same time or in the same space during the multi-sensor data fusion process, so as to identify data conflicts and evaluate their impact on fusion accuracy. This coefficient quantitatively evaluates the degree of sensor data conflict and assists in optimizing the data fusion strategy by calculating the degree of difference and deviation trend between the measurement values of each sensor, combining methods such as spatiotemporal alignment error analysis, data consistency test and outlier detection. If the sensor data conflict coefficient is larger, it means that the sensor data conflict is more serious during the data fusion process, and the potential hidden dangers of the multi-sensor data fusion process are more obvious. If the sensor data conflict coefficient is smaller, it means that the sensor data conflict is less during the data fusion process, that is, the more stable the data fusion process is, the smaller the probability of potential hidden dangers in the multi-sensor data fusion process is. By calculating the sensor data conflict coefficient, the contradictions between multi-sensor measurement data can be accurately identified, and their impact on the fusion results can be evaluated, so that the system can dynamically detect and warn of potential data conflict risks, reduce the interference of data conflicts on fusion results, improve the data credibility and stability of the virtual reality immersive teaching system, and ensure that users get a more accurate and smoother immersive experience.

[0065] Therefore, by obtaining the sensor data conflict information in the multi-sensor data fusion process, analyzing the data conflict situation of the sensors in the data fusion process, and obtaining the sensor data conflict coefficient, the data conflict degree of the sensors in the data fusion process can be measured;

[0066] The acquisition logic of the sensor data conflict coefficient is as follows:

[0067] Obtain the multi-dimensional conflict distance of the sensor measurement values: Where represents the multi-dimensional conflict distance of the sensor measurement values, represents the data vector of the measurement value of the x-th sensor, represents the data vector of the measurement value of the y-th sensor, represents the transpose of the vector, ∑ -1 represents the inverse matrix of the covariance matrix of the measurement data; Calculate the mutual information of the sensor data conflict: Where represents the mutual information of the sensor data conflict, p(a,b) represents and the joint probability distribution between, p(a) represents the marginal probability distribution of, p(b) represents the marginal probability distribution of, a is the value in the data vector and b is the value in the data vector ;

[0068] The mutual information of the sensor data conflict is used to measure the degree of mutual dependence between the data vectors of the measurement values of different sensors; The joint probability distribution p(a,b) is and the probability of simultaneously taking the values a and b; The marginal probability distribution p(a) is the probability of taking the value a; The marginal probability distribution p(b) is the probability of taking the value b;

[0069] Calculate the conflict influence coefficient: Where CT xy represents the conflict influence coefficient between the x-th sensor and the y-th sensor, which is used to measure the conflict degree between the x-th sensor and the y-th sensor;

[0070] Calculate the information entropy of each sensor data: Where Hs represents the information entropy of the sensor data, p(q) represents the probability that the measurement value of the sensor falls into the interval q, q = {1, 2,..., Q}, and Q is a positive integer;

[0071] Calculate the sensor data conflict coefficient, and the expression is as follows: Among them, CGC represents the sensor data conflict coefficient, and Hs x represents the information entropy of the x-th sensor, and Hs y represents the information entropy of the y-th sensor, where x = {1, 2,..., X}, y = {1, 2,..., X}, X is a positive integer and x ≠ y;

[0072] Step S4, construct a data fusion hidden danger assessment model based on the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, output a data fusion hidden danger assessment index, and evaluate the potential hidden danger degree of the multi-sensor data fusion process;

[0073] Construct a data fusion hidden danger assessment model based on the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, and output a data fusion hidden danger assessment index SJR. The formula on which the model is based is as follows In the formula, d1, d2, and d3 respectively represent the preset proportionality coefficients of the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, and d1, d2, and d3 are all greater than 0;

[0074] It should be noted that before constructing the data fusion hidden danger assessment model, it is necessary to ensure that the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient have all been normalized. Common normalization methods include Min-Max normalization, Z-Score standardization, etc.; d1, d2, and d3 are set according to the actual situation. For example, the expert weighting method is adopted, that is, relevant experts in the field are invited to determine the preset proportionality coefficients of each index through professional opinion surveys and comprehensive evaluations;

[0075] It can be seen from the above calculation expressions that the larger the weight-adjusted delay coefficient, the larger the error data interference propagation coefficient, and the larger the sensor data conflict coefficient, the larger the data fusion hidden danger assessment index, indicating that the hidden danger existing in the data fusion process is greater, and the reliability of the fusion result may not meet the requirements of high-precision scenarios, resulting in misjudgment or operation risks. On the contrary, the smaller the weight-adjusted delay coefficient, the smaller the error data interference propagation coefficient, and the smaller the sensor data conflict coefficient, the smaller the data fusion hidden danger assessment index, indicating that the hidden danger existing in the data fusion process is smaller and the result of data fusion is more reliable;

[0076] Compare the data fusion hidden danger assessment index with the preset data fusion hidden danger assessment index threshold to determine the potential hidden danger degree of the multi-sensor data fusion process, as follows:

[0077] If the data fusion hidden danger assessment index is greater than the data fusion hidden danger assessment index threshold, it means that the potential hidden danger degree of the multi-sensor data fusion process is extremely high, and a fusion hidden danger signal is generated;

[0078] If the data fusion hidden danger assessment index is less than or equal to the data fusion hidden danger assessment index threshold, it indicates that the potential hidden danger degree of the multi-sensor data fusion process is relatively low, and its hidden danger risk is within the controllable range, and there is no need to generate a fusion hidden danger signal;

[0079] Step S5, give a warning prompt to the existing virtual reality immersive teaching system according to the potential hidden danger degree of the multi-sensor data fusion process;

[0080] When there is a fusion hidden danger signal, obtain multiple data fusion hidden danger assessment indexes output by the data fusion hidden danger assessment model subsequently, establish a fusion hidden danger data set, and mark the fusion hidden danger data set as rhy = {SJR β}, where β = {1, 2, 3,..., γ}, and γ is a positive integer;

[0081] Calculate the average value of the data fusion hidden danger assessment indexes in the fusion hidden danger data set. The expression is as follows: Among them represents the average value of the data fusion hidden danger assessment indexes in the fusion hidden danger data set, and SJR β represents the β-th output data fusion hidden danger assessment index;

[0082] Calculate the standard deviation of the data fusion hidden danger assessment indexes in the fusion hidden danger data set. The expression is as follows: Among them, BZC represents the standard deviation of the data fusion hidden danger assessment index;

[0083] Compare the standard deviation of the data fusion hidden danger assessment index with the preset standard deviation threshold, and give a warning prompt to the existing virtual reality immersive teaching system, specifically as follows:

[0084] If the standard deviation of the data fusion hidden danger assessment index is greater than the standard deviation threshold, it indicates that the volatility of the data fusion hidden danger index is strong, the fusion risk is unstable, and a warning signal is triggered;

[0085] If the standard deviation of the data fusion hidden danger assessment index is less than or equal to the standard deviation threshold, it indicates that the data fusion hidden danger index is relatively stable, the fusion risk is relatively stable, and there is no need to trigger a warning signal;

[0086] By obtaining and calculating the weight adjustment delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, the present invention can quantify the potential hazards existing in the data fusion process, and accordingly construct a data fusion hazard assessment model to achieve real-time monitoring and quantitative analysis of fusion hazards. The data fusion hazard assessment index output by this model can intuitively reflect the severity of potential problems in the multi-sensor fusion process, and then conduct intelligent early warning for the teaching system. When the fusion hazard index exceeds the preset threshold, the system can timely trigger an early warning prompt to remind users or system administrators to take corresponding adjustment measures to reduce the negative impact brought by error propagation, thereby effectively avoiding the misalignment problem between the virtual scene and the actual actions of users, improving the accuracy and immersion of interaction, effectively enhancing the reliability of multi-sensor data fusion in the virtual reality immersive teaching system, and strengthening the adaptive perception ability of the virtual reality immersive teaching system to potential fusion hazards. Even for hazards without obvious errors in a short period of time, possible system deviations can be predicted in advance through subsequent evaluations, so as to prevent the gradual accumulation of tracking errors after long-term operation from affecting the accuracy of spatial operations in the teaching process, making the virtual teaching environment more in line with real interaction needs, and enhancing the operability and practicality of teaching content.

[0087] Embodiment 2: This embodiment is an introduction to an immersive teaching system based on virtual reality, as Figure 2 shown, which includes a weight adjustment delay module, an error data interference propagation module, a data conflict module, a hazard degree assessment module, and an immediate warning module;

[0088] The weight adjustment delay module is used to obtain the weight adjustment delay information in the multi-sensor data fusion process and obtain the weight adjustment delay coefficient according to the weight adjustment delay information;

[0089] The error data interference propagation module is used to obtain the error data interference propagation information in the multi-sensor data fusion process and obtain the error data interference propagation coefficient according to the error data interference propagation information;

[0090] The data conflict module is used to obtain the sensor data conflict information in the multi-sensor data fusion process and obtain the sensor data conflict coefficient according to the sensor data conflict information;

[0091] The hazard degree assessment module is used to construct a data fusion hazard assessment model according to the weight adjustment delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, output a data fusion hazard assessment index, and evaluate the potential hazard degree of the multi-sensor data fusion process;

[0092] The immediate warning module is used to give an early warning prompt to the existing virtual reality immersive teaching system according to the potential hazard degree of the multi-sensor data fusion process;

[0093] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula that is closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0094] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0095] It should be understood that in various embodiments of the present application, the order numbers of the above processes do not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0096] Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described system and method can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0097] In several embodiments provided by the present application, it should be understood that the disclosed system and method can be implemented in other ways.

[0098] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An immersive teaching method based on virtual reality, characterized in that: It includes the following steps: Step S1: Obtain the weight adjustment delay information in the multi-sensor data fusion process, and obtain the weight adjustment delay coefficient according to the weight adjustment delay information; Step S2: Obtain the error data interference propagation information in the multi-sensor data fusion process, and obtain the error data interference propagation coefficient according to the error data interference propagation information; Step S3: Obtain the sensor data conflict information in the multi-sensor data fusion process, and obtain the sensor data conflict coefficient according to the sensor data conflict information; Step S4: Construct a data fusion hidden danger assessment model based on the weight adjustment delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, output the data fusion hidden danger assessment index, and evaluate the potential hidden danger degree of the multi-sensor data fusion process; Step S5: Give a warning prompt to the existing virtual reality immersive teaching system according to the potential hidden danger degree of the multi-sensor data fusion process.

2. The immersive teaching method based on virtual reality according to claim 1, characterized in that: By obtaining the weight adjustment delay information in the multi-sensor data fusion process, analyze the response of the weight adjustment in the multi-sensor data fusion process, and obtain the weight adjustment delay coefficient to measure the delay degree of the weight adjustment in the multi-sensor data fusion process; The acquisition logic of the weight adjustment delay coefficient is as follows: Obtain the weight values of a single sensor at two consecutive time steps \(t\) and \(t - \Delta t\), and calculate the weight adjustment rate: where \(\Delta W\) represents the weight adjustment rate of the sensor, \(W(t)\) represents the weight value of the sensor at time step \(t\), \(W(t - \Delta t)\) represents the weight value of the sensor at time step \(t - \Delta t\), and \(\Delta t\) represents the time interval for weight update; calculate the average value of the weight adjustment rate: where \(W_{delay}\) represents the average value of the weight adjustment rate, \(\Delta W\) i represents the weight adjustment rate calculated when the \(i\)-th sensor makes a weight adjustment, \(i=\{1,2,\cdots,I\}\), and \(I\) is a positive integer; compare the weight adjustment rate calculated when the sensor makes a weight adjustment with the average value of the weight adjustment rate. If the weight adjustment rate calculated when the sensor makes a weight adjustment is less than the average value of the weight adjustment rate, then mark the weight adjustment rate calculated at this time as the weight adjustment delay rate; Calculate the weight adjustment delay coefficient, and the expression is as follows: where Qzt represents the weight adjustment delay coefficient, and ΔWy mn represents the weight adjustment rate at which the m-th sensor is marked as the weight adjustment delay rate for the n-th time, n = {1, 2,..., N}, N is a positive integer, m = {1, 2,..., M}, and M is a positive integer.

3. The immersive teaching method based on virtual reality according to claim 1, wherein: By obtaining the error data interference propagation information in the multi-sensor data fusion process, analyze the propagation interference of the error data between different sensors, and obtain the error data interference propagation coefficient to measure the propagation interference degree of the error data between different sensors; The acquisition logic of the error data interference propagation coefficient is as follows: Obtain the measurement value Cl of the k-th sensor at time t k (t), and calculate the error residual: En k (t) = |Cl k (t) - Ctrue k (t)|, where En k (t) represents the error residual of the k-th sensor at time t, and Ctrue k (t) represents the expected true value; calculate the error propagation correlation coefficient: where ρ kh represents the error propagation correlation coefficient between sensors k and h, represents the mean value of the error residuals of sensor k, En h (t) represents the error residual of the h-th sensor at time t, represents the mean value of the error residuals of sensor h, T is a positive integer; construct the error propagation influence matrix MZ, and its matrix elements are: where mz kh is an element in the error propagation influence matrix, and ∈ is a very small positive value used to avoid the denominator being zero; calculate the error data interference propagation coefficient, and the expression is as follows: where k = {1, 2,..., K}, h = {1, 2,..., K}, K is a positive integer and h ≠ k.

4. The immersive teaching method based on virtual reality according to claim 1, wherein: By obtaining the sensor data conflict information in the multi-sensor data fusion process, analyze the data conflict situation of the sensors in the data fusion process, and obtain the sensor data conflict coefficient to measure the data conflict degree of the sensors in the data fusion process; The acquisition logic of the sensor data conflict coefficient is as follows: Obtain the multi-dimensional conflict distance of sensor measurements: Where represents the multi-dimensional conflict distance of sensor measurements, represents the data vector of the x-th sensor measurement, represents the data vector of the y-th sensor measurement, represents the transpose of the vector, ∑ -1 represents the inverse matrix of the covariance matrix of the measurement data; Calculate the mutual information of sensor data conflicts: where represents the mutual information of sensor data conflicts, and p(a,b) represents and the joint probability distribution between, p(a) represents the marginal probability distribution of, p(b) represents the marginal probability distribution of, a takes values in the data vector and b takes values in the data vector ; Calculate the conflict influence coefficient: where CT xy represents the conflict influence coefficient between sensor x and sensor y; Calculate the information entropy of each sensor data: where Hs represents the information entropy of the sensor data, p(q) represents the probability that the measured value of the sensor falls into the interval q, q = {1, 2,..., Q}, and Q is a positive integer; Calculate the sensor data conflict coefficient, and the expression is as follows: where CGC represents the sensor data conflict coefficient, and Hs x represents the information entropy of the x-th sensor, and Hs y represents the information entropy of the y-th sensor, x = {1, 2,..., X}, y = {1, 2,..., X}, X is a positive integer and x ≠ y.

5. The immersive teaching method based on virtual reality according to claim 1, characterized in that: Construct a data fusion hidden danger assessment model by adjusting the delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient according to weights, and output the data fusion hidden danger assessment index SJR. The formula on which the model is based is as follows In the formula, d1, d2, and d3 respectively represent the preset proportionality coefficients of the weight adjustment delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, and d1, d2, and d3 are all greater than 0.

6. The immersive teaching method based on virtual reality according to claim 5, characterized in that: Compare the data fusion hidden danger assessment index with the preset data fusion hidden danger assessment index threshold to determine the potential hidden danger degree of the multi-sensor data fusion process, specifically as follows: If the data fusion hidden danger assessment index is greater than the data fusion hidden danger assessment index threshold, generate a fusion hidden danger signal; If the data fusion hidden danger assessment index is less than or equal to the data fusion hidden danger assessment index threshold, there is no need to generate a fusion hidden danger signal.

7. The immersive teaching method based on virtual reality according to claim 6, wherein: When fusing potential hazard signals, obtain multiple data fusion potential hazard assessment indices output by the subsequent data fusion potential hazard assessment model, establish a fused potential hazard data set, and mark the fused potential hazard data set as rhy = SJR β}, where β = {1, 2, 3, …, γ}, and γ is a positive integer; Calculate the average value of the data fusion risk assessment index in the data fusion risk data set. The expression is as follows: Where represents the average value of the data fusion risk assessment index in the data fusion risk data set, and SJR β represents the data fusion risk assessment index of the β-th output; Calculate the standard deviation of the data fusion risk assessment index in the data fusion risk data set, and the expression is as follows: Where BZC represents the standard deviation of the data fusion risk assessment index; Compare the standard deviation of the data fusion hidden danger assessment index with the preset standard deviation threshold to give a warning prompt to the existing virtual reality immersive teaching system, specifically as follows: If the standard deviation of the data fusion hidden danger assessment index is greater than the standard deviation threshold, trigger a warning signal; If the standard deviation of the data fusion hidden danger assessment index is less than or equal to the standard deviation threshold, there is no need to trigger a warning signal.

8. An immersive teaching system based on virtual reality for implementing the immersive teaching method based on virtual reality according to any one of claims 1-7, characterized in that: It includes a weight adjustment delay module, an error data interference propagation module, a data conflict module, a hidden danger degree assessment module, and an immediate warning module; The weight adjustment delay module is used to obtain the weight adjustment delay information in the multi-sensor data fusion process, and obtain the weight adjustment delay coefficient according to the weight adjustment delay information; An error data interference propagation module, which is used to obtain the error data interference propagation information in the multi-sensor data fusion process and obtain the error data interference propagation coefficient according to the error data interference propagation information; A data conflict module, which is used to obtain the sensor data conflict information in the multi-sensor data fusion process and obtain the sensor data conflict coefficient according to the sensor data conflict information; A potential hazard degree evaluation module, which is used to construct a data fusion potential hazard evaluation model based on the weight-adjusted delay coefficient, error data interference propagation coefficient, and sensor data conflict coefficient, output a data fusion potential hazard evaluation index, and evaluate the potential hazard degree of the multi-sensor data fusion process; An immediate warning module, which is used to give a warning prompt to the existing virtual reality immersive teaching system according to the potential hazard degree of the multi-sensor data fusion process.

Images