Augmented reality-based education training method and system

Abstract

The present application relates to the technical field of power training, in particular to an education training method and system based on augmented reality, the method comprising: constructing a virtual generator set, students learning power generation technology through augmented reality equipment, and collecting learning data of the students; presenting operation parameters in real time, and dynamically adjusting display content according to the gaze points and interactive behaviors of the students; designing interactive operation exercises to learn to operate the generator set in a virtual environment; constructing a machine learning algorithm to comprehensively analyze operation scores, gaze point distribution and interactive behavior data of the students, and establishing a personalized ability model and an operation risk prediction model; based on the personalized ability model and the predicted operation risk probability of the students, the system adaptively adjusts the exercise difficulty and the prompt strategy to realize personalized learning arrangement. The present application realizes intelligent and personalized training methods for the power generation industry, and improves the safety, pertinence and effectiveness of the generator set operation training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power training, in particular to an education training method and system based on augmented reality. BACKGROUND

[0002] Traditional generator set operator training mainly adopts a combination of theoretical teaching and on-site operation, which has some limitations and deficiencies. First, theoretical teaching is difficult for students to deeply understand the complex structure and working principle of the generator set, and it is difficult for students to establish a connection between abstract theoretical knowledge and actual equipment. Second, there are safety risks and cost pressures in real generator set operation training, and students are difficult to repeatedly practice operation procedures and cannot freely explore the response characteristics of the equipment. In addition, the traditional training method lacks pertinence and interactivity, and cannot provide differentiated guidance according to the individual characteristics and learning progress of students, and students are prone to boredom and fatigue, which affects the training effect.

[0003] The traditional generator set operation training method has limitations such as disconnection between theory and practice, high safety risk, and poor pertinence. Although existing technologies have attempted to introduce emerging technologies such as VR and AR to assist training, existing virtual training systems still need to be strengthened in terms of intelligent data analysis and personalized learning support.

[0004] Therefore, an innovative training method is needed to fully leverage the advantages of VR / AR technology, through intelligent learning data collection and analysis, to build personalized capability models and risk prediction models, thereby achieving targeted, flexible and safe generator set operation training, and continuously optimizing the training effect and student satisfaction.

[0005] In view of this, the present application proposes an education training system and method based on augmented reality. SUMMARY

[0006] To achieve the above-mentioned purpose, the present application provides an education training method and system based on augmented reality, and the specific technical solutions are as follows:

[0007] In a first aspect, the present application provides an education training method based on augmented reality, comprising:

[0008] A virtual generator set is constructed using augmented reality technology, and students learn power generation technology in the virtual generator set through augmented reality technology, and learning data of the students is collected;

[0009] The running parameters of the generator set are presented in real time in the virtual generator set, and the parameter type and detail level of the display are dynamically adjusted according to the gaze point and interaction behavior of the students;

[0010] Design an augmented reality interactive operation exercise for a virtual generator set, where trainees simulate controlling the generator set to start and stop and adjust parameters in a virtual environment;

[0011] Build machine learning algorithms to comprehensively analyze students' operational scores, gaze distribution, and interactive behavior data, establish personalized ability models, and predict their operational risk probability.

[0012] Based on the learner's personalized ability model and the predicted probability of operational risks, the difficulty of practice and prompting strategies in the virtual generator set are adaptively adjusted to provide a personalized learning arrangement for the learner.

[0013] Preferably, the augmented reality-based education and training method uses 3D modeling software to establish a 3D model of the generator set according to the actual structure and size of the generator set;

[0014] The established 3D model of the generator set is imported into the modeling engine, and augmented reality technology is applied to realize the overlay display and interaction of the virtual generator set in the real environment;

[0015] Interactive trigger points are set in the virtual generator set. When a trainee uses an AR device to point at a selected component, the system displays the component's name, function, and technical parameters. Trainees can interact with the virtual component through gestures or voice commands. These interactions include rotating, disassembling, and zooming in to observe the internal structure.

[0016] The system collects learning data from students, including learning duration, interactive behaviors, and gaze distribution; the learning duration is calculated by recording the start and end times of students' use of AR devices.

[0017] Interactive behavior data includes the types and frequency of interactions triggered by learners; gaze distribution is achieved by capturing the learner's gaze direction using the AR device's front-facing camera and mapping it to the three-dimensional coordinate system of the virtual generator; the gaze direction is then converted into three-dimensional coordinates. ; ; ;in, , , These represent the x, y, and z coordinates of the point where the line of sight falls in three-dimensional space, respectively. Indicates the length of the line of sight; Indicates the angle between the line of sight and the y-axis; This represents the angle between the projection of the line of sight onto the xz plane and the z-axis.

[0018] Preferably, in the augmented reality-based education and training method, in the three-dimensional model of the virtual generator set, a corresponding sensor data interface is set for each component of the generator set, and the virtual reality device obtains virtual generator set data in real time through the sensor data interface;

[0019] The augmented reality device dynamically adjusts the type and level of detail of the displayed parameters based on the learner's gaze point and interaction behavior. When the learner's gaze is focused on a single component, the parameters of that component are automatically displayed. When the learner moves away or their gaze is away from the component, the parameter display is hidden or simplified.

[0020] Preferably, the augmented reality-based education and training method determines the gaze point of the learner's gaze by calculating the angle between the gaze vector and the surface normal vector of the component. Let the gaze vector be and the surface normal vector of the component be then the angle The calculation formula is: ; where represents the dot product of two vectors, and represent the modulus of two vectors respectively; when is greater than the gaze threshold , it is determined that the learner's gaze is focused on the component.

[0021] The learner's interaction behavior is determined by analyzing the learner's gestures or voice commands. When the learner makes a specific gesture or speaks a specific voice command, the augmented reality device adjusts the type and level of detail of the parameter display accordingly.

[0022] Preferably, the augmented reality-based education and training method further includes the core definition and dimension design steps of constructing a component-knowledge-operation three-dimensional topological matrix (C-K-OTM) before presenting the operating parameters of the virtual generator set in real time. Specifically:

[0023] Set the matrix to be a three-dimensional structure of MxLxK, where M is the total number of core components of the virtual generator set, L is the total number of knowledge dimensions, and K is the total number of operation types.

[0024] Define the nodes of each dimension of the matrix: the first dimension is the core component node, including the AR virtual set interactive components of the boiler, steam turbine, generator, and desuperheating system; the second dimension is the knowledge dimension node, including component structure, operating parameter meaning, fault mechanism, and safety specification; the third dimension is the operation type node, including start-stop operation, parameter adjustment, fault handling, and daily inspection interactive practice scenarios.

[0025] Clearly define the physical meaning of the matrix elements: C-K-OTM(i,j,k) represents the correlation strength coefficient of component i-knowledge j-operation k, with a value range of [0,1]. The closer the coefficient is to 1, the stronger the necessity for the learner to prioritize mastering knowledge j to support operation k when learning component i.

[0026] Preferably, the augmented reality-based education and training method further comprises, after the step of dynamically modifying the initial correlation strength, a step of dynamically updating the component-knowledge-operation three-dimensional topological matrix, specifically:

[0027] monitoring and judging whether the score of a certain component-operation combination of the trainee continuously exceeds a preset threshold value for a plurality of times, or whether the risk probability of a certain operation predicted by the operation risk prediction model exceeds a preset risk threshold value;

[0028] If yes, update logic is executed: if the operation score meets the standard, the correlation strength of the component-basic knowledge-operation is reduced, the correlation strength of the component-advanced knowledge-operation is increased, and the trainee is guided to learn in depth; if the risk probability exceeds the standard, the correlation strength of the component-risk related knowledge-operation is increased, and the priority of the risk related knowledge display is strengthened; the knowledge importance weight annotated by the expert is retained during the updating process.

[0029] Preferably, the augmented reality-based education and training method, the step of presenting the operating parameters of the generator set in the virtual generator set in real time, and dynamically adjusting the display content according to the gaze point and interactive behavior of the trainee, is specifically optimized in combination with the component-knowledge-operation three-dimensional topological matrix, and the step is:

[0030] The AR device judges the trainee's gaze on the component i by the angle between the line of sight vector and the surface normal vector of the component, extracts all knowledge-operation correlation coefficients C-K-OTM(i,j,k) corresponding to the component i in the matrix;

[0031] The extracted correlation coefficients are sorted in descending order, and the trainee is preferentially shown the knowledge j and operation k with high priority;

[0032] When the trainee triggers an interactive behavior through gestures or voice, the correlation coefficient of the corresponding operation k in the matrix is updated in real time, and the subsequent knowledge pushing content is dynamically adjusted.

[0033] Preferably, the augmented reality-based education and training method, the step of designing the augmented reality interactive operation practice of the virtual generator set, specifically optimizes the practice task generation in combination with the component-knowledge-operation three-dimensional topological matrix, and the step is:

[0034] Before generating the practice task, the required knowledge reserve threshold for completing the task is determined according to the component-knowledge correlation strength of the corresponding operation type k in the matrix;

[0035] Judge whether the current knowledge mastery of the trainee meets the knowledge reserve threshold, and only when the threshold is met, the corresponding interactive operation practice task is unlocked.

[0036] Preferably, the step of designing the augmented reality interactive operation practice of the virtual power generation set in the augmented reality-based education training method further comprises the steps of optimizing practice process support and scoring in combination with the component-knowledge-operation three-dimensional topological matrix, specifically:

[0037] During the practice process, the length of time that the student gazes at a certain component is monitored in real time, and if the length of time exceeds a preset determination threshold, the highly relevant knowledge content corresponding to the component and the current operation is extracted from the matrix;

[0038] When the student completes the practice and the score is evaluated, the matching degree of the knowledge j referred to by the student during the operation process and the highly relevant knowledge corresponding to the operation in the matrix is judged;

[0039] The final score is adjusted according to the matching degree: if the matching degree is high, the score is appropriately increased, and if the matching degree is low, the score is appropriately decreased.

[0040] In a second aspect, the technical scheme of the present application provides an augmented reality-based education training system for implementing the augmented reality-based education training method of any one of the first aspect, comprising a data acquisition module, a virtual scene reality module, an interactive operation module, a learning detection module and a learning arrangement module;

[0041] The data acquisition module uses augmented reality technology to construct a virtual power generation set, and the student learns power generation technology in the virtual power generation set through augmented reality technology, and acquires learning data of the student;

[0042] The virtual scene reality module presents the operating parameters of the power generation set in real time in the virtual power generation set, and dynamically adjusts the parameter types and the detailed degree of display according to the gaze point and the interactive behavior of the student;

[0043] The interactive operation module designs an augmented reality interactive operation practice of the virtual power generation set, and the student simulates the control of the start and stop of the power generation set and the adjustment of the parameters in the virtual environment;

[0044] The learning detection module constructs a machine learning algorithm, comprehensively analyzes the data of the operation score, the gaze point distribution and the interactive behavior of the student, establishes a personalized ability model, and predicts the operation risk probability;

[0045] The learning arrangement module adaptively adjusts the practice difficulty and the prompt strategy in the virtual power generation set based on the personalized ability model and the predicted operation risk probability of the student, and performs personalized learning arrangement on the student.

[0046] The present application has the following beneficial effects: the present application uses augmented reality technology to construct a virtual power generation set, provides an immersive and interactive learning environment for the student, improves the learning interest and initiative of the student, and provides a basis for subsequent personalized training by acquiring learning data.

[0047] The application presents operation parameters in real time in a virtual generator set, and dynamically adjusts display content according to the gaze point and interaction behavior of the trainee, realizes the pertinence and adaptability of information display, and helps the trainee quickly understand and master key parameters.

[0048] The application designs an augmented reality interactive operation practice, so that the trainee simulates a real operation process in a virtual environment, strengthens the practical ability and operation experience, and improves the proficiency and confidence of the trainee in controlling and adjusting the generator set.

[0049] The application constructs a machine learning algorithm to comprehensively analyze the performance of the trainee, establishes a personalized ability model and an operation risk prediction model, realizes accurate evaluation of the ability level and operation risk of the trainee, and provides data support for formulating a targeted training scheme.

[0050] Based on the personalized ability model and the operation risk prediction model of the trainee, the application adaptively adjusts the practice difficulty and the prompt strategy, makes personalized learning arrangement for the trainee, improves the training efficiency and the satisfaction of the trainee, and prevents operation risks. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A flowchart of an augmented reality-based education and training method is provided for the application;

[0052] Figure 2 A structure diagram of an augmented reality-based education and training system is provided for the application. DETAILED DESCRIPTION

[0053] In order to better understand the application, various aspects of the application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the application, and do not limit the scope of the application in any way. Throughout the specification, the same reference numbers refer to the same elements. Expressions and / or any and all combinations of the associated listed items are included.

[0054] In the drawings, the size, dimension and shape of the elements have been slightly adjusted for ease of illustration. The drawings are merely exemplary and are not drawn strictly to scale. As used in this document, the terms substantially, approximately, and like terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent deviations in measuring or calculating values that would be recognized by those of ordinary skill in the art. In addition, in the application, the order of the steps described does not necessarily represent the order in which the processes appear in actual operation, unless otherwise explicitly limited or derivable from the context.

[0055] It should also be understood that all references to specifics in this description are meant to be open-ended language that is to be construed in the context where this application is made. That is, each and every limit described herein are deemed a description of two exact dimensions rather than intended to be limiting. It should also be understood that the terms such as comprising, including, having and / or containing are open-ended language that is to be construed to cover both specific and alternative implementations. Furthermore, the terms such as including and / or comprising, when used in this description, specify the presence of stated features, elements, and / or components, but do not preclude the presence or addition of one or more other features, elements, components, and / or groups thereof. Also, when such as at least one of is present in a list of entities, this is to be interpreted to mean that one or more of the listed entities are present, but not excluding others.

[0056] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an overly literal or overly formal sense unless expressly so defined herein.

[0057] It should be noted that the embodiments and features of the present application can be combined with each other, if not in conflict. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0058] Example 1

[0059] Reference Figure 1 For a first embodiment of the present application, an augmented reality-based education and training method is provided, comprising:

[0060] S1: A virtual power generating set is constructed using augmented reality technology. Learners learn power generation technology in the virtual power generating set through augmented reality technology, and learning data of the learners is collected.

[0061] A three-dimensional model of the power generating set is established according to the structure and size of the actual power generating set using three-dimensional modeling software; the three-dimensional model of the power generating set includes a generator, a steam turbine, a boiler, and a cooling system component.

[0062] The established three-dimensional model of the power generating set is imported into a Unity game engine, and an augmented reality SDK such as ARCore, ARKit, or Vuforia is applied to realize superimposed display and interaction of the virtual power generating set in a real environment.

[0063] Interaction trigger points are set in the virtual power generating set. When a learner uses an AR device to aim at a selected component, the system displays the name, function, and technical parameter information of the component; the learner interacts with the virtual component through gestures or voice commands, and the interaction includes rotation, disassembly, and zoomed-in observation of the internal structure.

[0064] The learning data of the students is collected, including the learning duration, interaction behavior and gaze point distribution; the learning duration is calculated by recording the start and end time of the students using the AR device.

[0065] The interaction behavior data includes the type and frequency of the interaction triggered by the students (such as rotation, disassembly, zoom, etc.); the gaze point distribution is captured by the front camera of the AR device to determine the line of sight direction of the students, and is mapped into the three-dimensional coordinate system of the virtual generator set; the line of sight direction is converted into three-dimensional coordinates using the following formula: ; ; ; wherein, , , respectively represent the x, y, z coordinates of the line of sight landing point in the three-dimensional space; represents the line of sight length; represents the angle between the line of sight and the y-axis; represents the angle between the projection of the line of sight on the xz plane and the z-axis.

[0066] S2: Real-time presentation of the operating parameters of the generator set in the virtual generator set, including the temperature, pressure and speed of the generator set, and dynamic adjustment of the type and detail level of the displayed parameters according to the gaze point and interaction behavior of the students.

[0067] In the three-dimensional model of the virtual generator set, a corresponding sensor data interface is set for each component of the generator set, which corresponds to the temperature, pressure, speed and other sensors in the actual generator set, and can receive and update data in real time; the virtual reality device obtains the virtual generator set data in real time through the sensor data interface.

[0068] The augmented reality device dynamically adjusts the type and detail level of the displayed parameters according to the gaze point and interaction behavior of the students; when the student's line of sight focuses on a single component, the parameters of the component are automatically displayed; when the student moves away or the line of sight leaves the component, the parameter display is hidden or simplified to avoid interfering with the student's view.

[0069] The judgment of the gaze point of the student's line of sight is realized by calculating the angle between the line of sight vector and the surface normal vector of the component; let the line of sight vector be , and the normal vector of a point on the surface of the component be , then the angle can be calculated as follows: ; wherein, represents the dot product of the two vectors, and respectively represent the modulus of the two vectors; when is greater than the line of sight threshold , it is judged that the gaze point of the student's line of sight focuses on the component.

[0070] S3: Design an augmented reality interactive operation practice for virtual power generating set, where the trainee simulates the control of starting and stopping the power generating set and adjusting parameters in a virtual environment.

[0071] The judgment of the trainee's interaction behavior is achieved by analyzing the trainee's gestures or voice commands. When the trainee makes a specific gesture (such as clicking, double-clicking, zooming, etc.) or speaks a specific voice command (such as displaying temperature, hiding pressure, etc.), the augmented reality device adjusts the type and level of detail of the parameter display accordingly.

[0072] To avoid the parameter display being too complex, the complexity of parameter display can be dynamically adjusted according to the trainee's learning progress and cognitive level. Different display modes such as beginner, intermediate, and advanced can be set to correspond to different parameter types and levels of detail. The trainee defaults to the beginner mode when first using the system, which only displays the most basic parameters. As the trainee's learning progresses, the system can automatically switch to higher display modes to display more parameter details.

[0073] According to the actual operation process and control logic of the real power generating set, a virtual power generating set interactive operation practice is designed. The interactive operation practice includes scenarios such as starting, stopping, load adjustment, and fault handling of the power generating set. It transitions from simple tasks to complex tasks to adapt to the learning needs of trainees of different levels.

[0074] In the three-dimensional model of the virtual power generating set, an interaction script is added for each controllable control element (such as buttons, switches, knobs, etc.). The interaction mode (such as clicking, dragging, rotating, etc.), trigger conditions (such as prerequisites, mutual exclusion conditions, etc.), and response actions (such as starting a specific device and adjusting related parameters, etc.) of the control element are defined. Through the user interface and interaction prompts, the trainee is guided to complete the operation practice. The interface should include clear operation instructions, real-time feedback, and scoring mechanisms to help the trainee understand the operation steps and assess their performance.

[0075] To enhance the realism and challenge of the operation practice, random faults and abnormal conditions are added to the virtual power generating set. The frequency of fault occurrence is controlled by a Poisson distribution probability model: where represents the average number of faults or abnormalities per unit time, represents the actual number of faults or abnormalities; by adjusting the value of , the frequency of fault or abnormal conditions is controlled, and the difficulty of the practice is adjusted.

[0076] After the trainee completes the operation practice, the trainee's operation performance is evaluated, and a quantitative score is given. The score is based on the trainee's operation accuracy, completion time, and error count indicators, and is calculated by a weighted average. The comprehensive score is denoted as:

[0077] ;

[0078] wherein, is an accuracy weight, is a completion time weight, is an error times weight, denotes an operation accuracy rate, denotes an actual completion time, denotes a maximum allowed time, denotes an error operation times, denotes a maximum allowed error times.

[0079] According to the performance of the trainee in the operation practice and the evaluation feedback of the system, the difficulty and the focus of the subsequent practice are dynamically adjusted.

[0080] S4: Construct a machine learning algorithm, comprehensively analyze the operation score, gaze point distribution, and interactive behavior data of the trainee, establish a personalized ability model, and predict the operation risk probability.

[0081] Collect multi-dimensional data generated by the trainee in the virtual generator set operation practice, including: an operation score sequence: wherein, denotes the operation score of the trainee at the th time step, and the value range is [0, 100]; a gaze point distribution sequence: wherein, denotes the gaze point coordinates and the gaze duration of the trainee at the th time step; an interactive behavior sequence: wherein, is a one-hot vector, which denotes the interactive event type triggered by the trainee at the th time step, , is the data volume.

[0082] Preprocess and feature engineer the collected multi-dimensional data, extract effective feature sequences, normalize the operation score sequence , and obtain wherein, ; perform clustering analysis on the gaze point distribution sequence , and obtain a clustering center sequence wherein, denotes the coordinates of the th clustering center; calculate the probability of the gaze point at each time step belonging to each clustering center, and obtain a probability sequence ; perform clustering analysis on the interactive behavior sequence The dimensionality reduction processing is performed to map the one-hot vector into a low-dimensional dense vector to obtain a dimensionality reduced sequence wherein is the interaction behavior dimensionality reduced vector of the th time step.

[0083] The personalized ability model is constructed, an RNN model is adopted, and the operation score, the gaze point distribution and the interaction behavior of the trainee are comprehensively considered; the input of the model is a feature sequence wherein denotes the feature vector of the th time step, denotes the operation score of the trainee at the th time step; denotes the gaze point distribution of the trainee at the th time step; denotes the interaction behavior of the trainee at the th time step; the output of the RNN model is an ability index sequence of the trainee , denotes the ability index of the trainee at the th time step.

[0084] The RNN model is represented as: , ; wherein, denote the weight matrixes of the input to the hidden layer, the hidden layer to the hidden layer and the hidden layer to the output layer respectively, and denote the bias terms of the hidden layer and the output layer respectively.

[0085] By training the RNN model, the RNN model predicts the ability index of the current time step according to the historical performance of the trainee.

[0086] On the basis of the personalized ability model, the operation risk prediction model is constructed; the output of the RNN model is taken as the input of the logistic regression model to predict the operation risk probability of the trainee at each time step:

[0087] ;

[0088] wherein, denotes whether the operation of the trainee at the th time step appears risk, and are the parameters of the logistic regression model, is a sigmoid function, is an exponential function with e as the base; by the maximum likelihood estimation method, the model parameters are estimated to obtain the operation risk prediction model.

[0089] The established personalized ability model and operation risk prediction model are used to perform real-time evaluation and prediction on the ability level and operation risk of the trainee.

[0090] S5: Based on the personalized ability model of the trainee and the predicted operation risk probability, the difficulty of practice and the prompt strategy in the virtual generator set are adaptively adjusted to arrange personalized learning for the trainee.

[0091] According to the performance of the trainee in the virtual generator set operation practice, the characteristic sequence of the trainee is dynamically updated and input into the RNN model and the logistic regression model to obtain real-time ability indicators and operation risk probabilities, and the subsequent learning of the trainee is arranged based on these indicators.

[0092] For example, during the interaction between the trainee and the virtual generator set, the gaze points and interaction behaviors of the trainee are continuously recorded and used as inputs for the ability evaluation and learning recommendation of the trainee; if the trainee frequently checks the temperature parameters of a certain component and attempts to adjust the related settings, it can be inferred that the trainee pays more attention to temperature control, and the subsequent learning content and practice difficulty are adjusted accordingly.

[0093] The embodiment aims to solve the problem of lack of structured association between AR component interaction and knowledge and operation display in the augmented reality-based generator set education and training scene. By constructing a component-knowledge-operation three-dimensional topological matrix (C-K-OTM), the AR device is upgraded from passive parameter display to active association push, and structured difficulty adjustment basis is provided for interactive practice. The specific implementation process, principle derivation and effect description are as follows:

[0094] Firstly, the core definition and dimension design of the component-knowledge-operation three-dimensional topological matrix (C-K-OTM) are carried out. Combined with the structural characteristics and training needs of the generator set, the matrix is set as MxLxK three-dimensional structure, where M corresponds to the total number of core components of the virtual generator set (such as boiler, steam turbine, generator, desuperheating system, etc. Interactive components), L corresponds to the total number of knowledge dimensions (such as component structure, operation parameter meaning, fault mechanism, safety specification, etc. Technical knowledge needs to be transmitted), and K corresponds to the total number of operation types (such as start-stop operation, parameter adjustment, fault handling, daily inspection, etc. Operation tasks in training scenarios). The first dimension (row, C class) is defined as the core component node, each node corresponds to the interactive entity component in the AR virtual generator set, and the matrix is ensured to correspond one by one with the physical structure of the virtual generator set; the second dimension (column, K class) is defined as the knowledge dimension node, covering the whole chain of knowledge from basic structure cognition to high-order fault mechanism, meeting the knowledge needs of different training stages; the third dimension (depth, O class) is defined as the operation type node, which is sorted from low to high according to the training difficulty (such as start-stop operation first, then parameter adjustment, and finally fault handling), which conforms to the skill advancement law of students. The matrix element C-K-OTM(i,j,k) represents the association strength coefficient of component i-knowledge j-operation k, the value range is [0,1], the closer the coefficient is to 1, the stronger the necessity for students to learn component i to master knowledge j to support operation k, for example, when learning the start-stop operation of the steam turbine, the coefficient corresponding to the structure of the steam turbine and the meaning of the start-stop parameter needs to be close to 1, because these two kinds of knowledge are the core support for correct execution of start-stop operation. The core requirement of generator set training is the closed loop of component cognition-knowledge understanding-operation practice, and the three-dimensional matrix integrates fragmented knowledge and operation into a structured system by quantifying the association strength of the three.

[0095] After completing the dimension definition of the matrix, the initial association strength of the matrix elements is calculated. First, two types of basic data are collected: one is the importance weight of knowledge dimension marked by power industry experts (such as the weight of safety specification is higher than that of ordinary parameter meaning, because safety specification directly affects the operation risk prevention and control); the second is the training data of historical students, from which (the number of times that the historical students successfully perform operation k after learning component i and accessing knowledge j). The initial association strength is calculated by the formula of frequency statistics + weight correction, which is:

[0096] ;

[0097] Among them, reflects the effectiveness of knowledge j supporting operation k in historical practice (the more the number of successful times, the more critical the support of knowledge j to operation k), The importance difference of knowledge itself is corrected, such as safety knowledge is given high weight even if it supports a small number of successful times; the denominator is the weighted total frequency of all knowledge-operation combinations of component i, which is normalized to make the initial coefficient fall within the [0, 1] interval, ensuring that the association strength of different component-knowledge-operation combinations is comparable. For example, for the parameter adjustment operation of the temperature reduction system, if the number of successful adjustments after accessing the meaning of the operating parameter is large in the historical data, and the weight of the meaning of the operating parameter is high, the initial coefficient of this combination will be significantly higher than that of the combination of successfully adjusting after accessing the component structure, accurately quantifying the priority of knowledge support for operation.

[0098] After the initial association strength is calculated, it needs to be dynamically corrected in combination with the ability of the students to adapt to the cognitive needs of students with different ability levels. First, based on the ability index output by the personalized ability model , the student ability adaptation coefficient is calculated, and the formula is:

[0099] ;

[0100] Where max(y) is the maximum value of all student ability indexes, ensuring that the value of falls within the range [0, 1]. The stronger the student's ability (such as high operation accuracy and fast fault identification), the closer to max(y), the closer to 1; otherwise, closer to 0. Then the initial association strength is corrected by , and the formula is:

[0101] ;

[0102] According to the cognitive learning law, students with weaker ability ( small) need to master basic knowledge to perform simple operations (such as using component structure knowledge to support start-stop operations), so the initial association strength of basic knowledge-simple operation combinations needs to be kept high, and the correction amplitude is small; students with stronger ability ( large) have a higher demand for depth of knowledge (such as needing to use fault mechanism knowledge to support fault handling operations), so the association strength of complex knowledge-high order operation combinations needs to be improved through a correction term of 0.5x to guide students to transition to deep learning. For example, when a student with strong ability learns turbine fault handling, the corrected coefficient of the fault mechanism-fault handling combination will be significantly higher than the initial value, while the coefficient of the component structure-fault handling combination will be relatively reduced, ensuring that the knowledge push matches the student's ability.

[0103] To make the matrix adapt to the training progress and the risk prevention needs, a dynamic updating mechanism is established. First, two updating trigger conditions are set: one is that the operation score of a student for a certain component-operation combination exceeds the preset threshold for multiple times in a row (such as the accuracy rate of the start-stop operation meets the standard for multiple times in a row), indicating that the student has mastered the basic knowledge corresponding to the operation and needs to advance learning; the second is that the operation risk prediction model prompts that the risk probability of a certain operation exceeds the preset risk threshold (such as the risk probability of parameter adjustment operation is high), indicating that the student's knowledge of risk related to the operation is insufficient and needs to be strengthened. When either condition is met, the matrix is updated: if the score meets the standard, the association strength of the component-basic knowledge-operation combination is reduced (such as the coefficient of the steam turbine structure-start-stop operation is reduced), and the association strength of the component-advanced knowledge-operation combination is increased (such as the coefficient of the steam turbine fault mechanism-parameter adjustment is increased), guiding the student to transition from basic operation to complex operation; if the risk exceeds the standard, the association strength of the component-risk related knowledge-operation combination is increased (such as when the risk of parameter adjustment is high, the coefficient of the desuperheating system-operation parameter meaning-parameter adjustment is increased), and the priority of the display of risk related knowledge is strengthened. During the updating process, the importance weight of the knowledge annotated by experts is always retained , only the corresponding frequency statistics part is adjusted , where is a fixed weight determined based on the specifications of the power industry (such as the importance of safety specifications does not change with the training progress), and adjusting may cause the association logic to deviate from the industry training standards, so only the frequency statistics is adjusted to adapt to the real-time learning state of the student.

[0104] Integrating C-K-OTM into the parameter and knowledge display of AR devices optimizes the traditional single logic of focusing on components to display parameters. When the AR device determines that the student is focusing on a certain component i by the angle between the line of sight vector and the surface normal vector of the component (the specific judgment method is: calculate the angle ψ between the line of sight vector of the student and the normal vector of the surface point of the component, calculate cosψ by the formula , where represents the dot product of two vectors, and represent the modulus of two vectors, and if cosψ exceeds the line of sight threshold If the gaze point of the student's line of sight is determined to focus on component i, the system automatically extracts all knowledge-operation correlation coefficients C-K-OTM(i,j,k) corresponding to component i in the matrix, and sorts them from high to low, and preferentially displays the top 3 knowledge j and operation k instructions to the student. For example, when the student gazes at the steam turbine, if the coefficients of start-stop parameter meaning-start-stop operation turbine structure-start-stop operation failure mechanism-failure handling are the highest, the AR device first displays the start-stop parameter threshold turbine internal structure disassembly diagram failure handling risk point prompt, rather than randomly pushing all parameters. When the student triggers an interactive behavior (such as gesture adjustment of the desuperheating system parameters) through gestures or voice, the system updates the correlation strength of the corresponding operation k in the matrix in real time (such as increasing the coefficient of operating parameter meaning-parameter adjustment), and adjusts the subsequent knowledge push according to the updated coefficient, for example, when executing parameter adjustment, automatically increasing the push frequency of real-time interpretation of operating parameters, ensuring that the knowledge push is synchronized with the current operation demand. Traditional AR display only relies on the single correlation between components and parameters, resulting in fragmented knowledge, and students cannot understand the correlation logic between knowledge and operation; the matrix covers the full link between knowledge support and operation demand through three-dimensional correlation, enabling students to understand why they learn the knowledge and how to use the knowledge to operate, and improving the conversion efficiency of knowledge to operation ability.

[0105] In the interactive operation practice session, the C-K-OTM optimization practice task generation, process support and scoring logic are combined. When generating the practice task, according to the component-knowledge association strength of the corresponding operation type k in the matrix, the threshold of the required knowledge reserve of the task is determined, for example, when generating a fault handling practice task, the average coefficient of the component i corresponding knowledge-fault handling combination (C-K-OTM(i, j, 3)) needs to be calculated, only when the average coefficient exceeds the preset threshold (indicating that the student has mastered enough fault-related knowledge), the task is unlocked, avoiding the student to operate blindly in the case of insufficient knowledge reserve. During the practice, the length of time that the student gazes at a certain component is monitored in real time, if the length of time exceeds the preset determination threshold (determined as operation stall, indicating that the student may be unable to continue operation due to insufficient knowledge), the system extracts the high correlation knowledge content of the component and the current operation from the matrix and pushes it, for example, the student stalls in the parameter adjustment of the desuperheating system, the knowledge of desuperheating system-operation parameter meaning-parameter adjustment is pushed (such as the association logic of parameter adjustment and steam temperature), which provides real-time knowledge support for the student. When scoring the practice, the matrix association matching degree index is introduced: by analyzing the knowledge j accessed by the student during the operation process, the high correlation knowledge corresponding to the operation in the matrix is matched, if the matching degree is high (such as accessing the operation parameter meaning instead of irrelevant component structure during parameter adjustment), the basic score is appropriately added; if the matching degree is low, the score is appropriately deducted. Among them, the traditional practice only adjusts the difficulty through the Poisson distribution control of fault frequency, lacks structured basis, and the scoring only focuses on the operation result (such as whether the start-stop is completed), ignores the understanding of the student on the knowledge-operation association; the matrix ensures that the practice difficulty matches the knowledge level of the student through the knowledge reserve threshold, and evaluates the ability of the student to know the phenomenon and the reason through the association matching degree, which is more in line with the core needs of the combination of theory and practice in power training.

[0106] Through the above steps, the embodiment realizes the improvement of training effect in multiple aspects: in terms of knowledge transmission efficiency, compared with the single parameter display of the traditional AR device, the matrix shortens the time for the student to obtain knowledge-operation association information through association priority sorting, avoiding invalid knowledge browsing; in terms of operation accuracy, the accurate pushing of high correlation knowledge reduces the operation errors caused by insufficient knowledge reserve, and reduces the risk probability of the operation risk prediction model; in terms of individualized adaptation, the dynamic modification and updating mechanism of the matrix can adjust the association strength according to the ability advancement of the student, avoiding the imbalance problem that the strong students repeat to learn the basic content or the weak students are forced to learn the complex content, and improving the training pertinence.

[0107] Embodiment 2

[0108] Reference Figure 2 For the second embodiment of the present application, an augmented reality-based education and training system is provided.

[0109] The system comprises a data collection module, a virtual scene reality module, an interactive operation module, a learning detection module and a learning arrangement module.

[0110] The data collection module constructs a virtual power generating set by using augmented reality technology, and the trainee learns power generation technology in the virtual power generating set by using augmented reality technology and collects learning data of the trainee.

[0111] The virtual scene reality module presents the running parameters of the power generating set in the virtual power generating set in real time, and dynamically adjusts the parameter types and the detailed levels of the display according to the gaze points and the interactive behaviors of the trainee.

[0112] The interactive operation module designs augmented reality interactive operation exercises of the virtual power generating set, and the trainee simulates the control of the start and stop of the power generating set and the adjustment of the parameters in the virtual environment.

[0113] The learning detection module constructs a machine learning algorithm, comprehensively analyzes the data of the operation scores, the gaze point distribution and the interactive behaviors of the trainee, establishes a personalized ability model and predicts the operation risk probability.

[0114] The learning arrangement module adaptively adjusts the exercise difficulty and the prompt strategy in the virtual power generating set based on the personalized ability model and the predicted operation risk probability of the trainee, and performs personalized learning arrangement on the trainee.

[0115] Embodiment 3

[0116] The application also provides an electronic device. The electronic device can include one or more processors and one or more memories. The memory stores computer readable code which, when executed by the one or more processors, can perform the augmented reality-based education and training method as described above.

[0117] The method or system according to the embodiments of the application can also be implemented by means of the architecture of the electronic device of the application.

[0118] The electronic device can include a bus, one or more CPUs, a read-only memory (ROM), a random access memory (RAM), a communication port connected to a network, an input / output component, a hard disk, etc.

[0119] The storage device in the electronic device, such as the ROM or the hard disk, can store the augmented reality-based education and training method provided by the application.

[0120] The augmented reality-based education training method comprises the following steps: constructing a virtual generator set by using augmented reality technology, enabling a trainee to learn power generation technology in the virtual generator set by using augmented reality technology, and collecting learning data of the trainee; presenting operation parameters of the generator set in the virtual generator set in real time, and dynamically adjusting the parameter types and the detailed levels of the display according to the gaze points and the interactive behaviors of the trainee; designing augmented reality interactive operation exercises of the virtual generator set, enabling the trainee to simulate control of starting and stopping of the generator set and adjustment of parameters in the virtual environment; constructing a machine learning algorithm, comprehensively analyzing data of operation scores, gaze point distributions and interactive behaviors of the trainee, establishing a personalized ability model, and predicting an operation risk probability; and based on the personalized ability model and the predicted operation risk probability of the trainee, adaptively adjusting the exercise difficulty and the prompt strategy in the virtual generator set, and making personalized learning arrangement for the trainee.

[0121] Further, the electronic device can further include a user interface. Of course, the architecture of the present application is only exemplary, and when implementing different devices, one or more components in the electronic device disclosed in the present application can be omitted according to actual needs.

[0122] Embodiment 4

[0123] The present application also discloses a computer readable storage medium.

[0124] The computer readable storage medium stores computer readable instructions.

[0125] When the computer readable instructions are run by the processor, the augmented reality-based education training method according to the embodiments of the present application described with reference to the above figures can be executed.

[0126] The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM), cache memory, and the like. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, and the like. In addition, according to the embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program.

[0127] For example, the application provides a non-transitory machine-readable storage medium storing machine-readable instructions executable by a processor to perform instructions corresponding to the method steps provided by the application, for example: constructing a virtual power generation set using augmented reality technology, and enabling a trainee to learn power generation technology in the virtual power generation set through the augmented reality technology and collect learning data of the trainee; presenting operation parameters of the power generation set in real time in the virtual power generation set, and dynamically adjusting the types and details of the displayed parameters according to a gaze point and interactive behavior of the trainee; designing augmented reality interactive operation exercises for the virtual power generation set, and enabling the trainee to simulate control of start and stop of the power generation set and adjustment of parameters in the virtual environment; constructing a machine learning algorithm, comprehensively analyzing data of operation scores, gaze point distributions, and interactive behaviors of the trainee, establishing a personalized ability model, and predicting an operation risk probability; and based on the personalized ability model and the predicted operation risk probability of the trainee, adaptively adjusting difficulty of exercises and prompt strategies in the virtual power generation set, and making personalized learning arrangements for the trainee.

[0128] When the computer program is executed by a central processing unit (CPU), the above-mentioned functions defined in the method of the application are performed. The method and apparatus, device of the application can be implemented in many ways. For example, the method and apparatus, device of the application can be implemented by software, hardware, firmware, or any combination of software, hardware, firmware.

[0129] The above-mentioned order of steps for the method is only for illustration, and the steps of the method of the application are not limited to the above specifically described order, unless otherwise specifically described.

[0130] In addition, in some embodiments, the application can also be implemented as programs recorded in recording media, which include machine-readable instructions for implementing the method according to the application. Thus, the application also covers recording media storing programs for executing the method according to the application.

[0131] In addition, parts of the above technical solutions provided in the embodiments of the application that are consistent with the implementation principles of corresponding technical solutions in the prior art are not described in detail to avoid excessive repetition.

[0132] The specific embodiments described above further illustrate the objects, technical solutions and advantages of the application. It should be understood that the above description is only a specific embodiment of the application and is not intended to limit the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application should be included in the protection scope of the application.

Claims

1. An augmented reality-based education training method, characterized by, The method comprises the following steps: Constructing a virtual power generation unit using augmented reality technology, and enabling students to learn power generation technology in the virtual power generation unit through augmented reality technology and collecting learning data of the students; Real-time presentation of the operating parameters of the power generation unit in the virtual power generation unit, and dynamic adjustment of the parameter types and details displayed according to the gaze points and interactive behaviors of the students; Designing augmented reality interactive operation exercises for the virtual power generation unit, and enabling the students to simulate the control of the start and stop of the power generation unit and the adjustment of parameters in the virtual environment; Building a machine learning algorithm to comprehensively analyze the operation scores, gaze point distribution and interactive behavior data of the students, establish a personalized ability model, and predict the operation risk probability; Based on the personalized ability model and the predicted operation risk probability of the students, the difficulty of the exercises and the prompt strategy in the virtual power generation unit are adaptively adjusted to arrange personalized learning for the students; Before the step of real-time presentation of the operating parameters of the power generation unit in the virtual power generation unit, the method further comprises the steps of core definition and dimension design of a component-knowledge-operation three-dimensional topological matrix, specifically as follows: Setting the matrix as a three-dimensional structure of MxLxK, wherein M is the total number of core components of the virtual power generation unit, L is the total number of knowledge dimensions, and K is the total number of operation types; Defining the nodes of each dimension of the matrix: the first dimension is the core component node, including the AR virtual unit interactive components of the boiler, steam turbine, generator and desuperheating system; the second dimension is the knowledge dimension node, including the component structure, operating parameter meaning, fault mechanism and safety specification; and the third dimension is the operation type node, including the start-stop operation, parameter adjustment, fault handling and daily inspection interactive exercise scene; Defining the physical meaning of the matrix elements: C-K-OTM(i,j,k) represents the correlation strength coefficient of component i-knowledge j-operation k, and the value range is [0,1]; the closer the coefficient is to 1, the stronger the necessity for the students to preferentially master the knowledge j to support the operation k when learning the component i.

2. The augmented reality-based education and training method of claim 1, wherein, Using a three-dimensional modeling software to establish a three-dimensional model of the power generation unit according to the structure and size of the actual power generation unit; Importing the established three-dimensional model of the power generation unit into a modeling engine, and applying augmented reality technology to realize the superimposed display and interaction of the virtual power generation unit in the real environment; Setting an interactive trigger point in the virtual power generation unit, and when the students use the AR device to aim at the selected component, the system displays the name, function and technical parameter information of the component; the students interact with the virtual component through gestures or voice commands, and the interaction includes rotation, disassembly and magnification to observe the internal structure; Collecting the learning data of the students, including the learning duration, interactive behavior and gaze point distribution; the learning duration is calculated by recording the start and end times of the students using the AR device; The interactive behavior data includes the type and frequency of the interaction triggered by the student; the gaze point distribution captures the student's line of sight direction through the front camera of the AR device and maps it into the three-dimensional coordinate system of the virtual generator set; the line of sight direction is converted into three-dimensional coordinates: ; ; ; wherein, , , respectively represent the x, y, z coordinates of the line of sight landing point in the three-dimensional space; represents the line of sight length; represents the angle between the line of sight and the y-axis; represents the angle between the projection of the line of sight on the xz plane and the z-axis.

3. The augmented reality-based education and training method of claim 2, wherein, In the three-dimensional model of the virtual power generation unit, a corresponding sensor data interface is set for each component of the power generation unit, and the virtual reality device acquires the virtual power generation unit data through the sensor data interface in real time; The augmented reality device dynamically adjusts the type and detail level of displayed parameters according to the learner's gaze point and interaction behavior; when the learner's line of sight focuses on a single component, the component's parameters are automatically displayed; when the learner moves away or the line of sight leaves the component, the parameter display is hidden or simplified.

4. The augmented reality-based education and training method of claim 3, wherein, The judging of the fixation point of the student's line of sight is realized by calculating the included angle between the line of sight vector and the normal vector of the component surface; let the line of sight vector be , and the normal vector of the point on the component surface be , then the included angle The calculation formula is: ; wherein, represents the dot product of two vectors, and respectively represent the modulus of two vectors; when is greater than the line of sight threshold , it is judged that the fixation point of the student's line of sight focuses on the component. The judgment of the learner's interaction behavior is achieved by analyzing the learner's gestures or voice commands; when the learner makes a specific gesture or speaks a specific voice command, the augmented reality device adjusts the type and detail level of parameter display accordingly.

5. The augmented reality-based education and training method of claim 1, wherein, It also includes the step of dynamically updating the component-knowledge-operation three-dimensional topological matrix, specifically: Real-time monitoring determines whether the score of a certain component-operation combination of the learner exceeds the preset threshold for multiple times in succession, or whether the risk probability of a certain type of operation predicted by the operation risk prediction model exceeds the preset risk threshold; If so, execute the update logic: if the operation score meets the standard, reduce the correlation strength of the component-basic knowledge-operation, and increase the correlation strength of the component-advanced knowledge-operation, to guide the learner to learn deeply; If the risk probability exceeds the standard, increase the correlation strength of the component-risk related knowledge-operation, and strengthen the display priority of risk related knowledge; The knowledge importance weight annotated by experts is preserved during the update process.

6. The augmented reality-based education and training method of claim 5, wherein, The step of presenting the operating parameters of the generator set in the virtual generator set in real time, and dynamically adjusting the display content according to the learner's gaze point and interaction behavior, is specifically optimized in combination with the component-knowledge-operation three-dimensional topological matrix, and the steps are: After the AR device determines that the learner is gazing at component i by the angle between the line of sight vector and the surface normal vector of the component, it extracts all knowledge-operation correlation coefficients C-K-OTM(i,j,k) corresponding to component i in the matrix; Sort the extracted correlation coefficients in descending order, and preferentially display the knowledge j and operation k instructions with higher ranking to the learner; When the learner triggers an interaction behavior through gestures or voice, the correlation coefficients of the corresponding operation k in the matrix are updated in real time, and the subsequent knowledge pushing content is dynamically adjusted.

7. The augmented reality-based education and training method of claim 6, wherein, The step of designing the augmented reality interactive operation practice of the virtual generator set is specifically optimized in combination with the component-knowledge-operation three-dimensional topological matrix to generate practice tasks, and the steps are: Before generating the practice tasks, determine the knowledge reserve threshold required to complete the task according to the component-knowledge correlation strength of the corresponding operation type k in the matrix; Determine whether the learner's current knowledge mastery meets the knowledge reserve threshold, and only when the threshold is met, the corresponding interactive operation practice task is unlocked.

8. The augmented reality-based education and training method of claim 7, wherein, The step of designing the augmented reality interactive operation practice of the virtual generator set also includes the step of optimizing the practice process support and scoring in combination with the component-knowledge-operation three-dimensional topological matrix, specifically: During the practice process, the learner's gaze time on a certain component is monitored in real time, and if the time exceeds the preset determination threshold, the high-correlation knowledge content corresponding to the component and the current operation is extracted from the matrix; When the learner completes the practice and scores, the matching degree of the knowledge j referred to by the learner during the operation process with the high-correlation knowledge corresponding to the operation in the matrix is determined; Adjust the final score according to the matching degree: if the matching degree is high, increase the score appropriately, and if the matching degree is low, decrease the score appropriately.

9. An augmented reality-based education and training system for implementing the augmented reality-based education and training method according to any one of claims 1 to 8, characterized in that, It includes: The data acquisition module, the virtual scene reality module, the interactive operation module, the learning detection module and the learning arrangement module; The data acquisition module uses augmented reality technology to construct a virtual generator set, and the trainee learns power generation technology in the virtual generator set through augmented reality technology and acquires learning data of the trainee; The virtual scene reality module presents the operation parameters of the virtual generator set in real time, and dynamically adjusts the parameter types and the detailed levels of the display according to the gaze points and the interactive behaviors of the trainee; The interactive operation module designs augmented reality interactive operation exercises of the virtual generator set, and the trainee simulates the control of the start and stop of the generator set and the adjustment of the parameters in the virtual environment; The learning detection module constructs a machine learning algorithm, comprehensively analyzes the operation scores, the gaze point distribution and the interactive behavior data of the trainee, establishes a personalized ability model and predicts the operation risk probability; The learning arrangement module adaptively adjusts the exercise difficulty and the prompt strategy in the virtual generator set based on the personalized ability model and the predicted operation risk probability of the trainee, and makes personalized learning arrangement for the trainee.

Images