An electric power failure emergency augmented reality intelligent device interaction training method and system

Abstract

The application discloses an emergency augmented reality intelligent device interaction training method and system for power failure, comprising the following steps: constructing an intelligent device interaction training system; performing teaching scene control; realizing communication data transmission through measurement and operation; adjusting virtual size according to imaging size of an actual object; performing training scene control; performing test scene control; and setting a comprehensive score index. The technical scheme realizes interaction training between devices for interactive operation in a real scene, and realizes interaction between a teacher and an operator, and effective transmission of actual judgment experience by means of data interaction through a central control platform with the aid of a VR intelligent mother mirror and a teaching manipulator cooperating with a VR intelligent son mirror and a practical operation manipulator.

Description

Technical Field

[0001] This invention relates to the field of intelligent interactive technology, and in particular to an interactive training method and system for augmented reality intelligent devices in power fault emergency response. Background Technology

[0002] Currently, the industrial metaverse is emerging, but training in industrial scenarios using VR glasses can only be conducted in a fully virtual simulation environment. Due to the inconsistent quality of modeling, it is difficult for operators to have an immersive experience, and they still do not know how to operate in a real environment. Moreover, the training only involves interaction between the operator and the system, without interaction between the instructor and the operator. It is all about exploring and operating by triggering system conditions, which makes the teaching method rather rigid and difficult to achieve efficient teaching results. Especially in scenarios that require experience to make judgments, it is still impossible to make effective judgments and reach the level of actual work.

[0003] Chinese patent document CN111308910A discloses a "Power System Simulation Teaching Platform." The platform includes: a host computer for developing a target equipment model; a real-time simulator for downloading and running the target equipment model; and power equipment. The real-time simulator is communicatively connected to at least one of the power equipment, forming a simulation system with the power equipment. The host computer is also used to display characteristic signals during the operation of the simulation system, and these characteristic signals are generated based at least on the parameters of the target equipment model and the parameters of the power equipment. This technical solution makes it difficult to achieve interaction between instructors and operators, and hinders the effective transmission of practical judgment experience. Summary of the Invention

[0004] This invention primarily addresses the problem that existing technical solutions struggle to achieve interaction between instructors and operators, and effectively transfer practical judgment experience. It provides a method and system for interactive training using augmented reality intelligent devices in power fault emergency response. This system combines real-world scenarios with interactive training between devices, utilizing a VR intelligent master mirror and teaching controller in conjunction with a VR intelligent sub-mirror and practical controller. Data interaction is achieved through a central control platform, enabling interaction between instructors and operators and effectively transferring practical judgment experience.

[0005] The above-mentioned technical problems of the present invention are mainly solved by the following technical solutions:

[0006] A method for interactive training of augmented reality smart devices in power failure emergency response includes the following steps:

[0007] S1 builds an interactive training system for intelligent devices;

[0008] S2 controls the training scenario;

[0009] S3 achieves communication data transmission through measurement and calculation;

[0010] S4 adjusts the virtual size based on the actual image size of the object;

[0011] S5 is used for training scenario control;

[0012] S6 is used for test scenario control;

[0013] S7 sets comprehensive scoring indicators.

[0014] Interactive training between devices that combine real-world scenarios utilizes a VR smart master mirror and teaching controller, along with a VR smart sub-mirror and practical controller. Data interaction is achieved through a central control platform, enabling interaction between instructors and operators and facilitating the effective transfer of practical judgment experience.

[0015] Preferably, step S2 specifically includes:

[0016] The S2.1 VR smart mirror uses a built-in camera to capture the real image in front of the smart mirror and displays it on the built-in screen. The user can point to a location or device using the teaching controller. The teaching controller measures and calculates the relative position of the point pointed to by the teaching controller on the VR smart mirror and the real image formed on the screen, and then overlays the image on the screen.

[0017] S2.2 acquires the real image in front of the smart sub-mirror through its built-in camera, projects it onto the user's screen, performs transformation and deviation calculations on the point pointed to by the teaching controller, and transmits the teaching controller data to the VR smart sub-mirror via a communication device. Simultaneously, it uses the VR smart sub-mirror's built-in direction-finding device to determine the relative position of the point pointed to by the teaching controller on the VR smart sub-mirror and the real image formed on the screen, and then superimposes this image onto the screen.

[0018] Preferably, step S2 also includes teaching equipment maintenance, operation, and emergency response: the user can generate a virtual model at the endpoint pointed to by the teaching controller through the operation keys of the teaching controller. The teaching controller obtains the relative positional relationship between the virtual model generated by the teaching controller and the real image formed by the display screen on the VR smart mirror through measurement and calculation, and superimposes the image on the display screen.

[0019] If the relative positional relationship between the virtual model imaging and the real image imaging satisfies the conditions for triggering interactive animation, the system will overlay the stored virtual animation of that point onto the bound real image. The user can use the teaching controller to call the virtual instruments and virtual devices stored in the system on the display screen to complete the correct interaction with the virtual animation and complete the teaching of equipment operation and maintenance and emergency response.

[0020] The VR smart sub-mirror acquires the real image in front of the smart sub-mirror through its built-in camera, and projects it onto the user's screen. It then performs transformation and deviation calculations on the virtual image on the VR smart master mirror, transmits the data from the VR smart master mirror to the VR smart sub-mirror through a communication device, and uses the VR smart sub-mirror's built-in orientation-finding device to obtain the relative positional relationship between the virtual image on the VR smart sub-mirror and the real image formed on the screen, and superimposes the images onto the screen.

[0021] Preferably, the measurement and calculation method in step S3 is as follows:

[0022] S3.1 When the teaching controller points to a certain location or a certain device, the ranging light in the teaching controller is emitted from the endpoint of the teaching controller to measure the distance p1 from the teaching controller to the pointed point. At the same time, the direction finding device built into the teaching controller measures the emission angle φ1 of the ranging light with a certain fixed horizontal line as the reference line.

[0023] S3.2 The pointing point, the teaching controller, and the VR smart mirror are three points that can form a triangle on a certain plane. The VR smart mirror and the teaching controller are equipped with positioning devices that periodically update their relative positions. The VR smart mirror's built-in orientation-finding device uses a fixed horizontal line as a baseline and interacts with the teaching controller's built-in orientation-finding device to measure the relative angle φ2. The distance p2 between the VR smart mirror and the teaching controller is measured using a distance measuring device. Therefore, the imaging distance of the virtual pointing point on the VR smart mirror is:

[0024]

[0025] S3.3 Confirm the imaging path based on the imaging distance ω of the virtual pointing point. Establish an actual spatial coordinate system with the VR smart mirror as the origin, the horizontal tangent direction of the VR smart mirror as the x-axis, the perpendicular direction of the horizontal tangent towards the pointing point as the y-axis, and the vertical direction as the z-axis. Measure the angles between the line connecting the two points corresponding to the lengths p1, p2, and ω and the horizontal plane using the direction-finding device on the teaching controller and the VR smart mirror. These angles are τ1, τ2, and τ3, respectively. Calculate the projections of the line connecting the two points onto the horizontal plane using the formula, i.e., l1, l2, and l3. Calculate the horizontal angle.

[0026]

[0027] Where θ1 is obtained by the direction-finding device of the VR smart mirror, and is the angle that the VR smart mirror rotates relative to a fixed horizontal line on the horizontal plane; where θ2 is the relative angle on the horizontal plane measured by the direction-finding device built into the VR smart mirror, using the same fixed horizontal line as a reference line, through interaction with the direction-finding device built into the teaching controller; given the distances from the optical path to the imaging medium and to the imaging substrate, the distance parameters between the imaging medium and the imaging substrate are fixed values ​​y0.

[0028]

[0029] S3.4 Measure the angles between the line connecting the two points corresponding to lengths p1, p2, and ω and the central plane of the VR smart mirror using the orientation-finding device on the teaching controller and the VR smart mirror. These angles are ε1, ε2, and ε3, respectively. Calculate the projections of the two-point line onto the central plane of the VR smart mirror using the formula, i.e., s1, s2, and s3. Calculate the perpendicular angle.

[0030]

[0031] θ6 is obtained by the direction-finding device of the VR smart mirror, and is the angle rotated by the VR smart mirror relative to a fixed horizontal line on the central axis plane. θ7 is the relative angle on the central axis plane measured interactively with the direction-finding device built into the teaching controller, using the same fixed horizontal line as a reference line. Given the distances from the light path to the imaging medium and to the imaging substrate, the distance parameters between the imaging medium and the imaging substrate are fixed values ​​y0.

[0032]

[0033] On the VR smart mirror's display screen, with the center point of the display screen as the origin, the horizontal axis as the x-axis, and the vertical axis as the z-axis, the coordinates of the point pointed to by the teaching controller on the VR smart mirror are (x0, z0), that is...

[0034]

[0035] As a preferred option, the transformation and deviation operations are as follows: obtain the coordinates of the point pointed to by the teaching controller on the VR smart master mirror, and based on the real image obtained by the camera on the VR smart master mirror and the real image obtained by the camera on the VR smart sub-mirror, identify the same feature point through the algorithm and calculate the displacement of the same feature point Δs(Δx,Δz). Then, after the displacement transformation, the coordinates of the point pointed to by the teaching controller on the VR smart sub-mirror should originally be (x0+Δx,z0+Δz).

[0036] To reduce image distortion that causes distance discrepancies, identify the diagonal line closest to the pointed-to point in the image. Calculate the deviation between the VR smart master mirror and this diagonal line on the VR smart master mirror, where the deviation value is Δs”-Δs'(Δx”-Δx', Δz”-Δz'). After deviation calculation, the coordinates of the point pointed to by the teaching controller on the VR smart sub-mirror should be...

[0037] Preferably, step S4 specifically includes:

[0038] S4.1 In the modeling stage of virtual models, virtual animations, virtual instruments and virtual devices, the standard size of the virtual model and the standard size of the actual imaging are set and bound. The size of the model and the imaging are determined by the distance between the feature marker points, that is, the standard size of the virtual model and the standard size of the actual imaging are u0 and w0 respectively.

[0039] S4.2 Acquires actual images through cameras on the VR smart master mirror and VR smart slave mirror, and obtains the real-time value w1 of the actual image size by identifying feature marker points through an algorithm. Then, the real-time value of the virtual model size is calculated. To determine the scaling factor, if there are several straight lines defined by the feature markers, the feature line closest to the endpoint pointed to by the teaching controller, i.e., the point where the virtual model is generated, is selected as the real-time value of the actual imaging size. If the point is moving, the value will be changed in real time according to the movement.

[0040] Preferably, step S5 specifically includes:

[0041] The S5.1 VR smart sub-mirror uses a built-in camera to capture the real image in front of the smart sub-mirror and projects it onto the user's screen. Based on the conditions triggered by the control panel for a specific training scenario, the system retrieves the sequentially ordered operation flow for that training scenario from the backend storage, obtains user operation data from the control panel, identifies the operation data, and compares it with the standard data for the current step in the operation flow. If the operation data matches the standard data, the operation flow moves to the next step; if the operation data differs from the standard data, the system provides the user with an error signal through voice, image, or vibration prompts, and virtually projects the correct standard data onto the screen.

[0042] S5.2 If the data obtained by the actual controller cannot trigger the operation process to proceed to the next step within the set time, the standard data of the current step will be virtually imaged on the display screen to guide the user to complete the current step.

[0043] S5.3 Repeat S5.1 and S5.2 based on the judgment conditions until the operation process is completed.

[0044] Preferably, step S6, which involves controlling the test scenario, specifically includes:

[0045] S6.1 The examiner selects a test scenario within the system and triggers the VR smart sub-mirror to enter the test scenario. The system calls the operation flow of the test scenario stored in the background, arranged in sequence, obtains user operation data from the actual operator, identifies the operation data, compares it with the standard data of the current step in the operation flow, analyzes the comparison data and stores it.

[0046] The S6.2 test scenario ends when the user completes the last step in the operation process, the examiner chooses to terminate the test in the system, or the time set for the test scenario has expired. After the test scenario ends, the system outputs the user's final score based on the comparison data of each step and the comprehensive scoring index calculation formula.

[0047] Preferably, the setting of the comprehensive scoring index in step S7 is as follows:

[0048] S7.1 obtains the set full score value through the system. If no critical steps are defined in the system, then based on the number of steps n in the defined operation process, calculate the average score of each step to obtain an array [μ(i)]. If the system sets key steps and corresponding key score values ​​λ, then based on the number m of key steps in the set operation process, obtain the array [μ(i)] of score values ​​for each step. * (i) = λ, This array represents the base score for the comprehensive scoring index;

[0049] S7.2 The comparison data for each step is divided into a completion score υ and an accuracy score ω. Based on the identification of operational data feature points and comparison with standard data feature points, the number of operational data feature points 'a' is counted, and the number of standard data feature points 'a0' is retrieved. The completion score υ = a / a0 is calculated. The deviation value between the operational data and the standard data is calculated, that is, the sum of the numerical deviations of all feature points in the operational data and the corresponding feature points in the standard data is Δb. The sum of the numerical deviations of all feature points in the standard data is retrieved. sum Calculate the accuracy score ω = 1 - Δb / b sum ;

[0050] S7.3 The total value of the comprehensive scoring index is calculated according to the summation formula.

[0051] A working system for an interactive training method using augmented reality intelligent devices in power fault emergency response includes an intelligent master module, intelligent sub-modules, and a central control platform. The intelligent master module and intelligent sub-modules are wirelessly connected to the central control platform. The intelligent master module includes a VR intelligent master mirror and a teaching controller, and the intelligent sub-modules include a VR intelligent sub-mirror and a practical controller. The VR intelligent master mirror and the teaching controller are wirelessly connected, and the VR intelligent sub-mirror and the practical controller are wirelessly connected. The VR master mirror and the VR sub-mirror interact with each other through the central control platform.

[0052] The beneficial effects of this invention are: it combines interactive training between devices that operate in real-world scenarios, utilizes a VR smart master mirror and teaching controller in conjunction with a VR smart sub-mirror and practical controller, and achieves data interaction through a central control platform, enabling interaction between instructors and operators and effectively transferring practical judgment experience. Attached Figure Description

[0053] Figure 1 This is a flowchart of the present invention.

[0054] Figure 2 This is a schematic diagram of a system setup according to the present invention.

[0055] Figure 3 This is a multi-view diagram of the present invention.

[0056] Figure 4 This is a schematic diagram illustrating the working principle of a VR smart master mirror according to the present invention.

[0057] Figure 5 This is a schematic diagram illustrating the working principle of a VR smart sub-mirror according to the present invention.

[0058] Figure 6 This is a schematic diagram illustrating the working principle of a VR smart master mirror and slave mirror according to the present invention. Detailed Implementation

[0059] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings.

[0060] Example: This example describes an interactive training method and system for augmented reality intelligent devices in power fault emergency response, such as... Figure 1 As shown, including

[0061] An interactive training system for augmented reality smart glasses devices in power outage emergency response includes a smart master module, smart sub-modules, and a central control platform. The smart master module and smart sub-modules are wirelessly connected to the central control platform. The smart master module includes a VR smart master mirror and a teaching controller. The smart sub-modules include a VR smart slave mirror and a practical controller. The VR smart master mirror and the teaching controller are wirelessly connected, and the VR smart slave mirror and the practical controller are wirelessly connected. The VR smart master mirror and the VR smart slave mirror interact with each other through the central control platform.

[0062] The device-to-device interactive training system includes teaching scenarios, training scenarios, and testing scenarios.

[0063] The specific method for controlling the teaching scenario is as follows:

[0064] The S1.VR smart mirror uses a built-in camera to capture the real image in front of the smart mirror and displays it on the built-in screen. The user can point to a location or device using the teaching controller. The teaching controller measures and calculates the relative position of the point pointed to by the teaching controller on the VR smart mirror and the real image formed on the display screen, and then overlays the image on the display screen.

[0065] S2. Acquire the real image in front of the smart sub-mirror through the built-in camera, and project it onto the user's eyes through the built-in display screen. Perform transformation and deviation calculations on the point pointed to by the teaching controller, and transmit the teaching controller data to the VR smart sub-mirror through the communication device. At the same time, combine the direction finding device built into the VR smart sub-mirror to obtain the relative positional relationship between the point pointed to by the teaching controller on the VR smart sub-mirror and the real image formed on the display screen, and superimpose the image on the display screen.

[0066] Furthermore, in teaching scenarios, users can not only teach about dangerous and important points, but also about equipment maintenance, operation, and emergency response. Specifically, this can include:

[0067] S3. Users can generate virtual models at the endpoints pointed to by the teaching controller using the operation keys. The teaching controller measures and calculates the relative positional relationship between the virtual model generated by the teaching controller and the real image formed by the display screen on the VR smart mirror, and then superimposes the image on the display screen. If the relative positional relationship between the virtual model image and the real image meets the conditions for triggering interactive animation, the system will superimpose the stored virtual animation of that point onto the bound real image. Users can use the teaching controller to call the virtual instruments and virtual devices stored in the system on the display screen and complete the correct interaction with the virtual animation to complete the teaching of equipment operation and maintenance and emergency fault response.

[0068] The S4.VR smart sub-mirror acquires the real image in front of the smart sub-mirror through its built-in camera and projects it onto the user's screen. It then performs transformation and deviation calculations on the virtual image (i.e., virtual models, virtual animations, virtual instruments, and virtual devices projected onto the real screen) on the VR smart master mirror. The data from the VR smart master mirror is transmitted to the VR smart sub-mirror via a communication device. Simultaneously, the relative positional relationship between the virtual image on the VR smart sub-mirror and the real image projected onto the screen is obtained using the direction-finding device built into the VR smart sub-mirror, and the images are superimposed onto the screen.

[0069] like Figure 2 and Figure 3 As shown, the measurement and calculation methods described therein can specifically be:

[0070] S101. When the teaching controller points to a certain location or a certain device, the ranging light in the teaching controller is emitted from the endpoint of the teaching controller to measure the distance p1 from the teaching controller to the pointed point. At the same time, the direction finding device built into the teaching controller measures the emission angle φ1 of the ranging light with a certain fixed horizontal line as the reference line.

[0071] S102. Since the pointing point, the teaching controller, and the VR smart mirror are three points that can form a triangle on a certain plane, and the VR smart mirror and the teaching controller are equipped with positioning devices (which can be direction-finding devices or distance-measuring devices), the relative positional relationship (distance and direction) between the VR smart mirror and the teaching controller is updated periodically. The direction-finding device built into the VR smart mirror uses the same fixed horizontal line as a reference line and measures the relative angle φ2 through interaction with the direction-finding device built into the teaching controller. The distance p2 between the VR smart mirror and the teaching controller is measured through the distance-measuring device. Then, the imaging distance of the virtual pointing point on the VR smart mirror is:

[0072]

[0073] S103. Confirm the imaging path based on the imaging distance ω of the virtual pointing point. Establish an actual spatial coordinate system with the VR smart mirror as the origin, the horizontal tangent direction of the VR smart mirror as the x-axis, the perpendicular direction of the horizontal tangent towards the pointing point as the y-axis, and the vertical direction as the z-axis. Measure the angles between the line connecting the two points corresponding to the lengths p1, p2, and ω and the horizontal plane using the teaching controller and the direction-finding device on the VR smart mirror. These angles are τ1, τ2, and τ3, respectively. Calculate the projections of the line connecting the two points onto the horizontal plane using the formula, i.e., l1, l2, and l3. Calculate the horizontal angle. Where θ1 is obtained by the direction-finding device of the VR smart mirror, and is the angle that the VR smart mirror rotates relative to a fixed horizontal line on the horizontal plane; where θ2 is the relative angle on the horizontal plane measured by the direction-finding device built into the VR smart mirror, using the same fixed horizontal line as a reference line, through interaction with the direction-finding device built into the teaching controller; given the distances from the optical path to the imaging medium and to the imaging substrate, the distance parameters between the imaging medium and the imaging substrate are fixed values ​​y0. The angles between the line connecting the two points corresponding to lengths p1, p2, and ω and the central plane of the VR smart mirror are measured using the direction-finding device on the teaching controller and the VR smart mirror. These angles are ε1, ε2, and ε3, respectively. The projections of the line connecting the two points onto the central plane of the VR smart mirror, s1, s2, and s3, are calculated using the formula. The perpendicular angle is then calculated. θ6 is obtained by the direction-finding device of the VR smart mirror, and is the angle rotated by the VR smart mirror relative to a fixed horizontal line on the central axis plane. θ7 is the relative angle on the central axis plane measured interactively with the direction-finding device built into the teaching controller, using the same fixed horizontal line as a reference line. Given the distances from the light path to the imaging medium and to the imaging substrate, the distance parameters between the imaging medium and the imaging substrate are fixed values ​​y0. On the VR smart mirror's display screen, with the center point of the display screen as the origin, the horizontal axis as the x-axis, and the vertical axis as the z-axis, the coordinates of the point pointed to by the teaching controller on the VR smart mirror are (x0, z0), that is...

[0074]

[0075] like Figure 4 , Figure 5 , Figure 6 As shown, the transformation operation and deviation operation can specifically be:

[0076] S201. According to the steps of S101-S103, the coordinates of the point pointed to by the teaching controller on the VR smart master mirror are obtained. Based on the real image obtained by the camera on the VR smart master mirror and the real image obtained by the camera on the VR smart sub-mirror, the same feature point is identified by the algorithm, and the displacement of the same feature point Δs(Δx,Δz) is calculated. Then, after the displacement transformation, the coordinates of the point pointed to by the teaching controller on the VR smart sub-mirror should originally be (x0+Δx,z0+Δz).

[0077] S202. Due to image distortion causing distance deviations, to reduce these deviations, identify the diagonal line closest to the pointed-to point in the image, and calculate the deviation between the VR smart master mirror and this diagonal line on the VR smart master mirror. The deviation value is Δs”-Δs'(Δx”-Δx', Δz”-Δz'). After the deviation calculation, the coordinates of the point pointed to by the teaching controller on the VR smart sub-mirror should be...

[0078] Furthermore, the sizes of the virtual models, virtual animations, virtual instruments, and virtual devices need to be adjusted according to the size of the actual objects displayed on the screen. The specific methods are as follows:

[0079] S5. In the modeling stage of virtual models, virtual animations, virtual instruments and virtual devices, set and bind the standard size of the virtual model and the standard size of the actual imaging. The size of the model and the imaging is determined by the distance between the feature marker points, that is, the standard size of the virtual model and the standard size of the actual imaging are u0 and w0 respectively.

[0080] S6. Acquire actual images through cameras on the VR smart master mirror and VR smart slave mirror, and obtain the real-time value w1 of the actual image size by identifying feature marker points through an algorithm. Then, calculate the real-time value of the virtual model size. To determine the scaling factor, if there are several straight lines defined by the feature markers, the feature line closest to the endpoint pointed to by the teaching controller (the point where the virtual model is generated) is selected as the real-time value of the actual imaging size. If the point is moving, the value will be changed in real time according to the movement.

[0081] The specific method for controlling the training scenario is as follows:

[0082] The S7.VR smart sub-mirror uses a built-in camera to capture real images of the front of the smart sub-mirror and projects them onto the user's screen. When a training scenario is triggered by the control panel, the system retrieves the sequentially ordered operation flow for that training scenario from the backend storage. It then obtains the user's operation data from the control panel, identifies the data, and compares it with the standard data for the current step in the operation flow. If the data matches the standard data, the operation flow moves to the next step; otherwise, if the data differs, the system provides an error signal to the user through voice, image, or vibration prompts, and virtually projects the correct standard data onto the screen.

[0083] S8. If the data obtained by the actual controller cannot trigger the operation process to proceed to the next step within the set time, the standard data of the current step will be virtually imaged on the display screen to guide the user to complete the step.

[0084] S9. Repeat S7 and S8 according to the judgment conditions until the operation process is completed.

[0085] The specific method for controlling the test scenario is as follows:

[0086] S10. The examiner selects a test scenario within the system and triggers the VR smart sub-mirror to enter the test scenario. The system calls the operation flow of the test scenario stored in the background, arranged in sequence, obtains the user operation data from the hands-on controller, identifies the operation data, compares it with the standard data of the current step in the operation flow, analyzes the comparison data and stores it.

[0087] S11. The test scenario ends when the user completes the last step in the operation process, the examiner chooses to terminate the test in the system, or the time has expired as set for the test scenario. After the test scenario ends, the system will output the user's final score based on the comparison data of each step and the comprehensive scoring index calculation formula.

[0088] The specific settings of the comprehensive scoring indicators mentioned in S11 are as follows:

[0089] S301. Obtain the set full score value through the system. If no critical steps are defined in the system, then based on the number of steps n in the defined operation process, calculate the average score of each step to obtain an array [μ(i)]. If the system defines key steps and corresponding key scores λ, then based on the number m of key steps in the defined operation process, the array [μ(i)] of scores for each step is obtained. * (i) = λ, This array represents the base score for the comprehensive scoring index;

[0090] S302. The comparison data for each step is divided into a completion score υ and an accuracy score ω. Based on the identified feature points of the operation data and their comparison with the feature points of the standard data, the number of feature points 'a' in the operation data is counted, and the number of feature points 'a0' in the standard data is retrieved. The completion score υ = a / a0 is then calculated. The deviation value between the operation data and the standard data is calculated, that is, the sum of the numerical deviations of all feature points in the operation data and the corresponding feature points in the standard data is Δb. The sum of the numerical deviations of all feature points in the standard data is retrieved. sum Calculate the accuracy score ω = 1 - Δb / b sum ;

[0091] S303. Calculate the total value of the comprehensive scoring index according to the summation formula.

Claims

1. A method for interactive training of augmented reality intelligent devices in power fault emergency response, characterized in that, Includes the following steps: S1 builds an interactive training system for intelligent devices; S2 is used for teaching scenario control; S3 achieves communication data transmission through measurement and calculation; S4 adjusts the virtual size based on the actual image size of the object; S5 is used for training scenario control; S6 is used for test scenario control; S7 sets comprehensive scoring indicators; Step S2 includes generating a virtual model at the pointing endpoint using a teaching controller. When the virtual model image and the real image meet the trigger conditions, the system overlays a virtual animation to complete the interactive teaching. The measurement and calculation in step S3 includes calculating the imaging distance and coordinates using a formula based on the triangle positioning principle formed by the pointing point, the teaching controller, and the VR smart mirror.

2. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 1, characterized in that, Step S2 specifically includes: The S2.1 VR smart mirror uses a built-in camera to capture the real image in front of the smart mirror and displays it on the built-in screen. The user can point to a location or device using the teaching controller. The teaching controller measures and calculates the relative position of the point pointed to by the teaching controller on the VR smart mirror and the real image formed on the screen, and then overlays the image on the screen. S2.2 acquires the real image in front of the smart sub-mirror through the built-in camera, and projects it onto the user's eyes through the built-in display screen. It performs transformation and deviation calculations on the point pointed to by the teaching controller, and transmits the teaching controller data to the VR smart sub-mirror through the communication device. At the same time, it combines the direction finding device built into the VR smart sub-mirror to obtain the relative positional relationship between the point pointed to by the teaching controller on the VR smart sub-mirror and the real image formed on the display screen, and superimposes the image on the display screen.

3. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 1 or 2, characterized in that, Step S2 also includes training on equipment maintenance, operation, and emergency response. Users can generate virtual models at the endpoints pointed to by the teaching controller by using the operation keys on the teaching controller. The teaching controller obtains the relative positional relationship between the virtual model generated by the teaching controller and the real image formed on the display screen through measurement and calculation, and then superimposes the image on the display screen. If the relative positional relationship between the virtual model imaging and the real image imaging satisfies the conditions for triggering interactive animation, the system will overlay the stored virtual animation of that endpoint onto the bound real image. The user can use the teaching controller to call the virtual instruments and virtual devices stored in the system on the display screen to complete the correct interaction with the virtual animation and complete the teaching of equipment operation and maintenance and emergency fault response. The VR smart sub-mirror acquires the real image in front of the smart sub-mirror through its built-in camera, and projects it onto the user's screen. It then performs transformation and deviation calculations on the virtual image on the VR smart master mirror, transmits the data from the VR smart master mirror to the VR smart sub-mirror through a communication device, and uses the VR smart sub-mirror's built-in orientation-finding device to obtain the relative positional relationship between the virtual image on the VR smart sub-mirror and the real image formed on the screen, and superimposes the images onto the screen.

4. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 1, characterized in that, The specific measurement and calculation method in step S3 is as follows: S3.1 When the teaching controller points to a certain location or a certain device, the ranging light in the teaching controller is emitted from the endpoint of the teaching controller to measure the distance p1 from the teaching controller to the pointed point. At the same time, the direction finding device built into the teaching controller measures the emission angle φ1 of the ranging light with a certain fixed horizontal line as the reference line. S3.2 The pointing point, the teaching controller, and the VR smart mirror are three points that can form a triangle on a certain plane. The VR smart mirror and the teaching controller are equipped with positioning devices that periodically update their relative positions. The VR smart mirror's built-in orientation-finding device uses a fixed horizontal line as a baseline and interacts with the teaching controller's built-in orientation-finding device to measure the relative angle φ2. The distance p2 between the VR smart mirror and the teaching controller is measured using a distance measuring device. Therefore, the imaging distance of the virtual pointing point on the VR smart mirror is: S3.3 Confirm the imaging path based on the imaging distance ω of the virtual pointing point. Establish an actual spatial coordinate system with the VR smart mirror as the origin, the horizontal tangent direction of the VR smart mirror as the x-axis, the perpendicular direction of the horizontal tangent towards the pointing point as the y-axis, and the vertical direction as the z-axis. Measure the angles between the line connecting the two points corresponding to the lengths p1, p2, and ω and the horizontal plane using the direction-finding device on the teaching controller and the VR smart mirror. These angles are τ1, τ2, and τ3, respectively. Calculate the projections of the line connecting the two points onto the horizontal plane using the formula, i.e., l1, l2, and l3. Calculate the horizontal angle. Where θ1 is obtained by the direction-finding device of the VR smart mirror, and is the angle that the VR smart mirror rotates relative to a fixed horizontal line on the horizontal plane; where θ2 is the relative angle on the horizontal plane measured by the direction-finding device built into the VR smart mirror, using the same fixed horizontal line as a reference line, through interaction with the direction-finding device built into the teaching controller; given the distances from the optical path to the imaging medium and to the imaging substrate, the distance parameters between the imaging medium and the imaging substrate are fixed values ​​y0. S3.4 Measure the angles between the line connecting the two points corresponding to lengths p1, p2, and ω and the central plane of the VR smart mirror using the orientation-finding device on the teaching controller and the VR smart mirror. These angles are ε1, ε2, and ε3, respectively. Calculate the projections of the two-point line onto the central plane of the VR smart mirror using the formula, i.e., s1, s2, and s3. Calculate the perpendicular angle. θ6 is obtained by the direction-finding device of the VR smart mirror, and is the angle rotated by the VR smart mirror relative to a fixed horizontal line on the central axis plane. θ7 is the relative angle on the central axis plane measured interactively with the direction-finding device built into the teaching controller, using the same fixed horizontal line as a reference line. Given the distances from the light path to the imaging medium and to the imaging substrate, the distance parameters between the imaging medium and the imaging substrate are fixed values ​​y0. On the VR smart mirror's display screen, with the center point of the display screen as the origin, the horizontal axis as the x-axis, and the vertical axis as the z-axis, the coordinates of the point pointed to by the teaching controller on the VR smart mirror are (x0, z0), that is...

5. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 4, characterized in that, The transformation and deviation operations are as follows: obtain the coordinates of the point pointed to by the teaching controller on the VR smart master mirror; based on the real image obtained by the camera on the VR smart master mirror and the real image obtained by the camera on the VR smart sub-mirror, identify the same feature point through the algorithm, calculate the displacement of the same feature point Δs(Δx,Δz); then after the displacement transformation, the coordinates of the point pointed to by the teaching controller on the VR smart sub-mirror should originally be (x0+Δx,z0+Δz). To reduce image distortion that causes distance discrepancies, identify the diagonal line closest to the pointed-to point in the image. Calculate the deviation between the VR smart master mirror and this diagonal line on the VR smart master mirror, where the deviation value is Δs”-Δs'(Δx”-Δx', Δz”-Δz'). After deviation calculation, the coordinates of the point pointed to by the teaching controller on the VR smart sub-mirror should be...

6. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 1, characterized in that, Step S4 specifically includes: S4.1 In the modeling stage of virtual models, virtual animations, virtual instruments and virtual devices, the standard size of the virtual model and the standard size of the actual imaging are set and bound. The size of the model and the imaging are determined by the distance between the feature marker points, that is, the standard size of the virtual model and the standard size of the actual imaging are u0 and w0 respectively. S4.2 Acquires actual images through cameras on the VR smart master mirror and VR smart slave mirror, and obtains the real-time value w1 of the actual image size by identifying feature marker points through an algorithm. Then, the real-time value of the virtual model size is calculated. To determine the scaling factor, if there are several straight lines defined by the feature markers, the feature line closest to the endpoint pointed to by the teaching controller, i.e., the point where the virtual model is generated, is selected as the real-time value of the actual imaging size. If the point is moving, the value will be changed in real time according to the movement.

7. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 1, characterized in that, Step S5 specifically includes: The S5.1 VR smart sub-mirror uses a built-in camera to capture the real image in front of the smart sub-mirror and projects it onto the user's screen. Based on the conditions triggered by the control panel for a specific training scenario, the system retrieves the sequentially ordered operation flow for that training scenario from the backend storage, obtains user operation data from the control panel, identifies the operation data, and compares it with the standard data for the current step in the operation flow. If the operation data matches the standard data, the operation flow moves to the next step; if the operation data differs from the standard data, the system provides the user with an error signal through voice, image, or vibration prompts, and virtually projects the correct standard data onto the screen. S5.2 If the data obtained by the actual controller cannot trigger the operation process to proceed to the next step within the set time, the standard data of the current step will be virtually imaged on the display screen to guide the user to complete the current step. S5.3 Repeat S5.1 and S5.2 based on the judgment conditions until the operation process is completed.

8. The interactive training method for power fault emergency augmented reality intelligent devices according to claim 1, characterized in that, Step S6, which involves controlling the test scenario, specifically includes: S6.1 The examiner selects a test scenario within the system to trigger the VR smart sub-mirror to enter that test scenario. The system then calls the sequentially numbered operation flow stored in the background for that test scenario, retrieves user operation data from the hands-on controller, identifies the operation data, compares it with the standard data for the current step in the operation flow, analyzes and stores the comparison data. S6.2 The test scenario ends when the user completes the last step in the operation flow, the examiner selects to terminate the test within the system, or the time set for the test scenario has elapsed. After the test scenario ends, the system calculates the user's final score based on the comparison data for each step using a comprehensive scoring index formula.

9. A power fault emergency augmented reality intelligent device interactive training method according to claim 1 or 8, characterized in that, The specific steps for setting the comprehensive scoring index in step S7 are as follows: S7.1 obtains the set full score value through the system. If no critical steps are defined in the system, then based on the number of steps n in the defined operation process, calculate the average score of each step to obtain an array [μ(i)]. If the system sets key steps and corresponding key score values ​​λ, then based on the number m of key steps in the set operation process, obtain the array [μ(i)] of score values ​​for each step. * (i) = λ, This array represents the base score for the comprehensive scoring index; S7.2 The comparison data for each step is divided into a completion score υ and an accuracy score ω. Based on the identification of operational data feature points and comparison with standard data feature points, the number of operational data feature points 'a' is counted, and the number of standard data feature points 'a0' is retrieved. The completion score υ = a / a0 is calculated. The deviation value between the operational data and the standard data is calculated, that is, the sum of the numerical deviations of all feature points in the operational data and the corresponding feature points in the standard data is Δb. The sum of the numerical deviations of all feature points in the standard data is retrieved. sum Calculate the accuracy score ω = 1 - Δb / b sum ; S7.3 The total value of the comprehensive scoring index is calculated according to the summation formula.

10. A working system for an interactive training method for augmented reality intelligent devices in power failure emergency response, comprising executing the interactive training method for augmented reality intelligent devices in power failure emergency response as described in any one of claims 1 to 9, characterized in that, The system includes an intelligent master module, intelligent sub-modules, and a central control platform. The intelligent master module and intelligent sub-modules are wirelessly connected to the central control platform. The intelligent master module includes a VR intelligent master mirror and a teaching controller. The intelligent sub-modules include a VR intelligent sub-mirror and a practical controller. The VR intelligent master mirror and the teaching controller are wirelessly connected, and the VR intelligent sub-mirror and the practical controller are wirelessly connected. The VR master mirror and the VR sub-mirror interact with each other through the central control platform. The device interactive training system includes teaching scenarios, training scenarios, and testing scenarios.

Images