Electric power work operation guidance method based on AR technology and AR equipment

Abstract

The invention provides an AR technology-based electric power work operation guidance method and AR equipment, and the method can comprise the steps: sequentially guiding the work of each operation item of a current electric power work operation ticket according to a set guidance process; wherein the set guidance process comprises the following steps: aiming at a first operation item of a current electric power work operation order, collecting a first image in a current environment, and the first image comprises an operation environment corresponding to the current electric power work operation order; comparing the first image with a pre-constructed twin space, and determining a real-time position in the twin space; determining and displaying route guidance information based on the real-time position and the position of the to-be-operated entity equipment corresponding to the first operation item in the operation environment; a third image in the current environment is collected, and operation indication information for the entity equipment to be operated is displayed in the third image in an overlapped mode; and collecting a fourth image in the current environment, and determining whether the first operation item is operated or not based on the fourth image.

Description

Technical Field

[0001] This application relates to the field of power operation technology, and more specifically, to a power operation guidance method and AR device based on AR technology. Background Technology

[0002] Currently, the execution of operation tickets in the power industry mainly relies on manual experience, using paper documents or electronic form systems as the medium. Operators need to hold printed documents or mobile terminals and manually compare equipment numbers, check off each step, and repeat confirmation to complete the work process. This method is highly dependent on human experience, resulting in low efficiency and a high risk of omissions or errors. Summary of the Invention

[0003] The purpose of this application is to provide a power operation guidance method and AR device based on AR (Augmented Reality) technology, which can improve the efficiency and accuracy of power operations.

[0004] In a first aspect, the present invention provides a power operation guidance method based on AR technology, applied to an AR device, comprising: sequentially guiding the operation of each operation item of the current power operation ticket according to a set guidance process; wherein, the set guidance process includes: for the first operation item of the current power operation ticket, acquiring a first image in the current environment, the first image containing the operation environment corresponding to the current power operation ticket; comparing the first image with a pre-constructed twin space to determine the real-time position in the twin space; wherein, the twin space includes the operation environment of the current power operation and a physical device located in the operation environment, the operation of the physical device and the operation presented by the twin space are shown in the image. The system performs environment binding; based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment, route guidance information is determined and displayed; if the entity device to be operated appears in the second image acquired for the current environment, a third image of the current environment is acquired, and operation instruction information for the entity device to be operated is superimposed and displayed in the third image; the operation instruction information is used to indicate the actions required for the power operation of the entity device to be operated, and the third image includes the area of ​​the entity device to be operated; a fourth image of the current environment is acquired, and based on the fourth image, it is determined whether the first operation item has been completed, and the fourth image includes the area of ​​the entity device to be operated.

[0005] In the above implementation, the real-world power operation environment and the actual physical equipment to be operated are first spatially bound together to form a twin space. Then, when power work is required, the user's real-time location and its corresponding real-time location in the twin space can be used to guide the user's route. Once the user arrives at the work location, operation instructions are displayed to guide the user and help them perform the correct actions. Finally, illustrations can be used to detect whether the operation is completed. Verifying the completion of each operation ensures the correctness of the power work, thus forming a complete power operation guide. This comprehensive power operation guide improves the accuracy and completion of power operations, and also increases efficiency.

[0006] In an optional implementation, the operation instruction information includes an operation instruction animation; the step of overlaying the operation instruction for the entity device to be operated in the third image includes: overlaying the operation instruction animation on the entity device to be operated in the third image, wherein the operation instruction animation includes an operation action for the entity device to be operated.

[0007] In the above implementation method, using animation to indicate operation actions can make the operation instructions more intuitive and easier to understand.

[0008] In an optional implementation, the operation instruction animation is overlaid on the physical device to be operated in the third image, including: searching for an operation instruction animation that matches the first operation item in a preset animation library; and overlaying the operation instruction animation that matches the first operation item on the physical device to be operated in the third image.

[0009] In the above implementation method, multiple operation animations of actual required operations can be stored in advance, which can better match multiple power operation scenarios and operation items.

[0010] In an optional implementation, the operation instruction information includes the device information of the entity device to be operated; the step of overlaying the operation instruction information for the entity device to be operated in the third image includes: overlaying the device information for the entity device to be operated in the third image.

[0011] In the above implementation, the device information of the physical device can also be overlaid on the third image, thus serving as an auxiliary prompt.

[0012] In an optional implementation, the method further includes: displaying operation text instructions for the physical device to be operated; the method further includes: receiving voice information input by the user; matching the voice information with the operation text instructions; and, if the matching is successful, executing subsequent processes.

[0013] In the above implementation method, the user can first understand the device details of the operation to be performed through text prompts.

[0014] In an optional implementation, determining and displaying route guidance information based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment includes: determining one or more candidate paths based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment; selecting route guidance information from one or more candidate paths, wherein the route guidance information does not include pre-marked dangerous areas in the operating environment; and displaying the route guidance information in the real-time acquired image with directional arrows.

[0015] In the above implementation method, by identifying multiple feasible paths and then selecting a path that avoids obstacles and dangerous areas from among these paths, the guidance of power operations can be made safer and more reliable.

[0016] In an optional implementation, determining whether the first operation item has been completed based on the fourth image includes: identifying the area of ​​the entity device to be operated in the fourth image to determine the first current state of the entity device to be operated; comparing the first current state with the expected state corresponding to the first operation item to determine whether the first operation item has been completed; and, if it is determined that the entity device to be operated has been completed, initiating the guidance process for the next operation item of the current power operation.

[0017] In an optional implementation, comparing the current state with the expected state corresponding to the first operation item to determine whether the first operation item has been completed includes: obtaining a second current state monitored by the status monitoring module of the entity device to be operated; comparing the first current state, the second current state with the expected state corresponding to the first operation item to determine whether the first operation item has been completed; wherein, if the first current state, the second current state and the expected state corresponding to the first operation item are the same, the first operation item is confirmed to have been completed.

[0018] In the above implementation method, the detection of whether the operation is completed can be achieved from two dimensions: image recognition and device state sensing detection, which can make the detection results more reliable.

[0019] In an optional implementation, the twin space is constructed by: obtaining a panoramic scan of the operating environment of the current power operation; generating a three-dimensional spatial model of the operating environment of the current power operation based on the panoramic scan; and binding the physical equipment involved in the operating environment of the current power operation to the three-dimensional spatial model, wherein the information binding the physical equipment to the three-dimensional spatial model includes spatial coordinates and attitude data.

[0020] In the above implementation method, the spatial coordinates and attitude data of the physical device can be bound to the three-dimensional spatial model.

[0021] In a second aspect, the present invention provides an AR device, comprising: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the AR device is running, the machine-readable instructions are executed by the processor to perform the steps of the method described in any of the foregoing embodiments.

[0022] In an optional implementation, the twin space is an augmented reality digital twin space; the AR device includes augmented reality glasses for displaying the visual content involved in the above-described operation guidance method. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A block diagram of an AR device provided in an embodiment of this application; Figure 2 A flowchart illustrating the AR-based power operation guidance method provided in this application embodiment; Figure 3 A schematic diagram of the operation instruction interface provided in the embodiments of this application; Figure 4 A schematic diagram illustrating the operation text instructions provided in the embodiments of this application; Figure 5 This is a schematic diagram of the operation interface of the mobile terminal provided in this application embodiment, which realizes the binding of the physical device with the three-dimensional spatial model. Detailed Implementation

[0025] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0026] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0027] In the field of operation guidance technology in the power production industry, the execution of operation tickets in the power industry mainly relies on human experience. Operation tickets are usually carried out in the form of paper documents or electronic form systems. Operators need to hold a printed operation ticket or a mobile terminal displaying an electronic operation ticket and manually compare the equipment number and check the steps to be performed on the operation ticket to complete the power operation process. However, this implementation method has the following obvious shortcomings: (1) It relies heavily on the experience accumulation and continuous focus of personnel. In complex or high-load operation environments, it is easy to make human errors such as omissions, errors, or misreading of equipment numbers due to fatigue, distraction, etc.; (2) The operation information is separated from the on-site physical environment, lacks the ability to automatically verify the identity of the operation object, and cannot perceive changes in equipment status in real time, making it difficult to intervene in time when deviations occur in the operation; (3) It relies on a static list of text descriptions. The operation guidance lacks accurate spatial location correspondence and real-time interaction capabilities. In the case of identifying similar equipment or complex wiring scenarios, it is easy to cause misoperation due to misunderstanding, which poses a safety hazard.

[0028] Based on the above research, the embodiments of this application can provide an AR-based power operation guidance method and AR device, which can not only provide guidance for power operations, but also detect the actual operation of electronic operations and whether the operation is completed, thereby improving the accuracy, effectiveness and efficiency of power operations.

[0029] To facilitate understanding of this embodiment, the AR device that performs the AR-based power operation guidance method disclosed in this application embodiment will first be described in detail.

[0030] like Figure 1 The diagram shown is a block illustration of an AR device. The AR device 100 may include a memory 111, a memory controller 112, a processor 113, a peripheral interface 114, an input / output unit 115, and a display unit 116. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the AR device 100. For example, the AR device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0031] The aforementioned memory 111, memory controller 112, processor 113, peripheral interface 114, input / output unit 115, and display unit 116 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The aforementioned processor 113 is used to execute executable modules stored in the memory.

[0032] The memory 111 can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory 111 stores programs, and the processor 113 executes these programs upon receiving execution instructions. The methods executed by the AR device 100 as defined in any embodiment of this application can be applied to the processor 113, or implemented by the processor 113.

[0033] In this embodiment, the AR device 100 can store a digital twin space within the operating environment of the power operation requiring guidance. This digital twin space may contain not only a schematic diagram of the operating environment and a schematic diagram of the physical equipment to be operated, but also other related information. The operating environment and the physical equipment can be linked together, and the linked information may include spatial coordinates and attitude data. For example, the coordinates and attitude of the physical equipment in the operating space can be recorded.

[0034] The aforementioned processor 113 may be an integrated circuit chip with signal processing capabilities. The processor 113 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a digital signal processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor.

[0035] The peripheral interface 114 described above couples various input / output devices to the processor 113 and the memory 111. In some embodiments, the peripheral interface 114, the processor 113, and the memory controller 112 can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0036] The input / output unit 115 described above is used to provide user input data. The input / output unit 115 may be, but is not limited to, a mouse and keyboard, etc.

[0037] The aforementioned display unit 116 provides an interactive interface (e.g., a user interface) between the AR device 100 and the user, or displays image data for the user's reference. In this embodiment, the display unit can be a liquid crystal display or a touch display. If it is a touch display, it can be a capacitive touchscreen or a resistive touchscreen that supports single-point and multi-point touch operations. Supporting single-point and multi-point touch operations means that the touch display can sense touch operations generated simultaneously from one or more locations on the touch display, and hand over the sensed touch operations to the processor for calculation and processing.

[0038] The AR device 100 may also include a data acquisition unit, which can be used to acquire image data. In specific power scenarios, the acquisition unit can acquire real-time environmental images of the power operation environment.

[0039] In this embodiment, the AR device 100 can be AR glasses. For example, the acquisition unit can be located on the side of the AR glasses that is furthest from the wearer's eyes when in contact with the eyes, and is used to acquire images of the surrounding environment. The AR glasses can display the actual operating environment of power operations in an augmented reality manner.

[0040] Optionally, the AR device may have a built-in IMU sensor. The IMU sensor may include an accelerometer and a gyroscope.

[0041] The AR device 100 in this embodiment can be used to execute various steps in the various methods provided in the embodiments of this application. The implementation process of the AR-based power operation guidance method is described in detail below through several embodiments.

[0042] Please see Figure 2 This is a flowchart of an AR-based power operation guidance method provided in an embodiment of this application. The AR-based power operation guidance method provided in this embodiment can be applied to AR devices, which then execute the steps in the AR-based power operation guidance method. The following will describe... Figure 2 The specific process shown will be explained.

[0043] The AR-based power operation guidance method provided in this application includes: sequentially guiding the operation of each operation item of the current power operation ticket according to the set guidance process.

[0044] A current power work operation ticket may include multiple operation items, such as switching on a voltage regulator or tripping a circuit breaker. Once a current power work operation ticket is generated, the corresponding operation items and the expected state after each operation item is performed can be determined.

[0045] The setup guide for each operation item may include the following steps.

[0046] Step 210: For the first operation item of the current power operation ticket, acquire the first image of the current environment.

[0047] The first image contains the operating environment corresponding to the current power operation ticket.

[0048] For example, the first image may represent the location of the actual operator of the current power operation.

[0049] Optionally, the AR device can capture a first image of the operating environment corresponding to the current power operation ticket in real time.

[0050] Step 230: Compare the first image with the pre-constructed twin space to determine the real-time position in the twin space.

[0051] The twin space includes the current power operation environment and the physical equipment located in the operation environment. The physical equipment is bound to the operation environment presented by the twin space.

[0052] For example, the real-time location may not only represent the real-time location of the actual operator in the twin space, but also the real-time location of the actual operator in the operating environment corresponding to the current power operation ticket.

[0053] Optionally, by combining the first image with IMU data (including acceleration and gyroscope data) collected by the IMU sensor built into the AR device, the real-time position of the operator is matched with the three-dimensional spatial model provided by the twin space to achieve centimeter-level dynamic positioning of the operator during movement.

[0054] Step 250: Based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment, determine the route guidance information and display it.

[0055] In this embodiment, the position of the physical device to be operated corresponding to the first operation item in the operating environment can be determined based on the twin space, and the distance, orientation and other information between the real-time position and the position of the physical device to be operated corresponding to the first operation item in the operating environment can be determined based on the twin space.

[0056] Route guidance information can be textual, illustrative, or other forms of information.

[0057] Taking text-based route guidance as an example, the text could be "Go straight for 10 meters, then turn left." For instance, the text could also provide route guidance based on environmental factors in the operating environment corresponding to the power work operation ticket. For example, after seeing equipment XX, you can turn right, and then go straight for XX meters.

[0058] Taking route guidance information as an example, the illustration can include directional arrows, route simulation graphics, etc.

[0059] Optionally, the route guidance information may include graphic information and text information. The route guidance information is presented in the form of holographic directional arrows and real-time distance information, and is dynamically updated based on the first image acquired in real time.

[0060] Step 270: If the physical device to be operated appears in the second image acquired for the current environment, a third image of the current environment is acquired, and operation instruction information for the physical device to be operated is superimposed and displayed in the third image.

[0061] Operation instruction information is used to indicate the actions required for electrical work on the physical equipment to be operated, and the third image includes the area of ​​the physical equipment to be operated.

[0062] Step 290: Acquire a fourth image in the current environment, and determine whether the first operation item has been completed based on the fourth image.

[0063] The fourth image includes the area of ​​the physical device to be operated. By recognizing the fourth image, the AR device can determine whether the physical device corresponding to the first operation item is in a state that has already been operated, and thus determine that the first operation item has been completed.

[0064] In this embodiment, after the first operation is completed, the execution of the second operation can continue. The specific setup and guidance process for the second operation can be found in steps 210 to 290 above, and will not be repeated here. Similarly, steps 210 to 290 above guide all operation items on the current operation ticket.

[0065] Through the steps described above, and with the assistance of a twin space linked to the physical device, users can be guided more accurately to the location of the physical device requiring operation. Furthermore, upon arrival at the desired location, operation instructions can guide the user to perform specific actions, enhancing the accuracy of their actions. Finally, a fourth image is used to verify the accuracy of each operation, further ensuring the accuracy of each user's actions.

[0066] In one alternative implementation, the operation instruction information may include operation instruction animations.

[0067] Step 270 above may include: overlaying an operation instruction animation onto the physical device to be operated in the third image.

[0068] The operation instruction animation includes the operation actions for the physical device to be operated.

[0069] like Figure 3 As shown, operation instruction animations can be gesture animations indicating the execution of an operation. For example... Figure 3 As shown, gestures can be transparent diagrams. For example, if the first operation is platen cutting, a semi-transparent virtual hand model can be overlaid on the illustrated position of the physical device to be operated, dynamically demonstrating the "press down-lock" operation trajectory, achieving visual and precise operation guidance.

[0070] In this embodiment, the operation instruction information can also be text-based. Text-based instruction information can describe the specific operation to be performed in words. For example... Figure 3 As shown, the text instruction could be "Lift the pressure plate and rotate clockwise to disconnect it." Of course, the content of the text instruction will vary depending on the specific operation. Figure 3 This is just one example for illustration.

[0071] Optionally, step 270 above may include steps 271 and 272.

[0072] Step 271: Search the preset animation library for the operation instruction animation that matches the first operation item.

[0073] The preset animation library can store operation instruction animations for various operation items. When guidance is needed for a specific operation item, the corresponding operation instruction animation can be selected from the preset animation library.

[0074] For example, each operation item can have a corresponding operation code. In power operations, the operation code of each operation item can uniquely identify an operation item. Each operation instruction animation can also be stored in association with the operation code. When it is necessary to guide the operation of the first operation item, the corresponding operation instruction animation of the first operation item can be found from the preset animation library based on the operation code of the first operation item.

[0075] Step 272: Overlay an operation instruction animation matching the first operation item onto the entity device to be operated in the third image.

[0076] For details, please refer to [link / reference]. Figure 3 As shown, the physical device to be operated is a pressure plate, and the operation instruction animation is superimposed on the pressure plate.

[0077] In the steps described above, displaying operation instructions through animation allows users to understand the actions that need to be performed more clearly and accurately.

[0078] In one optional implementation, the operation instruction information includes device information of the physical device to be operated.

[0079] Step 270 above includes: overlaying device information for the entity device to be operated on onto the third image.

[0080] like Figure 3 As shown, the device information of the entity to be operated can be displayed next to the entity. Figure 3 In the example shown, equipment information may include name, model, manufacturer, contact person, and original price of the equipment.

[0081] Optionally, when an operator approaches the physical device to be operated within a set distance range, the display interface of the AR device can display the device information of the physical device to be operated.

[0082] The above steps can also display the equipment information of the physical device to be operated, allowing users to better understand the device being operated.

[0083] The AR-based power operation guidance method of this embodiment may further include: step 281, displaying operation text instructions for the physical equipment to be operated.

[0084] The method also includes: step 282, receiving voice information input by the user; and matching the voice information with the operation text instructions.

[0085] If a match is successful, the subsequent process is executed. The subsequent process could be to allow the user to perform the first operation, and then use step 290 to detect whether the operation has been completed.

[0086] like Figure 4 As shown, when the first operation is closing the circuit breaker, the operation text instruction can be "Please repeat: Close circuit breaker #1". Figure 4 In the example shown, other buttons may also be displayed: "Manual Confirmation," "Previous Item," and "Repeat." After "Manual Confirmation" is triggered, the repeat step can be skipped. After "Previous Item" is triggered, the user can return to the previous operation's text instructions. After "Repeat" is triggered, the user can switch back to audio reception mode to receive the operator's repeated voice.

[0087] After the AR device receives the user's voice input, it uses speech recognition technology to convert the repeated speech into text. Then, it compares the converted text with the operational text instructions using semantic similarity matching rules. Only after successful verification is the next step allowed, ensuring better alignment between the user's intention and the actions performed.

[0088] In this embodiment, step 250 may include steps 251 and 252.

[0089] Step 251: Based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment, determine one or more candidate paths.

[0090] Step 252: Select route guidance information from one or more candidate routes.

[0091] The route guidance information does not include pre-marked hazardous areas in the operating environment. Hazardous areas may include the location of hazardous equipment, live areas, trenches, etc.

[0092] Step 253: Display route guidance information in the real-time acquired images using directional arrows.

[0093] By using the route planning described above, the resulting route guidance information can be made safer and more reliable.

[0094] In this embodiment, step 290 may include steps 291 and 292.

[0095] Step 291: Identify the area of ​​the entity device to be operated in the fourth image to determine the first current state of the entity device to be operated.

[0096] Taking a pressure plate as an example, the first current state can be either the pressure plate being disconnected or the pressure plate being locked. Of course, depending on the actual operation and the specific device being operated, the states included in the first current state may also differ.

[0097] Optionally, the AR device can capture a fourth image containing the area of ​​the entity to be operated at a set frame rate. For example, a fourth image containing the area of ​​the entity to be operated can be captured at a frame rate of 30 fps.

[0098] Step 292: Compare the first current state with the expected state corresponding to the first operation item to determine whether the first operation item has been completed.

[0099] The expected state corresponding to the first operation item can be determined based on the current power operation ticket.

[0100] The guidance process involves initiating the next operation for the current power work only after confirming that the operation of the physical equipment to be operated has been completed.

[0101] In this embodiment, in order to further improve the accuracy of whether the operation is completed, step 292 may include steps 2921 to 2923.

[0102] Step 2921: Obtain the second current status monitored by the status monitoring module of the physical device to be operated.

[0103] Step 2922: Compare the first current state, the second current state, and the expected state corresponding to the first operation item to determine whether the first operation item has been completed.

[0104] Specifically, if the first current state, the second current state, and the expected state corresponding to the first operation item are the same, the operation of the first operation item is confirmed to be completed.

[0105] Specifically, if any one of the first current state, the second current state, and the expected state corresponding to the first operation item differs from the other two states, it is confirmed that the first operation item has not been completed. The AR device can send a prompt message to the designated device, or display a prompt message on the AR device indicating that the first operation item has failed and requires further confirmation.

[0106] In this embodiment, each physical device in the operating environment corresponding to the current power operation ticket can be configured with a status monitoring module to realize real-time status monitoring.

[0107] AR devices can obtain the status of the physical device to be operated in real time through IoT sensors and remote sensing and telemetry systems.

[0108] The above-mentioned methods enable the detection of the physical device to be operated, and can detect whether the operation result of the first operation item meets the expectations, thus realizing vision-based closed-loop verification and error prevention confirmation.

[0109] In this embodiment, before step 210, the power operation guidance method based on AR technology may also include steps 310 to 330.

[0110] Step 310: Obtain a panoramic scan of the current power operation environment.

[0111] For example, a panoramic scan image can be an image obtained by scanning the operating environment of the current power operation using a panoramic camera.

[0112] Step 320: Based on the panoramic scan image, generate a three-dimensional spatial model of the current power operation environment.

[0113] A three-dimensional spatial model with centimeter-level precision is generated using point cloud data processing technology.

[0114] Step 330: Bind the physical equipment involved in the current power operation environment to the three-dimensional spatial model.

[0115] The information bound between the physical device and the 3D spatial model includes spatial coordinates and attitude data.

[0116] Operators can access the existing 3D spatial model's environment via mobile terminals and automatically identify the spatial model they are in. When approaching the physical device, the interface guides the setting of spatial anchor points: by moving the handheld mobile terminal to align the positioning mark in the center of the screen with a key part of the physical device, the mobile terminal can then use the physical device's position and orientation information to bind the physical device to the 3D spatial model. The mobile terminal can record the current spatial coordinates (x, y, z) and quaternion orientation (q1, q2, q3, q4) of the aiming frame and bind them to the physical device ID.

[0117] like Figure 5 As shown, it illustrates a schematic diagram of the user interface for a mobile terminal in binding a physical device to a 3D spatial model. Figure 5In the example shown, the diagram displays the coordinates and orientation information of the physical device. The coordinates of this physical device correspond to the position of the positioning marker at the center of the screen. The diagram also shows directional arrows (movement-related and rotation-related arrows) used to align the positioning marker at the center of the screen with key parts of the physical device. For example, the positioning marker at the center of the screen can be aligned with the key parts of the physical device to be entered by moving the mobile device. Once the positioning marker at the center of the screen aligns with the key parts of the physical device to be entered, clicking the "Enter" button on the display interface will bind the positioning marker at the center of the screen to the currently bound physical device.

[0118] exist Figure 5 The positioning marker in the center of the screen is shown as an example of a white rectangle.

[0119] exist Figure 5 In the example shown, the display interface may also include a "Lock" button and a "Reset" button.

[0120] By binding physical devices to a 3D spatial model, it is possible to better guide users to the location of the physical devices when actually performing operations. Intelligent guidance and operation instructions based on real-time status awareness and operation tickets enable intelligent error prevention and dynamic verification.

[0121] Furthermore, combining AR devices can provide standardized operation guidance and closed-loop verification through first-person perspective and virtual-real interaction. This can solve the problem of "information disconnect from the actual situation" in traditional operation tickets, transforming abstract text instructions into intuitive, visually precise guidance anchored to specific devices, thus reducing misoperations caused by searching for or misidentifying devices.

[0122] In this embodiment, the error prevention function is upgraded from traditional post-event logical judgment to pre-event proactive safety intervention based on vision and real-time data, constructing a dual safety barrier of "human defense + technical defense", which improves the accuracy of operation and also enhances the safety and efficiency of power operations.

[0123] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the AR-based power operation guidance method described in the above method embodiments.

[0124] The computer program product of the AR-based power operation guidance method provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the AR-based power operation guidance method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0126] In addition, the method steps in the various embodiments of this application can be integrated together to form an independent part for execution, or each method step can be executed by a separate module, or two or more steps can be formed into an independent part for execution.

[0127] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0128] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A power operation guidance method based on AR technology, characterized in that, Applied to AR devices, including: Following the established guidance process, guide the operation of each item on the current power work operation ticket in sequence; The setup guidance process includes: For the first operation item of the current power operation ticket, a first image of the current environment is captured, and the first image contains the operation environment corresponding to the current power operation ticket; The first image is compared with a pre-constructed twin space to determine the real-time location in the twin space; wherein the twin space includes the operating environment of the current power operation and physical equipment located in the operating environment, and the physical equipment is bound to the operating environment presented by the twin space; Based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment, route guidance information is determined and displayed; If the physical device to be operated appears in the second image acquired in the current environment, a third image of the current environment is acquired, and operation instruction information for the physical device to be operated is superimposed and displayed in the third image; the operation instruction information is used to indicate the actions required for electrical work on the physical device to be operated, and the third image includes the area of ​​the physical device to be operated; A fourth image of the current environment is acquired, and the first operation item is determined based on the fourth image, wherein the fourth image includes the area of ​​the entity device to be operated.

2. The method according to claim 1, characterized in that, The operation instruction information includes operation instruction animations; The step of overlaying and displaying operation instructions for the entity device to be operated on in the third image includes: The operation instruction animation is overlaid on the entity device to be operated in the third image, wherein the operation instruction animation includes operation actions for the entity device to be operated.

3. The method according to claim 2, characterized in that, The operation instruction animation is overlaid on the entity device to be operated in the third image, including: Search the preset animation library for an operation instruction animation that matches the first operation item; An operation instruction animation matching the first operation item is overlaid on the entity device to be operated in the third image.

4. The method according to claim 1, characterized in that, The operation instruction information includes the device information of the entity device to be operated; The operation instruction information for the entity device to be operated, which is overlaid and displayed in the third image, includes: The device information for the entity device to be operated is overlaid in the third image.

5. The method according to claim 1, characterized in that, The method further includes: displaying operation text instructions for the physical device to be operated; The method further includes: receiving voice information input by the user; matching the voice information with the operation text instruction; and executing subsequent processes if the matching is successful.

6. The method according to claim 1, characterized in that, The step of determining and displaying route guidance information based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment includes: Based on the real-time location and the location of the entity device to be operated corresponding to the first operation item in the operating environment, one or more candidate paths are determined; Select route guidance information from one or more of the candidate paths, wherein the route guidance information does not include pre-marked hazardous areas in the operating environment; The route guidance information is displayed in the real-time captured image using directional arrows.

7. The method according to claim 1, characterized in that, Determining whether the first operation item has been completed based on the fourth image includes: The region of the entity device to be operated in the fourth image is identified to determine the first current state of the entity device to be operated; The first current state is compared with the expected state corresponding to the first operation item to determine whether the first operation item has been completed. Once it is confirmed that the operation of the physical equipment to be operated has been completed, the guidance process for the operation guidance of the next operation item of the current power operation will be initiated.

8. The method according to claim 7, characterized in that, The step of comparing the current state with the expected state corresponding to the first operation item to determine whether the first operation item has been completed includes: Obtain the second current state monitored by the status monitoring module of the entity device to be operated; The first current state, the second current state, and the expected state corresponding to the first operation item are compared to determine whether the first operation item has been completed; wherein, if the first current state, the second current state, and the expected state corresponding to the first operation item are the same, the first operation item is confirmed to have been completed.

9. The method according to any one of claims 1-8, characterized in that, The twin space is constructed in the following ways: Obtain a panoramic scan of the current power operation environment; Based on the panoramic scan image, a three-dimensional spatial model of the current power operation environment is generated; The physical equipment involved in the current power operation environment is bound to the three-dimensional spatial model, wherein the information binding the physical equipment to the three-dimensional spatial model includes spatial coordinates and attitude data.

10. An AR device, characterized in that, include: The processor and memory, wherein the memory stores machine-readable instructions executable by the processor, and when the AR device is running, the machine-readable instructions are executed by the processor to perform the steps of the method as described in any one of claims 1 to 9.

11. The AR device according to claim 10, characterized in that, The twin space is an augmented reality digital twin space; The AR device includes augmented reality glasses for displaying the visual content involved in the operation guidance method according to any one of claims 1 to 9.

Images