Substation equipment simulation maintenance method, device and terminal

Abstract

The application provides a substation equipment simulation maintenance method and device and a terminal. The method comprises the following steps: acquiring a field image of a substation equipment through an AR terminal; identifying the substation equipment in the field image and acquiring actual operation data and a virtual toolbox corresponding to the substation equipment; and sending the virtual toolbox to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual toolbox and the field image; acquiring a target virtual tool sent by the AR terminal and a maintenance action selected by a maintenance personnel; and determining simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data, wherein the simulation data comprises a three-dimensional model and simulation operation data. The application uses augmented reality technology, so that the maintenance personnel can perform simulation maintenance in combination with the real structure of the equipment at the maintenance site, and the on-site operation and training learning effect of the maintenance personnel can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of substation maintenance, and in particular to a simulation maintenance method and device for substation equipment and a terminal. BACKGROUND

[0002] Currently, the equipment in a substation has complex structure, long operation time and many times of transformation, which results in many cases of inconsistency between as-built drawings and actual field wiring, and the visualization degree of connection relationship is low because the secondary circuit is realized by relying on the optical fiber connection between SCD (Substation Configuration Description) and equipment.

[0003] In order to improve the maintenance level of maintenance personnel, the maintenance level of maintenance personnel is currently trained and tested by a virtual reality (VR) system, but the above characteristics of substation equipment make the actual structure of equipment in the substation inconsistent with the virtual scene in the VR system, which results in poor training effect of maintenance personnel using the VR system, and low efficiency and poor effect of maintenance personnel in on-site maintenance. SUMMARY

[0004] The present application provides a simulation maintenance method and device for substation equipment and a terminal to solve the problem of poor training effect of maintenance personnel using a VR system.

[0005] In a first aspect, the present application provides a simulation maintenance method for substation equipment, applied to a server, comprising:

[0006] obtaining a field image of the substation equipment by an AR terminal;

[0007] recognizing the substation equipment in the field image, obtaining actual operation data and a virtual toolbox corresponding to the substation equipment, and sending the virtual toolbox to the AR terminal to make the AR terminal fuse and display the actual operation data, the virtual toolbox and the field image;

[0008] obtaining a target virtual tool sent by the AR terminal and a maintenance action selected by the maintenance personnel, the target virtual tool being a virtual tool selected by the maintenance personnel from the virtual toolbox;

[0009] determining simulation data of the substation equipment after implementing the maintenance action based on the target virtual tool and the actual operation data, the simulation data including a three-dimensional model and simulation operation data.

[0010] In a possible implementation manner, determining the simulation data of the substation equipment after implementing the maintenance action based on the target virtual tool and the actual operation data includes:

[0011] An acquisition target part is a part selected by a maintenance personnel in a live image for simulating maintenance of the substation equipment;

[0012] Finding a three-dimensional model of the target part after performing the maintenance action in the equipment database;

[0013] After determining the simulation data of the substation equipment after performing the maintenance action based on the target virtual tool and the actual operation data, the method further comprises:

[0014] Sending the three-dimensional model of the target part after performing the maintenance action to the AR terminal, so that the AR terminal fuses and displays the three-dimensional model of the target part after performing the maintenance action with the live image.

[0015] In a possible implementation, determining the simulation data of the substation equipment after performing the maintenance action based on the target virtual tool and the actual operation data comprises:

[0016] Determining a corresponding operation data change rule based on the substation equipment, the target virtual tool and the maintenance action;

[0017] Calculating the simulation operation data of the substation equipment after performing the maintenance action based on the actual operation data and the operation data change rule.

[0018] In a possible implementation, the simulation data further comprises a simulation maintenance result;

[0019] Determining the simulation data of the substation equipment after performing the maintenance action based on the target virtual tool and the actual operation data comprises:

[0020] Comparing the specific data of the target inspection item obtained from the AR terminal with the specific data of the corresponding item in the preset task work order, if they are consistent, the simulation maintenance result of the target inspection item is correct, if they are inconsistent, the simulation maintenance result of the target inspection item is incorrect; the target inspection item comprises a virtual tool, a maintenance action and an execution sequence.

[0021] In a possible implementation, identifying the substation equipment in the live image comprises:

[0022] Identifying the substation equipment in the live image using a YOLO v3 model.

[0023] In a second aspect, the present application provides a simulation maintenance method of substation equipment, applied to an AR terminal, comprising:

[0024] Collecting a live image of the substation equipment; and sending the live image to a server, so that the server identifies the substation equipment in the live image and obtains actual operation data and a virtual tool box corresponding to the substation equipment;

[0025] The actual operation data and the virtual tool box sent by the server are acquired, and the actual operation data, the virtual tool box and the field image are fused and displayed;

[0026] The target virtual tool and the maintenance action selected by the maintenance personnel are acquired, and the target virtual tool is the virtual tool selected by the maintenance personnel from the virtual tool box;

[0027] The target virtual tool and the maintenance action selected by the maintenance personnel are sent to the server, so that the server determines the simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data;

[0028] The simulation data of the substation equipment after the maintenance action is implemented sent by the server is received, and the simulation data and the field image are fused and displayed, and the simulation data includes a three-dimensional model and simulation operation data.

[0029] In a third aspect, the present application provides a simulation maintenance device of a substation equipment, comprising:

[0030] The first acquisition module is configured to acquire a field image of the substation equipment through the AR terminal;

[0031] The device identification module is configured to identify the substation equipment in the field image, acquire actual operation data and a virtual tool box corresponding to the substation equipment, and send the virtual tool box to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual tool box and the field image;

[0032] The second acquisition module is configured to acquire the target virtual tool and the maintenance action selected by the maintenance personnel sent by the AR terminal, and the target virtual tool is the virtual tool selected by the maintenance personnel from the virtual tool box;

[0033] The data calculation module is configured to determine simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data, and the simulation data includes a three-dimensional model and simulation operation data.

[0034] In a fourth aspect, the present application provides a terminal, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method shown in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0035] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method shown in the first aspect or any possible implementation manner of the first aspect.

[0036] The application provides a simulation maintenance method, device and terminal of a substation equipment, the method comprises the following steps: obtaining a field image of the substation equipment through an AR terminal; identifying the substation equipment in the field image, and obtaining actual operation data and a virtual toolbox corresponding to the substation equipment; and sending the virtual toolbox to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual toolbox and the field image; obtaining a target virtual tool and a maintenance action selected by a maintenance personnel, the target virtual tool being a virtual tool selected by the maintenance personnel from the virtual toolbox; determining simulation data of the substation equipment after implementing the maintenance action based on the target virtual tool and the actual operation data, the simulation data comprising a three-dimensional model and simulation operation data. The application uses augmented reality technology, so that the maintenance personnel can perform simulation maintenance in combination with the real structure of the equipment at the maintenance site, and the on-site operation and training learning effect of the maintenance personnel can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative labor.

[0038] Figure 1 is an implementation flowchart of the simulation maintenance method of the substation equipment provided by an embodiment of the present application;

[0039] Figure 2 is an implementation flowchart of the simulation maintenance method of the substation equipment provided by another embodiment of the present application;

[0040] Figure 3 is a structural schematic diagram of the simulation maintenance device of the substation equipment provided by the embodiment of the present application;

[0041] Figure 4 is a schematic diagram of the terminal provided by the embodiment of the present application. DETAILED DESCRIPTION

[0042] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons of ordinary skill in the art will readily recognize that embodiments of the application can be practiced without some or all of the specific details set forth herein. In other instances, well-known structures, circuits, and processes have not been described in detail in order to not unnecessarily obscure aspects of the application.

[0043] In order to make the objects, technical solutions and advantages of the present application clearer, the following will be described by specific embodiments in combination with the drawings.

[0044] Referring to Figure 1 It shows an implementation flowchart of a simulation maintenance method of a substation device provided by an embodiment of the application, the method being applied to a server, and is described in detail as follows:

[0045] In step 101, a live image of the substation device is acquired through an AR terminal.

[0046] In this embodiment, the method is applied to a server in a simulation maintenance system of a substation device, and the system further includes an Augmented Reality (AR) terminal. The AR terminal in the simulation maintenance system of the substation device is used to collect a live image of the substation device and send it to the server. The server processes the live image and then sends data required by the AR terminal to the AR terminal, so as to realize fusion and interaction of virtual information and real images.

[0047] In step 102, the substation device in the live image is identified, actual operation data corresponding to the substation device and a virtual toolbox are acquired, and the virtual toolbox is sent to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual toolbox and the live image.

[0048] In this embodiment, the server needs to identify the substation device in the live image, determine a virtual toolbox required for simulation maintenance of the device based on the identified substation device, and acquire actual operation data of the device through a sensor or a communication module arranged in the substation. The AR terminal fuses and displays the actual operation data, the virtual toolbox and the live image, so that a maintenance personnel selects a corresponding maintenance tool in the virtual toolbox based on the actual operation data and performs simulation maintenance.

[0049] The virtual toolbox includes a plurality of virtual maintenance tools, including a virtual screwdriver, a virtual cable, a virtual multimeter, a virtual phase sequence meter, a virtual three-phase / single-phase voltage and current source, a virtual mutual inductor polarity tester, a virtual network tester, a virtual oscilloscope and a virtual clock tester. They can complete various functions possessed in an actual scene in a virtual scene.

[0050] Specifically, the virtual screwdriver can complete the disconnection and connection of the secondary terminal of the substation in the augmented reality environment. The virtual cable can complete the short-circuiting and cross-connection of the secondary terminal of the substation in the augmented reality environment. The virtual multimeter can complete the measurement of the voltage, current, direct resistance, and loop on-off of the secondary terminal of the substation in the augmented reality environment. The virtual phase sequence meter can complete the measurement of the positive and negative phase sequence of the three-phase voltage of the substation in the augmented reality environment. The virtual three-phase / single-phase voltage and current source can apply the required voltage and current to the secondary equipment in the augmented reality environment, including three-phase / single-phase voltage and current of different values and angles, and can set the continuous output quantity at a certain time step. The virtual mutual inductor polarity tester can complete the measurement of the connection polarity of the current transformer in the augmented reality environment. The virtual network tester can test the network throughput, time delay, packet loss rate, traffic, and storm suppression of the substation in the augmented reality environment. The virtual oscilloscope can display the waveforms of the analog and switching quantities of the primary and secondary equipment of the substation in the augmented reality environment, and can scale, position, and compare the relevant waveforms and convert the relevant transformation ratios. The virtual clock tester can test the accuracy of the time synchronization system of the substation in the augmented reality environment.

[0051] In step 103, the AR terminal sends the target virtual tool and the maintenance action selected by the maintenance personnel. The target virtual tool is the virtual tool selected by the maintenance personnel from the virtual tool box.

[0052] In this embodiment, after the maintenance personnel selects the target virtual tool, the AR terminal also displays one or more maintenance actions corresponding to the target virtual tool, and the maintenance personnel selects a maintenance action from them to use the maintenance action to maintain the substation equipment in the simulation maintenance process.

[0053] In step 104, the simulation data of the substation equipment after the implementation of the maintenance action is determined based on the target virtual tool and the actual running data. The simulation data includes a three-dimensional model and simulation running data.

[0054] In this embodiment, after the maintenance personnel uses the selected maintenance action to simulate the maintenance of the substation equipment, the simulation of the substation equipment will change to reflect the influence of the selected maintenance action on the substation equipment.

[0055] In some embodiments, determining the simulation data of the substation equipment after the implementation of the maintenance action based on the target virtual tool and the actual running data includes:

[0056] The target part is obtained, which is the part selected by the maintenance personnel in the field image for simulating the maintenance of the substation equipment;

[0057] The three-dimensional model of the target part after the implementation of the maintenance action is searched in the equipment database.

[0058] After determining the simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data, the method further comprises:

[0059] sending the three-dimensional model of the target site after the maintenance action is implemented to the AR terminal, so that the AR terminal fuses and displays the three-dimensional model of the target site after the maintenance action is implemented with the live image.

[0060] In this embodiment, since the substation equipment has a large volume and complex parts, the maintenance personnel can also select specific sites on the substation equipment for simulated maintenance. Specifically, after the server identifies the substation equipment in the live image, it identifies and divides each preset site in the substation equipment, so that the AR terminal displays the sites in the substation equipment that can be simulated for maintenance.

[0061] After the maintenance personnel selects the maintenance action, the server sends the three-dimensional model of the target site after the maintenance action is implemented to the AR terminal, so that the AR terminal fuses and displays the three-dimensional model of the target site after the maintenance action is implemented with the live image.

[0062] The equipment database includes model data, actual operation data and job data of each substation equipment.

[0063] The model data includes account information of the substation equipment, three-dimensional models of the substation equipment in various states, substation configuration description (SCD), substation loop configuration description (SLCD), substation physical configuration description (SPCD) and electronic drawings, etc. The model data is used to describe the attributes of the substation equipment, the logical relationship of the substation equipment operation data and the cable fiber connection relationship of each device in the substation.

[0064] The actual operation data includes voltage, current, temperature, optical port intensity and alarm data of the substation equipment, and is used to display the real-time operation state of the substation equipment.

[0065] The job data includes task work orders of substation jobs, including equipment inspection task work orders, equipment fault elimination task work orders and equipment debugging task work orders. The server can store and manage each task work order.

[0066] In some embodiments, determining the simulation data of the substation equipment after the maintenance action based on the target virtual tool and the actual operation data comprises:

[0067] determining a corresponding operation data change rule based on the substation equipment, the target virtual tool, and the maintenance action;

[0068] calculating the simulation operation data of the substation equipment after the maintenance action based on the actual operation data and the operation data change rule.

[0069] In this embodiment, the maintenance action on the substation equipment will change the operation state of the substation equipment, and the operation data of the substation equipment will also change accordingly. The specific change rule depends on the target virtual tool, and the simulation operation data of the substation equipment after the maintenance action can reflect the specific operation state of the substation equipment after the real-time maintenance action. For example, after connecting the secondary equipment with the virtual power supply, current will be generated inside the secondary equipment. The simulation operation data calculated by the server is the current parameter inside the secondary equipment, and the server can determine whether the secondary equipment can normally operate after the real-time maintenance action based on the calculated current parameter.

[0070] In some embodiments, the simulation data further comprises a simulation maintenance result.

[0071] In some embodiments, determining the simulation data of the substation equipment after the maintenance action based on the target virtual tool and the actual operation data comprises:

[0072] comparing the specific data of the target inspection item obtained from the AR terminal with the specific data of the corresponding item in the preset task work order. If they are consistent, the simulation maintenance result of the target inspection item is correct, and if they are not consistent, the simulation maintenance result of the target inspection item is incorrect. The target inspection item includes a virtual tool, a maintenance action, and an execution sequence.

[0073] In this embodiment, the server can also make error correction judgments on the simulation maintenance process of the maintenance personnel according to each inspection item in the preset task work order, and feed back the corresponding judgment result after the maintenance personnel completes each operation. Specifically, the server can check the virtual tool selected by the maintenance personnel, the maintenance action, and the sequence of the maintenance action, and issue reminders and warnings when errors or dangerous results occur. The server can also store the specific operation steps of the maintenance personnel in the simulation maintenance process in the operation data of the device database, and closed-loop manage the simulation maintenance.

[0074] In some embodiments, identifying the substation equipment in the field image comprises:

[0075] identifying the substation equipment in the field image using a YOLO v3 model.

[0076] In the embodiment, the YOLO v3 model is improved compared with the YOLO model as follows: the network structure is adjusted; multi-scale features are used for object detection; and a Logistic function is used instead of a softmax function for object classification. The prediction accuracy is improved while the speed advantage is maintained, and the ability to identify small objects is particularly enhanced.

[0077] The simulation maintenance method for the substation equipment provided in the embodiment comprises: acquiring a field image of the substation equipment through an AR terminal; identifying the substation equipment in the field image, and acquiring actual operation data and a virtual toolbox corresponding to the substation equipment; and sending the virtual toolbox to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual toolbox and the field image; acquiring a target virtual tool and a maintenance action selected by a maintenance personnel, the target virtual tool being a virtual tool selected by the maintenance personnel from the virtual toolbox; determining simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data, the simulation data comprising a three-dimensional model and simulation operation data. The simulation maintenance method uses augmented reality technology, so that the maintenance personnel can perform simulation maintenance in combination with the real structure of the equipment at the maintenance site, and the on-site operation and training learning effect of the maintenance personnel can be improved.

[0078] Referring to Figure 2 The simulation maintenance method for the substation equipment provided in the embodiment comprises: acquiring a field image of the substation equipment through an AR terminal; identifying the substation equipment in the field image, and acquiring actual operation data and a virtual toolbox corresponding to the substation equipment; and sending the virtual toolbox to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual toolbox and the field image; acquiring a target virtual tool and a maintenance action selected by a maintenance personnel, the target virtual tool being a virtual tool selected by the maintenance personnel from the virtual toolbox; determining simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data, the simulation data comprising a three-dimensional model and simulation operation data. The simulation maintenance method uses augmented reality technology, so that the maintenance personnel can perform simulation maintenance in combination with the real structure of the equipment at the maintenance site, and the on-site operation and training learning effect of the maintenance personnel can be improved.

[0079] Step 201: a field image of the substation equipment is acquired; and the field image is sent to a server, so that the server identifies the substation equipment in the field image and acquires actual operation data and a virtual toolbox corresponding to the substation equipment;

[0080] Step 202: the actual operation data and the virtual toolbox sent by the server are acquired, and the actual operation data, the virtual toolbox and the field image are fused and displayed;

[0081] Step 203: a target virtual tool and a maintenance action selected by a maintenance personnel are acquired, the target virtual tool being a virtual tool selected by the maintenance personnel from the virtual toolbox;

[0082] Step 204: the target virtual tool and the maintenance action selected by the maintenance personnel are sent to the server, so that the server determines simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data;

[0083] Step 205: the simulation data of the substation equipment after the maintenance action is implemented sent by the server is received, and the simulation data and the field image are fused and displayed, the simulation data comprising a three-dimensional model and simulation operation data.

[0084] In the embodiment, the AR terminal can be an AR smart glasses, a smart phone, a PDA, or the like smart device with high-definition camera function. The AR terminal includes a front-end acquisition module, a communication module, a calculation module, and a display module. The front-end acquisition module is configured to acquire images or videos of the substation equipment; the communication module is configured to communicate with the server, and send the on-site images, the target virtual tool, and the maintenance action to the server, and receive the actual operation data, the virtual toolbox, and the simulated data of the substation equipment after implementing the maintenance action sent by the server, and the communication with the server can be implemented through a 5G network; the calculation module is configured to complete three-dimensional registration of the virtual information and the on-site images, and fuse and superimpose the virtual information and the on-site images, so that the display module displays the superimposed images.

[0085] In some embodiments, the simulated data and the on-site images are fused and displayed, including:

[0086] identifying a region of the substation equipment in the on-site images, and taking the region of the substation equipment as a three-dimensional registration region;

[0087] converting the three-dimensional model from the world coordinate system to an imaging plane coordinate system of the on-site images, and superimposing the converted three-dimensional model to the three-dimensional registration region to obtain a simulated maintenance scene;

[0088] displaying the simulated maintenance scene.

[0089] In the embodiment, the AR terminal determines the positional relationship among the substation equipment, the AR terminal, and the imaging plane through three-dimensional registration, converts the three-dimensional model to the imaging plane coordinate system, and superimposes the three-dimensional model with the on-site images to obtain a simulated maintenance scene in which the virtual information is combined with the on-site images. The coordinate conversion relationship is:

[0090]

[0091] wherein, Z c is a scale factor, (u, v) is the coordinate of the substation equipment in the imaging plane coordinate system, (x w , y w , z w ) is the coordinate of the substation equipment in the world coordinate system, K0 is an intrinsic matrix of the camera of the AR terminal, which is related to the focal length and other parameters of the camera, and P0 is an extrinsic matrix of the camera, which is composed of a rotation matrix R and a translation vector T.

[0092] In a specific embodiment, when the maintenance personnel uses the simulated maintenance system to perform a simulated function debugging task on the 220kV line protection equipment, the specific steps are as follows:

[0093] Step one: the server adds and verifies the functions of various virtual tools in the device database, obtains all real-time operation data of secondary devices from the background of the secondary system monitoring of the substation, and completes the establishment of all model data, actual operation data and job data in the device database.

[0094] Step two: the maintenance personnel of the substation hold the AR terminal subsystem to the secondary screen cabinet of the 220kV line protection device.

[0095] Step three: the maintenance personnel use the AR terminal subsystem to take pictures of the secondary devices in the screen cabinet, complete image acquisition of the devices, and transmit the acquired images to the server for device attribute recognition.

[0096] Step four: the server identifies that the secondary device is "220kV line protection device", and matches it with the "220kV line protection device" keyword in the device database, extracts the data information corresponding to the 220kV line protection device, and sends the data information to the AR terminal.

[0097] Step five: the AR terminal performs three-dimensional registration of the collected images.

[0098] Step six: the AR terminal performs image fusion of the data information corresponding to the 220kV line protection device according to the registration coordinates and the collected images, thereby displaying the enhanced images.

[0099] Step seven: carry out the function debugging of "220kV line protection device" in the augmented reality environment, as follows:

[0100] S1: call the three-dimensional model of "220kV line protection device", set up safety measures under the three-dimensional model, view the running state of the protection, exit the protection function pad, exit the protection outlet pad; call the screwdriver in the virtual maintenance instrument to disconnect the secondary terminal voltage and current terminals of the device.

[0101] S2: call the three-phase / single-phase voltage and current source, cable and oscilloscope in the virtual maintenance instrument to connect the lines before the test of "220kV line protection device".

[0102] S3: call the three-phase / single-phase voltage and current source in the virtual maintenance instrument, and apply the corresponding three-phase voltage and current values according to the setting values of "220kV line protection device".

[0103] S4: call the oscilloscope in the virtual maintenance instrument to view the switching action of "220kV line protection device", and make a correctness judgment.

[0104] Step eight: the maintenance personnel complete the test of the protection function logic of the "220kV line protection equipment", the server stores and manages the working steps, forms a standard operation work order evaluation table, uploads the operation data to the equipment database, and records the history, thereby completing the closed-loop management of the secondary virtual maintenance operation.

[0105] The substation equipment simulation maintenance method provided in the embodiment includes: collecting a field image of substation equipment; and sending the field image to a server to enable the server to identify the substation equipment in the field image and obtain actual operation data and a virtual toolbox corresponding to the substation equipment; obtaining the actual operation data and the virtual toolbox sent by the server, and fusing and displaying the actual operation data, the virtual toolbox, and the field image; obtaining a target virtual tool and a maintenance action selected by a maintenance personnel; the target virtual tool is a virtual tool selected by the maintenance personnel from the virtual toolbox; sending the target virtual tool and the maintenance action selected by the maintenance personnel to the server to enable the server to determine simulation data of the substation equipment after the maintenance action is implemented based on the target virtual tool and the actual operation data; receiving the simulation data of the substation equipment after the maintenance action is implemented sent by the server, and fusing and displaying the simulation data and the field image, the simulation data including a three-dimensional model and simulation operation data. The substation equipment simulation maintenance method uses augmented reality technology, enables the maintenance personnel to perform simulation maintenance in combination with the real structure of the equipment at a maintenance site, and can improve the on-site operation and training learning effect of the maintenance personnel.

[0106] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0107] The following is a device embodiment of the application, and for details not described in detail, reference can be made to the corresponding method embodiments described above.

[0108] Figure 3 A structure diagram of a substation equipment simulation maintenance device provided by an embodiment of the application is shown, only parts related to the embodiment of the application are shown for ease of description, and details are described as follows:

[0109] As shown in Figure 3 The substation equipment simulation maintenance device 3 includes:

[0110] A first acquisition module 31 is configured to acquire a field image of substation equipment through an AR terminal;

[0111] The device identification module 32 is configured to identify the substation device in the field image, acquire actual operation data and a virtual toolbox corresponding to the substation device, and send the virtual toolbox to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual toolbox, and the field image.

[0112] The second acquisition module 33 is configured to acquire the target virtual tool and the maintenance action selected by the maintenance personnel, and the target virtual tool is a virtual tool selected by the maintenance personnel from the virtual toolbox.

[0113] The data calculation module 34 is configured to determine simulation data of the substation device after the maintenance action based on the target virtual tool and the actual operation data, and the simulation data includes a three-dimensional model and simulation operation data.

[0114] In some embodiments, the data calculation module 34 is specifically configured to:

[0115] acquire a target site, the target site being a site selected by the maintenance personnel in the field image for simulating maintenance of the substation device;

[0116] search for a three-dimensional model of the target site after the maintenance action in the device database;

[0117] The simulation maintenance device 3 of the substation device further includes:

[0118] The sending module is configured to, after determining the simulation data of the substation device after the maintenance action based on the target virtual tool and the actual operation data, send the three-dimensional model of the target site after the maintenance action to the AR terminal, so that the AR terminal fuses and displays the three-dimensional model of the target site after the maintenance action and the field image.

[0119] In some embodiments, the data calculation module 34 is specifically configured to:

[0120] determine a corresponding operation data change rule based on the substation device, the target virtual tool, and the maintenance action;

[0121] calculate simulation operation data of the substation device after the maintenance action based on the actual operation data and the operation data change rule.

[0122] In some embodiments, the simulation data further includes a simulation maintenance result.

[0123] The data calculation module 34 is specifically configured to:

[0124] The specific data of the target inspection item acquired from the AR terminal is compared with the specific data of the corresponding item in the preset task work order, if consistent, the simulation maintenance result of the target inspection item is correct, if inconsistent, the simulation maintenance result of the target inspection item is incorrect; the target inspection item includes a virtual tool, a maintenance action and an execution sequence.

[0125] In some embodiments, the device identification module 32 is specifically configured to:

[0126] The substation equipment in the field image is identified using a YOLO v3 model.

[0127] The simulation maintenance device for substation equipment provided by the embodiment of the application comprises: a first acquisition module, configured to acquire a field image of substation equipment through an AR terminal; a device identification module, configured to identify the substation equipment in the field image, and acquire actual operation data and a virtual tool box corresponding to the substation equipment; and send the virtual tool box to the AR terminal, so that the AR terminal fuses and displays the actual operation data, the virtual tool box and the field image; a second acquisition module, configured to acquire a target virtual tool sent by the AR terminal and a maintenance action selected by a maintenance personnel, the target virtual tool being a virtual tool selected by the maintenance personnel from the virtual tool box; and a data calculation module, configured to determine simulation data of the substation equipment after implementation of the maintenance action based on the target virtual tool and the actual operation data, the simulation data comprising a three-dimensional model and simulation operation data. The application uses augmented reality technology, so that the maintenance personnel can perform simulation maintenance in combination with the real structure of the equipment at the maintenance site, and the on-site operation and training learning effect of the maintenance personnel can be improved.

[0128] Figure 4 is a schematic diagram of a terminal provided by the embodiment of the application. As shown in Figure 4 , the terminal 4 of the embodiment comprises a processor 40, a memory 41, and a computer program 42 stored in the memory 41 and executable on the processor 40. The processor 40 implements the steps in each of the simulation maintenance method embodiments for substation equipment described above when executing the computer program 42, for example Figure 1 , steps 101 to 104. Alternatively, the processor 40 implements the functions of each module in each of the device embodiments described above when executing the computer program 42, for example Figure 3 , the functions of the modules 31 to 34.

[0129] For example, the computer program 42 can be divided into one or more modules stored in the memory 41 and executed by the processor 40 to complete the present application. The one or more modules can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 42 in the terminal 4. For example, the computer program 42 can be divided into Figure 3 the modules 31 to 34 shown.

[0130] The terminal 4 can be a desktop computer, a notebook computer, a palm computer, a cloud server and other computing devices. The terminal 4 can include, but is not limited to, the processor 40 and the memory 41. Those skilled in the art can understand that the terminal 4 can include more or fewer components than those shown, or combine some components, or include different components, for example, the terminal can also include an input / output device, a network access device, a bus, etc. Figure 4 The terminal 4 shown is only an example and does not constitute a limitation on the terminal 4, which can include more or fewer components than those shown, or combine some components, or include different components, for example, the terminal can also include an input / output device, a network access device, a bus, etc.

[0131] The processor 40 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0132] The memory 41 can be an internal storage unit of the terminal 4, such as a hard disk or a memory of the terminal 4. The memory 41 can also be an external storage device of the terminal 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 41 can include both the internal storage unit and the external storage device of the terminal 4. The memory 41 is used to store the computer program and other programs and data required by the terminal. The memory 41 can also be used to temporarily store data that has been output or will be output.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0134] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0135] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0136] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / terminal and method can be implemented by other ways. For example, the above-mentioned apparatus / terminal embodiments are only schematic, and the division of the modules or units is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0137] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0138] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0139] The integrated module, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer-readable storage medium. When the processor executes the computer program, the steps of the above-mentioned substation equipment simulation maintenance method embodiment can be realized. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0140] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for simulating maintenance of substation equipment, characterized in that, Applied to a server, the method includes: Acquire on-site images of substation equipment using AR terminals; The system identifies substation equipment in the field image and obtains the corresponding actual operating data and virtual toolbox for the substation equipment; it then sends the virtual toolbox to the AR terminal so that the AR terminal can fuse and display the actual operating data, the virtual toolbox, and the field image. The system acquires the target virtual tool sent by the AR terminal and the maintenance action selected by the maintenance personnel. The target virtual tool is the virtual tool selected by the maintenance personnel from the virtual toolbox. The virtual tools in the virtual toolbox include a virtual screwdriver, a virtual cable, and a virtual three-phase / single-phase voltage and current source. The virtual screwdriver completes the disconnection and connection of the substation secondary terminals in the augmented reality environment. The virtual cable completes the short-circuiting and bridging of the substation secondary terminals in the augmented reality environment. The virtual three-phase / single-phase voltage and current source completes the application of the required voltage and current to the secondary equipment in the augmented reality environment, including three-phase / single-phase voltage and current of different values ​​and angles, and can set the continuous output quantity of a certain time step. Based on the target virtual tool and the actual operating data, the simulation data of the substation equipment after the maintenance action is determined. The simulation data includes a three-dimensional model and simulation operating data. The actual operating data includes the voltage, current, temperature, optical port intensity, and alarm data of the substation equipment. The simulated data for determining the substation equipment after implementing the maintenance action based on the target virtual tool and the actual operating data includes: Based on the substation equipment, the target virtual tool, and the maintenance actions, determine the corresponding operational data change rules; Based on the actual operating data and the rules governing changes in operating data, simulated operating data of the substation equipment after the maintenance actions are performed are calculated; wherein, the simulated operating data is used to reflect the operating status of the substation equipment after the maintenance actions are performed. When the target virtual tool is a virtual power source, the simulated operation data of the substation equipment after the maintenance action is performed includes the internal current parameters of the secondary equipment after it is connected to the virtual power source.

2. The simulated maintenance method for substation equipment according to claim 1, characterized in that, The simulated data for determining the substation equipment after implementing the maintenance action based on the target virtual tool and the actual operating data includes: The target location is the part of the substation equipment selected by the maintenance personnel in the field image for simulated maintenance. Search the equipment database for a 3D model of the target part after the maintenance action is performed; After determining the simulated data of the substation equipment after the maintenance operation based on the target virtual tool and the actual operating data, the method further includes: The three-dimensional model of the target part after the maintenance action is performed is sent to the AR terminal, so that the AR terminal can fuse and display the three-dimensional model of the target part after the maintenance action is performed with the on-site image.

3. The simulated maintenance method for substation equipment according to claim 1, characterized in that, The simulation data also includes simulation inspection results; The simulated data for determining the substation equipment after implementing the maintenance action based on the target virtual tool and the actual operating data includes: The specific data of the target inspection item obtained from the AR terminal is compared with the specific data of the corresponding item in the preset task work order. If they match, the simulated inspection result of the target inspection item is correct; if they do not match, the simulated inspection result of the target inspection item is incorrect. The target inspection item includes virtual tools, inspection actions, and execution order.

4. The simulated maintenance method for substation equipment according to any one of claims 1 to 3, characterized in that, The method for identifying substation equipment in the field image includes: The YOLO v3 model was used to identify the substation equipment in the field images.

5. A method for simulating maintenance of substation equipment, characterized in that, Applied to AR terminals, the method includes: Collect on-site images of substation equipment; and send the on-site images to the server so that the server can identify the substation equipment in the on-site images and obtain the actual operating data and virtual toolbox corresponding to the substation equipment; The system acquires the actual operating data and the virtual toolbox sent by the server, and then merges and displays the actual operating data, the virtual toolbox, and the on-site image. The system acquires the target virtual tool and the maintenance action selected by the maintenance personnel. The target virtual tool is the virtual tool selected by the maintenance personnel from the virtual toolbox. The virtual tools in the virtual toolbox include a virtual screwdriver, a virtual cable, and a virtual three-phase / single-phase voltage and current source. The virtual screwdriver completes the disconnection and connection of the substation secondary terminals in the augmented reality environment. The virtual cable completes the short-circuiting and bridging of the substation secondary terminals in the augmented reality environment. The virtual three-phase / single-phase voltage and current source completes the application of the required voltage and current to the secondary equipment in the augmented reality environment, including three-phase / single-phase voltage and current of different values ​​and angles, and can set the continuous output quantity of a certain time step. The target virtual tool and the maintenance action selected by the maintenance personnel are sent to the server, so that the server can determine the simulated data of the substation equipment after the maintenance action is performed based on the target virtual tool and the actual operating data. The system receives simulation data of the substation equipment after the maintenance operation is performed, sent by the server, and fuses and displays the simulation data with the on-site images. The simulation data includes a three-dimensional model and simulation operation data. The simulated operating data of the substation equipment after the maintenance action is performed is calculated based on the actual operating data and the operating data change rules, and the operating data change rules are determined based on the substation equipment, the target virtual tool, and the maintenance action. When the target virtual tool is a virtual power source, the simulated operation data of the substation equipment after the maintenance action is performed includes the internal current parameters of the secondary equipment after it is connected to the virtual power source.

6. The simulated maintenance method for substation equipment as described in claim 5, characterized in that, The process of fusing and displaying the simulated data and the on-site images includes: Identify the area of ​​the substation equipment in the field image and use the area of ​​the substation equipment as a three-dimensional registration area; The three-dimensional model is converted from the world coordinate system to the imaging plane coordinate system of the on-site image, and the converted three-dimensional model is superimposed on the three-dimensional registration area to obtain a simulated maintenance scene. The simulated maintenance scenario is displayed.

7. A simulated maintenance device for substation equipment, characterized in that, include: The first acquisition module is used to acquire on-site images of substation equipment through an AR terminal; The equipment identification module is used to identify substation equipment in the field image, obtain the actual operating data and virtual toolbox corresponding to the substation equipment, and send the virtual toolbox to the AR terminal so that the AR terminal can fuse and display the actual operating data, the virtual toolbox and the field image. The second acquisition module is used to acquire the target virtual tool and the maintenance action selected by the maintenance personnel sent by the AR terminal. The target virtual tool is the virtual tool selected by the maintenance personnel from the virtual toolbox. The virtual tools in the virtual toolbox include a virtual screwdriver, a virtual cable, and a virtual three-phase / single-phase voltage and current source. The virtual screwdriver completes the disconnection and connection of the substation secondary terminals in the augmented reality environment. The virtual cable completes the short circuit and bridging of the substation secondary terminals in the augmented reality environment. The virtual three-phase / single-phase voltage and current source completes the application of the required voltage and current to the secondary equipment in the augmented reality environment, including three-phase / single-phase voltage and current of different values ​​and angles, and can set the continuous output quantity of a certain time step. The data calculation module is used to determine the simulated data of the substation equipment after the maintenance action is performed, based on the target virtual tool and the actual operating data. The simulated data includes a three-dimensional model and simulated operating data. The actual operating data includes the voltage, current, temperature, optical port intensity, and alarm data of the substation equipment. The data calculation module is specifically used for: Based on the substation equipment, the target virtual tool, and the maintenance actions, determine the corresponding operational data change rules; Based on the actual operating data and the rules governing changes in operating data, simulated operating data of the substation equipment after the maintenance actions are performed are calculated; wherein, the simulated operating data is used to reflect the operating status of the substation equipment after the maintenance actions are performed. When the target virtual tool is a virtual power source, the simulated operation data of the substation equipment after the maintenance action is performed includes the internal current parameters of the secondary equipment after it is connected to the virtual power source.

8. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the simulated maintenance method for substation equipment as described in any one of claims 1 to 4.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the simulated maintenance method for substation equipment as described in any one of claims 1 to 4.

Images