Substation interactive operation simulation method and device and electronic equipment

Abstract

The invention discloses a transformer substation interactive operation simulation method and device and electronic equipment, and the method comprises the steps: determining a transformer substation environment simulation model, a transformer substation tool simulation model and a transformer substation equipment simulation model based on an electric power operation simulation demand; obtaining human body structure information corresponding to the target object, and establishing a virtual software robot body model corresponding to the target object according to the human body structure information; and based on the transformer substation environment simulation model, the transformer substation tool simulation model, the transformer substation equipment simulation model and the virtual software robot body model, establishing an interactive operation simulation scene corresponding to the power operation simulation demand, and performing interactive operation simulation according to the interactive operation simulation scene. According to the virtual software robot body model, the transformer substation environment simulation model, the transformer substation tool simulation model and the transformer substation equipment simulation model, an interactive operation simulation scene is established, and the authenticity in the simulation drilling process is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality and augmented reality three-dimensional scene simulation, and in particular to a simulation method, device and electronic equipment for substation interactive operation. Background Art

[0002] With the continuous development of power technology, the power system has become more and more complex, and affected by natural or human factors, various failures and accidents will occur. In order to ensure the stability of the power system and improve the efficiency of operation and maintenance, it is necessary to improve the technical skills of the operation and maintenance personnel in the power system so that the operation and maintenance personnel can quickly deal with emergencies.

[0003] However, the existing training programs for operation and maintenance personnel often simulate power equipment failures, practice operations on drawings or simulation boards, and conduct simulation drills, which lack realism and not only waste human resource costs but also lead to low efficiency in the training of operation and maintenance personnel.

[0004] However, there are two major difficulties in computer simulation modeling for substation simulation training. On the one hand, due to the inefficiency of manual modeling and the lack of sufficient quantity and fineness of 3D modeling, the simulation system cannot be used due to lack of content. On the other hand, the excessive number of models often overburdens the simulation system, which directly affects the user experience and training effect of the training system. These two major problems are traditionally solved by two means: one is to strengthen the hardware, but for platforms with low hardware performance requirements, such as embedded devices and mobile devices, the application of 3D simulation is limited; the other is to use fixed-view 3D simulation, that is, fake 3D method, but this limits the expressiveness of 3D simulation.

[0005] In addition, although substation simulation technology has developed to a relatively mature level, currently one person completes all projects on the simulation system, which does not conform to the actual situation of substation production. In actual substation production, there are different roles such as duty manager, duty officer, deputy duty officer, etc. It can also be divided into different roles such as operator, guardian, work supervisor, work permit holder, work issuer, etc. Each role has different job responsibilities.

[0006] Therefore, the use of advanced 3D virtual reality rapid modeling and model optimization technology can not only realize the 3D efficient modeling and 3D visualization of substations, but also realize the digital transformation from "single person single device" to "multiple people multiple devices". As the basis of the 3D simulation system, it is urgent to study the modeling and control technology of advanced virtual software robots that realistically simulate the interactive operation of substation operation and maintenance personnel based on the 3D characteristics of typical electrical equipment, tools and environment in the substation field.

[0007] However, there are technical bottlenecks in the rapid 3D modeling and model optimization technology for typical electrical equipment, tools and basic working environments in the substation field. Due to the large number of equipment and complex models in the substation, it is a difficult problem to accurately and efficiently convert the entities of existing equipment into data files used in virtual reality in a short period of time. At the same time, it is also a major problem to establish logical interactive relationships between virtual electrical equipment according to different working conditions. Summary of the invention

[0008] The present invention provides a simulation method, device and electronic equipment for substation interactive operation. By constructing a virtual software robot body model corresponding to the target object, an interactive operation simulation scene is established according to the virtual software robot body model and the substation environment simulation model, the substation tool simulation model and the substation equipment simulation model, thereby improving the realism of the simulation exercise process and thus improving the training efficiency of operation and maintenance personnel.

[0009] According to one aspect of the present invention, there is provided a method for simulating substation interactive operation, comprising:

[0010] Determine a substation environment simulation model, a substation tool simulation model and a substation equipment simulation model based on power operation simulation requirements, wherein the environment simulation model, the tool simulation model and the equipment simulation model are pre-established virtual three-dimensional models corresponding to the substation;

[0011] Acquire human body structure information corresponding to the target object, and establish a virtual software robot body model corresponding to the target object according to the human body structure information;

[0012] An interactive operation simulation scenario corresponding to the power operation simulation requirement is established based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and an interactive operation simulation is performed according to the interactive operation simulation scenario.

[0013] According to another aspect of the present invention, there is provided a simulation device for substation interactive operation, comprising:

[0014] A model acquisition module, used to determine a substation environment simulation model, a substation tool simulation model and a substation equipment simulation model based on power operation simulation requirements, wherein the environment simulation model, the tool simulation model and the equipment simulation model are pre-established virtual three-dimensional models corresponding to the substation;

[0015] A virtual software robot construction module is used to obtain human body structure information corresponding to the target object, and to establish a virtual software robot body model corresponding to the target object according to the human body structure information;

[0016] An operation simulation module is used to establish an interactive operation simulation scenario corresponding to the power operation simulation requirements based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and perform interactive operation simulation according to the interactive operation simulation scenario.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program can be executed by the at least one processor so that the at least one processor can execute the simulation method of substation interaction operation described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for simulating substation interaction operations described in any embodiment of the present invention when executed.

[0022] The technical solution of the embodiment of the present invention determines the substation environment simulation model, the substation tool simulation model and the substation equipment simulation model based on the power operation simulation requirements, and obtains the human body structure information corresponding to the target object, establishes a virtual software robot body model corresponding to the target object according to the human body structure information, and then establishes an interactive operation simulation scene corresponding to the power operation simulation requirements based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and performs interactive operation simulation according to the interactive operation simulation scene. Based on the above technical solution, by constructing a virtual software robot body model corresponding to the target object, and then establishing an interactive operation simulation scene based on the virtual software robot body model and the substation environment simulation model, the substation tool simulation model and the substation equipment simulation model, the authenticity and realism of the simulation exercise process are improved, thereby improving the training efficiency of operation and maintenance personnel.

[0023] The present invention is based on a fully digital substation simulation training system, combines virtual reality technology and augmented reality technology, studies and applies interactive devices such as Kinect somatosensory technology and stereo projection, and constructs a substation simulation training system with a high sense of reality for operation and maintenance personnel in terms of vision, hearing and interaction. It realizes modeling and simulation of substation electrical equipment, tools and other auxiliary equipment, and environmental scenes, and performs switching operations, accident handling, and daily patrol interactive operations, thereby improving the sense of reality of simulation training and the flexibility of operation.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 It is a flow chart of a simulation method for substation interactive operation provided by an embodiment of the present invention;

[0027] Figure 2 is a driving flow chart of a virtual software robot body model provided by an embodiment of the present invention;

[0028] Figure 3 is a virtual software robot interaction flow chart based on Kinect motion capture provided by an embodiment of the present invention;

[0029] Figure 4 It is a flow chart of a simulation method of substation interactive operation provided by an embodiment of the present invention;

[0030] Figure 5 It is a structural block diagram of a simulation device for substation interactive operation provided by an embodiment of the present invention;

[0031] Figure 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0032] Figure 7 2 is another structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 This is a flow chart of a simulation method for interactive operation of a substation provided by an embodiment of the present invention. This embodiment can be applied to establish a corresponding interactive operation simulation scenario according to the power operation simulation requirements, and then perform simulation and operation and maintenance training according to the interactive operation of the target object. The method can be executed by a simulation device for interactive operation of a substation, which can be implemented in the form of hardware and / or software. The simulation device for interactive operation of a substation can be configured in an electronic device, which can be a terminal device or a server, etc. Figure 1 As shown, the method includes:

[0037] S110, determining a substation environment simulation model, a substation tool simulation model, and a substation equipment simulation model based on power operation simulation requirements.

[0038] The power operation simulation requirement may be a simulation requirement input by a trainer or operation and maintenance personnel according to the actual operation requirements on site, and may include information such as simulation operation, simulation project, target substation, etc. The substation environment simulation model may be understood as a pre-established three-dimensional model corresponding to the substation environment, and may include a substation three-dimensional scene simulation model and a substation three-dimensional meteorological simulation model. The substation three-dimensional scene simulation model includes a three-dimensional scene simulation model of the substation under normal operating conditions and a three-dimensional scene simulation model of the substation when an accident occurs; the substation three-dimensional meteorological simulation model includes the establishment of a model library under different meteorological conditions such as sunny, foggy, rainy, typhoon, and lightning, as well as a dynamic model of the transmission line corresponding to the meteorological conditions under different meteorological conditions of the substation. Further, on the basis of the above models, realistic simulations can be performed according to the equipment operation conditions, equipment failure conditions, emergency measures, etc. of the substation under different meteorological conditions (such as sunny, foggy, rainy, typhoon, lightning, etc.). It should be noted that in terms of the impact of severe weather on the substation model library, the impact of severe weather on the connecting wire model in the substation can be automatically simulated according to wind force and wind deflection, the dancing of the connecting wires in the substation can be simulated, and the effect of the connecting wires in the substation in severe weather environments can be vividly displayed.

[0039] For example, the substation three-dimensional scene simulation model realizes the three-dimensional reproduction and virtual operation of the substation electrical equipment on the basis of three-dimensional modeling, vividly reflects the normal, abnormal, accident status and action process of the substation electrical equipment, and can not only realize the equipment inspection, observation, inspection, equipment rotation roaming motion mode, photography, infrared temperature measurement, and defect reporting in the virtual three-dimensional scene, but also perform virtual operations. Establishing a three-dimensional scene simulation model of the appearance structure, operation function and action process of the virtual electrical equipment can also realize the built-in electrical logic of the substation virtual electrical equipment and respond correctly to the operation, so that the operating personnel can correctly and standardizedly make on-site judgments and operate the substation virtual electrical equipment under different initial conditions and different working conditions.

[0040] Furthermore, the substation three-dimensional scene simulation model can realize the simulation process of the geometric characteristics, behavioral characteristics and interactive characteristics of the virtual electrical equipment of the substation realistically simulating the real electrical equipment. Specifically, the substation three-dimensional scene simulation model can realize that the action behavior, sound, lighting, meter and other phenomena of the substation virtual electrical equipment operation response process in normal operation and accident conditions are consistent with the actual situation. The appearance display mode and operation mode of the display interface and the operating mechanism are consistent with the actual substation virtual electrical equipment. The action characteristics, reset characteristics and time characteristics of the relay protection device, automatic device, measurement system and automatic standby power supply device are consistent with the actual device. In normal, abnormal or accident conditions, the display of the device tag number, tag and meter is consistent with the site. In addition to being able to simulate the substation three-dimensional scene under normal operation, the substation three-dimensional scene simulation model can also simulate the substation three-dimensional accident scenes under various accident conditions, which improves the authenticity and on-site feeling of the model.

[0041] In addition, the substation 3D scene simulation model can be used for training on various equipment such as switches, switches, transformers, busbars, lines, capacitors and their related accident handling. It can train substation operators to discover accidents and anomalies, judge faults and handle accidents based on the power grid environment of the substation 3D scene simulation model. After the training, the accidents are analyzed and the whole process of accident occurrence and handling is played in a loop. Through simulation training of the substation 3D scene simulation model, operators can understand the phenomena, causes and changes of various accidents, summarize and accumulate accident handling experience, and enhance the confidence of operators in handling accidents.

[0042] For example, the three-dimensional meteorological simulation model of the substation needs to meet the environmental simulation under extreme complex environmental conditions, which can solve the quantitative simulation of various extreme complex climate environments in the existing training environment. In the training environment, the weather conditions such as wind speed, wind direction, whether it is raining, rainfall, and lightning coverage can be accurately defined. In a certain training environment, multiple body senses such as smell, temperature, humidity, and sound can also be added to enhance the training effect.

[0043] On the basis of the above technical solution, the dynamic model of the transmission line corresponding to the meteorological conditions under different meteorological conditions of the substation includes: obtaining the static equally divided coordinates of the transmission line in the station corresponding to the substation, and determining the dynamic equally divided coordinates corresponding to the static equally divided coordinates according to the wind deviation angle; determining the hanging points and equally divided nodes of the transmission line in the station based on the dynamic equally divided coordinates, and connecting the hanging points and the equally divided nodes to determine the dynamic model of the transmission line.

[0044] Among them, the in-station transmission line can be understood as the transmission line set inside the substation. The static equal-division coordinates can be the equal-division coordinates of the transmission line without the influence of bad weather, and the equal-division coordinates can be determined according to the pre-set equal-division distance. The wind deflection angle refers to the angle formed by the transmission line with its vertical position when it deviates from its vertical position due to the action of wind. It should be noted that the wind deflection angle is formed when the transmission line deviates from its original vertical position due to the action of wind.

[0045] Specifically, calculating and monitoring the wind deflection angle can help prevent the occurrence of flashover accidents in transmission lines and ensure the safe operation of the lines. For example, by calculating the wind deflection angle, the type and number of insulators, the air gap between the conductor and the tower frame, etc. can be determined to ensure the safe operation of the line under various meteorological conditions. The methods for calculating the wind deflection angle include the rigid straight rod method and the finite element simulation method. The rigid straight rod method takes into account the rigidity and wind force of the conductor, while the finite element simulation method can more accurately consider the effects of different conductor models, initial stress, span and height difference on the wind deflection angle. In addition, meteorological factors such as wind speed and wind direction will also significantly affect the size of the wind deflection angle, and then the possible wind deflection angle can be simulated according to different meteorological factors, and then the suspension points and equal-division nodes of the transmission line in the station can be calculated according to the wind deflection angle, and the suspension points and the equal-division nodes are connected to determine the dynamic model of the transmission line.

[0046] It is understandable that the substation tool simulation model can be a pre-established three-dimensional model corresponding to the substation tools. The substation tool simulation model includes a safety tool simulation model, a protective equipment simulation model and an instrument simulation model. For example, the safety tool simulation model mainly simulates various tools used to ensure the safety of personnel and equipment in the substation, such as insulating rods, electroscopes, grounding wires, etc. The substation tool simulation model is not only highly similar to the real object in appearance, but also accurately simulates its functions and characteristics. For example, the insulating rod model can simulate its insulation performance in live work, and the electroscope model can simulate its function of detecting voltage. The protective equipment simulation model mainly simulates the equipment used to protect workers from harm in the substation, such as safety helmets, protective glasses, protective gloves, protective clothing, etc. The protective equipment simulation model also makes detailed simulations in appearance, material and wearing method, aiming to help operators better understand the use and importance of protective equipment. For example, the protective clothing model can simulate its role in resisting dangerous environments such as chemicals, high temperature or electric shock. The instrumentation simulation model mainly simulates the equipment used to monitor and control various power parameters in the substation, such as voltmeters, ammeters, power meters, oscilloscopes, etc. The instrumentation simulation model is not only consistent with the real object in appearance, but also can simulate its function of displaying and recording power parameters. By operating the model, operators can become familiar with the use and reading skills of instruments and improve their accuracy and efficiency in actual work.

[0047] Specifically, in order to facilitate the completion of various control instructions in the virtual three-dimensional scene, you can use the left mouse button to click the "Panoramic Navigation" button on the tool bar of the substation tool simulation model to call the navigation map. In this mode, you can use the left mouse button to click the up, down, left, and right directions of the navigation map to drag it to achieve panoramic movement in the navigation map, and keep the left mouse button pressed. Double-click a device on the navigation map with the left mouse button, and the patrol observation point will automatically jump to the patrol observation point just operated. For example, during the user's simulation process, you can use the mouse to click the "Safety Tool Box" button on the tool bar in the virtual three-dimensional scene of the safety tool simulation model to enter the safety tool selection scene, select the required safety tool, and then click the confirmation button. The safety tool simulation model can include tool models such as safety helmets, safety belts, portable short-circuit grounding wires, personal safety wires, capacitive testers, insulating rods, insulating gloves, insulating boots, insulating rubber pads, foot pedals, foot buckles, insulating ladders, etc., and can also include safety warning sign models such as "No Closing, Someone is Working"; "No Closing, Someone is Working on the Line"; "Working Here"; "Enter and Exit from Here"; "Stop, High Voltage Danger"; "No Climbing" signs. Specifically, use the mouse to click the "Safety Tool Box" button on the tool bar in the virtual three-dimensional scene of the safety tool simulation model to enter the safety tool model selection scene, and then use the left mouse button to double-click the type and quantity of the safety tool model you need to collect and use in the virtual three-dimensional scene of the safety tool simulation model. After double-clicking, a dialog box for selecting the safety tool model to collect and use will appear. Select the safety tool model you need to collect and use, and click the confirmation button. Specifically, use the mouse to click the "Hanging Method" button on the tool bar in the virtual three-dimensional scene of the safety tool simulation model to enter the safety warning sign model selection scene, and then use the left mouse button to double-click the location where the sign model needs to be hung in the virtual three-dimensional scene of the safety warning sign model. After double-clicking, a dialog box for selecting the hanging sign model will appear. Select the sign model to be hung and click the confirmation button.

[0048] The protective equipment simulation model can include personal safety protective equipment. Personal safety protective equipment is issued to individuals on demand. The models of protective equipment such as safety helmets, arc protection clothing, protective glasses, protective shoes, anti-smashing and anti-puncture non-slip shoes issued to individuals can be kept by individuals. Other special protective equipment models issued to individuals are centrally stored and fixedly managed with the team as the smallest unit. Use the mouse to click the "Protective Equipment Box" button on the tool bar in the virtual 3D scene of the protective equipment simulation model to enter the protective equipment model selection scene, select the required protective equipment model, and then click the confirmation button.

[0049] Instrument simulation models can include: thermometer, camera, drone, multimeter, megohmmeter, AC (or DC) voltmeter, ammeter, clamp ammeter, active power meter, active energy meter, watt-hour meter, temperature meter and other instrument models. Use the mouse to click the "Instrument Library" button on the tool bar in the virtual three-dimensional scene of the instrument simulation model to enter the instrument model selection scene, and then use the left mouse button to double-click the type and quantity of the instrument model you want to collect and use in the virtual three-dimensional scene of the instrument simulation model. After double-clicking, a dialog box for selecting the instrument model to collect and use will appear. Select the instrument model you want to collect and use, and click the confirmation button.

[0050] The technical solution provided by the embodiment of the present invention can be used as a training tool through the substation tool simulation model to help operators become familiar with the use and precautions of various tools and improve their practical operation capabilities. In addition, safety drills are carried out through the simulation model to test the operators' response capabilities and decision-making levels in emergency situations, thereby improving the safety level of the entire substation. In the design and optimization process of the substation, the simulation model can be used as an auxiliary tool to help engineers better understand the layout and functional requirements of the equipment, thereby improving the rationality and practicality of the design.

[0051] The substation equipment simulation model can be a pre-established three-dimensional model corresponding to the substation equipment, and the substation equipment simulation model includes a substation primary equipment simulation model and a substation secondary equipment simulation model. The substation equipment simulation model can show the internal structure, appearance, key parameters and other details of the substation equipment, realize the three-dimensional reproduction of the primary and secondary equipment of the substation, and vividly reflect the normal, abnormal, and accident status of the equipment and its action process. It can not only patrol, observe, inspect, rotate and roam around the equipment in the virtual scene, take pictures, measure infrared temperature, and report defects, but also perform virtual operations. The substation environment simulation model, the substation tool simulation model and the substation equipment simulation model are pre-established virtual three-dimensional models corresponding to the substation. For example, through the three-dimensional reproduction technology, the substation equipment simulation model can accurately display the primary and secondary equipment of the substation, including not only large equipment such as transformers and switch cabinets, but also various small components such as sensors and relays, and each device in the model is modeled according to the actual size and proportion, ensuring a high degree of authenticity and accuracy. On this basis, the simulation model can vividly reflect the normal, abnormal and accident status of the equipment and its action process. Under normal conditions, operators can observe the stable operation of the equipment and real-time monitoring of various parameters. Under abnormal or accident conditions, the model can simulate the failure performance and action process of the equipment, helping operators to better understand and respond to various emergencies. In addition to the static display of the equipment, the substation equipment simulation model also supports a variety of interactive functions. Operators can patrol, observe, and inspect the equipment in the virtual scene, and can fully understand the structure and details of the equipment through the equipment's rotation and roaming motion mode. In addition, the model also supports functions such as taking pictures and infrared temperature measurement, which can easily record the status of the equipment and analyze it. The substation equipment simulation model also supports virtual operation functions. Operators can perform virtual operations on the equipment in the model, such as turning on or off switches, adjusting equipment parameters, etc. Virtual operations can not only help operators become familiar with the equipment's operating procedures and precautions, but also conduct simulation drills before actual operations to improve the accuracy and safety of operations.

[0052] Specifically, after obtaining the power operation simulation requirements of the training personnel or the operation and maintenance personnel, the corresponding substation environment simulation model, substation tool simulation model and substation equipment simulation model can be determined according to the power operation simulation requirements. For example, the substation environment simulation model, substation tool simulation model and substation equipment simulation model corresponding to the substation can be pre-established and stored in the database. When the power operation simulation requirements are detected, the corresponding substation environment simulation model, substation tool simulation model and substation equipment simulation model are queried from the database according to the requirements. For example, the target substation can be extracted from the power operation simulation requirements, and the model corresponding to the target substation can be obtained, and then the corresponding substation environment simulation model, substation tool simulation model and substation equipment simulation model can be called according to the simulation operation and simulation project.

[0053] It should be noted that the primary equipment model includes: main transformer, busbar, circuit breaker, disconnector (including grounding switch), current transformer, voltage transformer, high-voltage fuse, lightning arrester, wave arrester, combined filter, capacitor, reactor, incoming and outgoing lines, etc.; the secondary equipment model includes: various protection device LCD panels, switching handles, air switches, pressure plates, buttons, signal lights, light signs, protection screens, etc.

[0054] Specifically, the primary equipment model can include a microcomputer five-defense system simulation model and a primary equipment action process model. In the microcomputer five-defense system simulation model, the operation of the operation and maintenance personnel can be simulated by the microcomputer five-defense system, and the operation that complies with the five-defense rules is regarded as a valid operation. In the primary equipment action process model, after the operation and maintenance personnel operate in the simulation system interface or the equipment fails, the simulation support program performs an anti-error logic judgment and changes the state of the primary equipment model such as switches and knife switches in the three-dimensional scene.

[0055] Exemplarily, the secondary equipment model can truly reflect the various operating controls and specific locations of the secondary equipment. The operation and maintenance personnel can open the back door of the screen panel to check the connection status of the terminals. Combined with the secondary circuit simulation, the operation and maintenance personnel can even use virtual instruments to measure and check the connection status of the terminal block to meet the simulation training function of secondary line checking. Further, taking the operation of the operating mechanism of the switch and knife switch (including the terminal box and mechanism box) as an example, you can use the keyboard or mouse to control a certain operating mechanism body or door, and the secondary equipment model will automatically face the operating mechanism and enter the close-range operation mode. You can double-click the operating mechanism door with the left mouse button, and the secondary equipment model will automatically open the operating mechanism door. For each active control in the operating mechanism, you can double-click the active control to be operated with the left mouse button, and the secondary equipment model will pop up the corresponding dialog box, and you can choose to operate it.

[0056] In addition, the substation equipment simulation model can also be divided into spatial areas to construct a virtual three-dimensional model of the substation equipment based on the equipment working position, which can include the main transformer space area, 500kV space area, 220kV space area, 110kV space area, 10kV space area, main control room space area, relay protection room space area, and battery room space area, a total of 8 high-voltage places to realize full-position and full-space area construction of all sub-component virtual three-dimensional models in the substation equipment simulation model.

[0057] S120: Acquire human body structure information corresponding to the target object, and establish a virtual software robot body model corresponding to the target object according to the human body structure information.

[0058] The target object may be an operation and maintenance personnel or a training personnel who is performing interactive action simulation, and the target object includes any at least one of a work supervisor, a work permit holder, a work issuer, a guardian, and an operator.

[0059] The virtual three-dimensional model can realize multi-person collaborative work of multiple target objects; multiple target objects can join the training in the virtual three-dimensional model with multiple roles, supporting the interactive and collaborative completion of a task by different target objects. The virtual three-dimensional model has strong real-time interactive operation, convenient and friendly human-computer interaction and operation mode, and supports at least two target objects to interact and work collaboratively at the same time.

[0060] The human body structure information corresponding to the target object can be understood as the bone structure information and the mapping information of the virtual model, for example, it can include the bone layer, skin layer and outer layer, wherein the bone layer is used to control the posture and movement process of the virtual software robot's limbs, the skin layer uses a dynamic surface mapping method to represent the human body surface skin wrapped outside the bone layer, and the outer layer represents the clothing on the body of the virtual software robot. The three layers have an associated relationship with the movement layer. When the human bone layer moves, the skin layer and outer layer attached thereto will undergo surface deformation. The virtual software robot body model can be a virtual software model corresponding to the target object established by three-dimensional modeling software based on the human body structure information.

[0061] Specifically, it can be to use a 3D scanner to obtain the surface geometry data of the target object, and then clean, denoise, align and mesh the point cloud data obtained by the 3D scanner to generate a preliminary three-dimensional model, and use software tools or algorithms to segment the three-dimensional model into different body parts, such as the head, torso, limbs, etc., and then obtain the human body structure information corresponding to the target object. In the 3D modeling software, create an accurate polygonal mesh model based on the processed data, add a skeleton system to the model, and use skinning technology to bind the mesh to the skeleton, ensure that the mesh can deform naturally when the skeleton moves through weight drawing, and extract texture information from the scanned point cloud data or photos, and apply it to the model surface to increase authenticity and realism. Exemplary, it can be to use 3DSMAX to model the virtual software robot body model supported by Kinect, and build a virtual software robot body model in 3DSMAX according to the human skeleton structure in Kinect.

[0062] On the basis of the above technical solution, the obtaining of human body structure information corresponding to the target object includes: collecting multiple high-scale three-dimensional depth images corresponding to the target object based on a somatosensory camera, and determining the human body structure information corresponding to the target object according to the multiple high-scale three-dimensional depth images.

[0063] Among them, the somatosensory camera can be a device for collecting human body structure information, for example, it can be a Kinect somatosensory interaction device. The multiple high-proportion three-dimensional depth image can be an image containing a depth value. It should be noted that in the multiple high-proportion three-dimensional depth image, each pixel point has a conventional RGB (red, green, blue) color value and a depth value (Depth Value) attached, which indicates the distance of the position of the pixel point in the three-dimensional space relative to the observer. The human body structure information includes the human body bone structure, skin structure and outer surface structure.

[0064] Specifically, the somatosensory camera can be installed in a suitable position to ensure that it can clearly capture the target object, the tools required for use, and the surrounding environment, and then the camera parameters such as resolution, frame rate, and depth perception range can be configured according to needs. Then, the somatosensory camera can be started and start collecting multiple high-ratio three-dimensional depth images of the target object. The feature recognition algorithm is applied to the virtual three-dimensional model to identify the key parts of the human body, and then the human body structure information corresponding to the target object is obtained.

[0065] On the basis of the above technical solution, the method of collecting multiple high-scale three-dimensional depth images corresponding to the target object based on the somatosensory camera includes: when it is detected that the target object is located in the measurement area of ​​the somatosensory camera, controlling the somatosensory camera to project coded infrared rays to the measurement area; collecting original images corresponding to the measurement area, and determining the multiple high-scale three-dimensional depth images according to the original images and the coded infrared rays; establishing a massive database of multiple high-scale three-dimensional depth images based on the multiple high-scale three-dimensional depth images, so as to losslessly compress, store, restore, reconstruct, and display the multiple high-scale three-dimensional depth images based on the massive database of multiple high-scale three-dimensional depth images.

[0066] The measurement area may be a pre-set sensing area corresponding to the body sensing camera. The coded infrared ray may be understood as an infrared ray of a specific structure, and the projected coded infrared ray may have a specific shape, for example, it may be distributed in the measurement area in a rectangular shape. The original image may be an image containing infrared spots collected after the infrared ray is projected.

[0067] It should be noted that a massive database of multiple high-scale three-dimensional depth images is established based on multiple high-scale three-dimensional depth images. The amount of data in the massive database of multiple high-scale three-dimensional depth images is redundant and huge. Therefore, the redundant and huge amount of data generated in the process of constructing the virtual three-dimensional scene model of the substation is reduced by losslessly compressing and storing and restoring and reconstructing and displaying the multiple high-scale three-dimensional depth images based on the massive database of multiple high-scale three-dimensional depth images.

[0068] Specifically, by traversing all sub-component simulation model data in the substation three-dimensional simulation model, all sub-component simulation models in the substation three-dimensional simulation model are accurately classified and matched through a model feature matching method, a reference sub-component simulation model is extracted from each type of sub-component simulation model set, and then, based on the extracted reference sub-component simulation model, the spatial transformation parameters such as the translation size, rotation angle and scaling ratio of all sub-component simulation models in the sub-component simulation model set relative to the reference sub-component sub-model are calculated based on computer graphics 3D conversion technology. Finally, an index is established for each type of sub-component simulation model set in the traversed, filtered and filtered substation three-dimensional simulation model, and each sub-component simulation model file in each type of sub-component simulation model set is encoded according to the reference sub-component simulation model and the calculated spatial transformation parameters, and compressed and stored according to the index coding, so as to realize index coding lossless compression storage of massive data of multiple high-scale three-dimensional depth images of the substation three-dimensional simulation model, and then restore, reconstruct and display the multiple high-scale three-dimensional depth images of the substation simulation model.

[0069] For example, taking the substation equipment simulation model as an example, the three-dimensional simulation model of the substation equipment is comprehensively and carefully traversed, and each sub-component simulation model data is examined and processed one by one. In order to ensure the accuracy of classification, an advanced model feature matching method is adopted, which can accurately capture and identify the subtle differences between different sub-component simulation models, so as to achieve accurate classification and matching of all sub-component simulation models in the three-dimensional simulation model of the substation equipment. Then, after the classification and matching is completed, a representative benchmark sub-component simulation model is carefully selected from each type of sub-component simulation model set. Based on the benchmark sub-component simulation model, the high-precision 3D conversion technology in computer graphics is used to perform detailed spatial transformation parameter calculations on each simulation model in the same sub-component simulation model set, including but not limited to the translation size, rotation angle and scaling ratio between models, which together constitute the key information describing the relative position and morphological changes between simulation models. In addition, in order to further improve the efficiency of data storage and retrieval, each type of sub-component simulation model set in the traversed and filtered three-dimensional simulation model of the substation equipment is indexed and established. It ensures that each simulation model can be quickly located and accessed, which greatly facilitates subsequent data processing and analysis. At the same time, according to the benchmark sub-component simulation model and the calculated spatial transformation parameters, each sub-component simulation model file in each type of sub-component simulation model set is uniquely encoded, which not only retains all the key information of the simulation model, but also realizes lossless compression storage of massive data, greatly saving storage space. When it is necessary to restore and display multiple high-scale three-dimensional depth images of the substation equipment simulation model, it can be easily achieved through the decoding process. This process not only ensures the high fidelity and clarity of the image, but also makes the entire simulation model more vivid and realistic.

[0070] Exemplarily, model feature matching and precise classification can be performed on all sub-component simulation models in the three-dimensional simulation model of the substation equipment from the spatial area to establish all sub-component simulation models in the three-dimensional simulation model of the substation equipment. For a total of 8 high-voltage places, including the main transformer space area, 500kV space area, 220kV space area, 110kV space area, 10kV space area, main control room space area, relay protection room space area, and battery room space area, all sub-component simulation models in the three-dimensional simulation model of the substation equipment can be established in all positions and space areas. An index is established for each type of sub-component simulation model collection, and each sub-component simulation model file is encoded according to the benchmark sub-component simulation model and the spatial transformation parameters, 1 (index) -1 (encoding); 1 (index) -2 (encoding); 1 (index) -3 (encoding); 1 (index) -4 (encoding), and digital index coding is used to realize the compression and storage of multiple high-proportion three-dimensional depth image data of the substation equipment simulation model.

[0071] Specifically, Kinect uses the infrared camera in the device to capture human movements in real time to interact with the virtual three-dimensional scene of the substation. The Kinect body-sensing camera evenly disperses the coded infrared rays in the measurement space through the grating in front of the lens, and the infrared camera records each speckle with spatial markers. The raw data is calculated by the built-in chip to generate image data with 3D depth information. When the human body moves in the space, the change in displacement can be fully recorded. The 3D image data converted from the depth information is used to convert into human skeleton data.

[0072] On the basis of the above technical solution, a virtual software robot body model corresponding to the target object is established according to the human body structure information, including: determining the human skeletal structure of the target object based on the human body structure information, and determining a matching result between the human skeletal structure and a preset human body model; determining a target human body model based on the matching result, and establishing the virtual software robot body model according to the target human body model and the human body structure information.

[0073] The human skeleton structure may be a skeleton structure corresponding to the target object obtained by scanning with a somatosensory camera. The preset human body model may be understood as a plurality of pre-set original skeleton models. The target human body model may be a human body model selected from the preset human body models according to the matching result.

[0074] Specifically, Figure 2 As shown in the figure, based on the skeletal features of the human body, multiple sets of motion data are recorded to track the movements of the human torso, limbs and fingers. Kinect compares the captured image with the human body model inside itself. If the image is consistent with the internal human body model, a corresponding skeletal model is established, which is then used to drive a virtual software robot model in a three-dimensional scene. The model recognizes key parts of the human skeleton for action triggering, and can realize real-time acquisition and display of human movement movements.

[0075] Further, such as Figure 3 As shown, by acquiring Kinect's somatosensory interactive motion data in real time, the virtual software robot is driven to interact with the virtual three-dimensional scene through sample recognition, and the somatosensory interaction between the operator and the virtual three-dimensional scene is realized based on the precise collision detection between the virtual software robot and the operable three-dimensional objects in the virtual substation and combined with gesture judgment. That is, by analyzing the action data of the virtual software robot such as movement, jumping, and gestures, and then matching the actual operation actions of the operator according to the action data, and then controlling the operation actions of the virtual software robot according to the actual operation actions, the simulation of interactive operations can be realized.

[0076] The technical solution provided by the embodiment of the present invention can dynamically capture the user's somatosensory interactive motion data through the Kinect sensor, and map it to the virtual three-dimensional scene to drive the virtual software robot to interact with the virtual three-dimensional scene. The model realizes the somatosensory interaction between the trainee and the virtual three-dimensional scene through precise collision detection between the virtual software robot body and the operable three-dimensional objects in the virtual substation and combined with gesture judgment.

[0077] S130. Establish an interactive operation simulation scenario corresponding to the power operation simulation requirement based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and perform interactive operation simulation according to the interactive operation simulation scenario.

[0078] The interactive operation simulation scenario may be a simulation scenario ultimately established to simulate power operation simulation requirements.

[0079] Specifically, first, the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model are integrated into a unified virtual environment, and then the initial state in the scene is set according to the specific needs of the power operation simulation. For example, the equipment can be simulated in normal operation, fault state or maintenance state, and an interface for interacting with the virtual environment can be provided for the virtual software robot body model. Input devices such as keyboards, mice, buttons, handles, controls, and virtual reality (VR) or augmented reality (AR) devices can be included so that the target object can interact with the virtual environment more naturally, and then a series of specific operation tasks are defined according to the goals of the power operation simulation, and the defined operation tasks are assigned to the virtual software robot body model. During the simulation process, the virtual software robot body model will perform corresponding operations according to the task requirements, and the target object controls the virtual software robot body model through the input device to perform interactive operations in the virtual environment.

[0080] The technical solution of the embodiment of the present invention determines the substation environment simulation model, the substation tool simulation model and the substation equipment simulation model based on the power operation simulation requirements, and obtains the human body structure information corresponding to the target object, establishes a virtual software robot body model corresponding to the target object according to the human body structure information, and then establishes an interactive operation simulation scene corresponding to the power operation simulation requirements based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and performs interactive operation simulation according to the interactive operation simulation scene. Based on the above technical solution, by constructing a virtual software robot body model corresponding to the target object, and then establishing an interactive operation simulation scene based on the virtual software robot body model and the substation environment simulation model, the substation tool simulation model and the substation equipment simulation model, the authenticity and realism of the simulation exercise process are improved, thereby improving the training efficiency of operation and maintenance personnel.

[0081] Embodiment 2

[0082] Figure 4 The flowchart of a method for simulating interactive operation of a substation provided by an embodiment of the present invention further refines the technical solution of simulating interactive operation according to the interactive operation simulation scenario on the basis of the above embodiment. The specific implementation method can refer to the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiment are not repeated here. Figure 4 As shown, the method includes:

[0083] S210: Acquire an interaction operation with the target object, and determine an interaction control instruction according to the interaction operation.

[0084] The interactive operation can be understood as the operation information sent by the target object through the interactive device. The interactive control instruction can be understood as the instruction information used to control the virtual software robot body model in the interactive operation simulation scene.

[0085] Specifically, the interactive operations of the target object can be collected, for example, the interactive actions of the target object can be collected through a somatosensory camera device, and the corresponding interactive control instructions can be determined according to the interactive actions of the target object, such as walking forward, walking backward, turning left, turning right, stepping, moving forward, moving backward, raising the head, lowering the head, turning the head left, turning the head right, standing, squatting and other interactive operations; the interactive actions of virtual operators operating power equipment and wearing daily protective equipment can be simulated; various hand movements of virtual operators during operation can be simulated, and trainers can drive the virtual software robot in the scene and the electrical objects in the virtual scene to interact in real time.

[0086] S220: Determine a target screen based on the interactive control instruction and the interactive operation simulation scenario, and output the target screen to a display device.

[0087] The target screen can be understood as an animation corresponding to the interactive control instruction. The display device can be a device for playing the target screen, and the display device can include a large 3D arc screen, 3D glasses, an electronic touch sandbox, a somatosensory interactive system, a tablet computer, a workstation, etc.

[0088] Specifically, the received interactive control instructions are parsed to determine the operator's specific intentions and the operations to be performed. After parsing the interactive control instructions, the model needs to evaluate the current interactive operation simulation scene. This includes checking the equipment status, environmental conditions, tool usage, position and posture of the virtual software robot model in the scene, and finally determining the content of the target screen based on the parsed interactive control instructions and the evaluated interactive operation simulation scene, and outputting the determined target screen to the display device.

[0089] It should be noted that the animation corresponding to each instruction can also be pre-set, and then after receiving the interactive control instruction, the corresponding animation is queried from the database according to the interactive control instruction to determine the target picture, and there is no need to actually render the picture, which reduces the performance requirements for the simulation equipment. Under the premise of ensuring that the visual effect of the physical model is not distorted, the target picture is modeled as much as possible using a model with a small number of patches and texture mapping, and the texture baking technology is used to reduce the CPU calculation time required to express the lighting effect, so as to reduce the memory overhead and CPU calculation time of real-time rendering of the model. For example, it can be to pre-establish various operations related to primary equipment and protection and automatic devices, station power supply systems, grounding transformer systems, DC systems and other equipment; the placement and removal of secondary small air switches, fuses, and fuses; the switching on and off of protection automatic devices, mode switching and operation of their LCD panels, modification of protection settings, etc.; and include operation animations corresponding to the power test operations before the equipment is powered off, such as a list of contents for simulating daily switching operations of substations: including all normal simulation operations, switching operations, remote control operations, tap adjustment, normal operations (including operations and inspections of indoor and outdoor primary and secondary equipment and monitoring background), including: 1. Switch and knife switch operations, operations on the simulation interface of the substation monitoring background or on-site control screen, including on-site switch synchronization operations; 2. Switching of voltage transformers; 3. Automatic reclosing mode 1. Switching; 2. Switching of insulation monitoring voltmeter; 3. Operation of accident recorder; 4. Operation of switching on, switching off, mode switching, information inquiry and other operations of protection and automatic devices; 5. Switching on and off of standby automatic switching devices and operation of the devices themselves; 6. Finding of grounding faults in small current grounding systems; 7. Adjustment of transformer taps; 8. Switching on and off of reactors and adjustment; 11. Operation of AC and DC systems and UPS systems; 12. Operation of other safety automatic devices; 13. Safety measures for on-site substation work (power outage, power testing, grounding wire hanging, setting of signs, fences, etc.); 14. Periodic testing and switching: transformer cooling system, station transformer system, etc.; 15. Switching operation of station power supply system mode and DC system; 16. Switching operation of important accessories (such as main transformer cooler; switching of DC system).

[0090] Based on the above technical solution, the acquisition of interactive operations with the target object includes: collecting somatosensory motion data and / or gesture data corresponding to the target object based on a somatosensory camera; and / or, acquiring interactive operations sent by the target object through a control device.

[0091] The control device includes at least one of a touch gesture recognition, a mouse, a keyboard, a tablet computer, a PC and a large-screen device.

[0092] Specifically, in a virtual three-dimensional scene, the target object can move freely through touch gesture recognition, mouse, keyboard, tablet computer, PC computer and large-screen device. The mouse wheel, keyboard control, tablet touch, PC touch and large-screen device touch can be used to realize equipment inspection, observation, inspection, equipment rotation roaming motion mode, photography, infrared temperature measurement, defect reporting and other operations. For example, on a device that supports touch operation (such as a tablet computer, touch screen PC, etc.), the user can control the movement and rotation of the target object through gestures such as sliding and clicking of the finger. For example, two-finger sliding can realize the zoom function, single-finger sliding can move the viewing angle, and the rotation gesture can change the direction of the target object. In the simulation process, the user can control the target object to rotate or roam around a certain center point through the mouse wheel, keyboard control, tablet touch or large-screen device touch. In the virtual scene, the user can simulate the functions of taking pictures and infrared temperature measurement. For example, by clicking the "photograph" button on the screen, you can save a screenshot of the current perspective; by selecting the "infrared temperature measurement" function, you can simulate the operation of the infrared thermometer and display the temperature distribution of the device. When you find a device defect in the virtual scene, you can click the "report defect" button on the screen to fill in the relevant information and submit it. Furthermore, in order to ensure the smooth operation and good experience of the virtual three-dimensional scene on different devices, it is necessary to adapt and optimize the device, adjust the rendering quality of the virtual scene according to the screen resolution of the device, ensure that a clear picture can be presented on different devices, and optimize the sensitivity and accuracy of the touch operation according to the characteristics of the touch device to improve the user experience. According to the performance level of the device, adjust the rendering parameters and physical simulation accuracy of the virtual scene to ensure smooth operation on different devices. Exemplarily, the mouse wheel operation method may include: rolling the mouse wheel forward means backing away from the device, and rolling the mouse wheel backward means moving forward and pulling the device closer. Press and hold the mouse wheel to drag the scene left and right. Double-click a device with the mouse wheel, and then press Alt+wheel to rotate the device that was just double-clicked as the center. The keyboard control operation method includes: the left and right direction keys of the keyboard control represent turning the head left and right; the up and down direction keys of the keyboard control represent small steps forward and small steps backward; the Home and End of the keyboard control represent large steps forward and large steps backward; the PgUp and PgDn of the keyboard control represent upward and downward translation; the Q and E letter keys of the keyboard control represent left and right translation; the W and S letter keys of the keyboard control represent raising the head and lowering the head.

[0093] On the basis of the above technical solution, if the target object is in a virtual three-dimensional scene in the device's surround rotation and roaming motion mode, the target object can be observed around the device by pressing the left and right arrow keys on the keyboard control keys, and the surround mode can be exited by double-clicking the right button anywhere in the virtual three-dimensional scene. After entering the device's surround rotation and roaming motion mode, the normal roaming function of the keyboard control keys will become invalid, and it will return to normal after exiting the device's surround rotation and roaming motion mode. In the device's surround rotation and roaming motion mode, the keyboard control keys are basically the same as in the operating mode, but when rotating left and right, it orbits the geometric center of the device instead of rotating; when moving forward and backward, it closes or distances the device instead of moving forward and backward. If you want to exit the surround mode and return to the operating mode, just double-click the right mouse button anywhere on the screen.

[0094] On the basis of the above technical solution, if the target object is in the process of patrolling and observing a device in a virtual three-dimensional scene, the target object can be the device to be judged by double-clicking the left mouse button. The virtual three-dimensional scene will pop up the patrol point information of the device. After judging the abnormal state of the device, the virtual three-dimensional scene will pop up the exception handling dialog box, select the Equipment Defects button, and select the corresponding defective parts, defect types and defect levels, and then select the corresponding processing mode, and click the Report Defects button with the mouse.

[0095] On the basis of the above technical scheme, if the target object is simulated by touch operation on the LCD panel of a tablet computer, PC computer and large-screen device, the touch operation method on the LCD panel of the tablet computer, PC computer and large-screen device can adopt the front-end and back-end technology superposition to realize cross-platform development and display of the target screen. Through the touch operation on the LCD panel of the tablet computer, PC computer and large-screen device, the target screen audio playback control function is provided, including audio playback progress bar, audio playback, audio playback, pause, stop and other functions. The audio playback is to record the entire operation process of the target object in the form of video, so that the touch operator on the LCD panel of the tablet computer, PC computer and large-screen device can point out the irregular and substandard operation errors in the operation process of a certain target object, and at the same time enable the target object to review its own operation process and be familiar with the operation process. The audio playback function supports the export of recorded files and supports multiple media formats, so that the touch operator and the target object on the LCD panel of the tablet computer, PC computer and large-screen device can review it at any time. The simulation model simulation training of the game engine development is realized, and the operation task, task guidance, tool box and return function are provided. The touch operation method on the LCD panel of tablet computers, PCs and large-screen devices can realize two different display modes of the target screen: stand-alone version and network version. It can not only realize the operation of stand-alone mode but also realize multi-person online collaborative operation. The simulation model uses front-end and back-end technologies to complete cross-platform development docking and realize data interaction. For example, the simulation model can realize data interaction with the network education and training system of the prior art and realize normal operation on the network education and training system platform of the prior art. The simulation model has the characteristics of high cohesion and loose coupling, and can realize the expansion of functional modules in the form of plug-ins, supporting the secondary development of the simulation model.

[0096] Furthermore, the Kinect sensor can be used to capture the user's somatosensory interactive motion data and map it to the virtual scene to drive the virtual software robot to interact with the three-dimensional virtual scene. The precise collision detection between the virtual software robot body and the operable three-dimensional objects in the virtual substation combined with gesture judgment can realize the somatosensory interaction between the trainees and the three-dimensional virtual scene. The interactive operation of the target object can also be obtained by detecting the hand movements of the target object.

[0097] Exemplarily, gesture recognition refers to various predefined forms of actions made by people based on a certain operation intention, including the extension and bending of fingers, the movement of hands in space, etc., or to complete a task, such as pressing a button, twisting an operating handle, rotating a valve, grabbing or releasing an object, etc., to express a certain operation action or intention. A gesture recognition method based on Kinect can be used to realize the function of translating user gestures into operation commands. The trainer makes gestures or directly makes gestures to express the operation command. The system matches the current user gesture with the predefined reference gesture library one by one through the collected motion data. If a reference gesture with a matching degree less than the matching threshold is found, the simulation model executes the operation command defined by the gesture to realize the gesture response. Furthermore, a variety of virtual operations of the virtual hand on the operation device object are realized, and common operations include pressing, twisting and grabbing, etc. The operation interaction of the virtual hand on the device object in the virtual scene includes the spatial transformation of the operation object, the movement collision obstacle of the virtual hand, etc. The implementation steps are as follows: 1) The virtual hand moves toward the target object: The operation and maintenance personnel or training personnel control the movement of the virtual hand in the virtual environment through interactive devices such as data gloves, position trackers or bare hands, so that the virtual hand moves toward the virtual object to be operated. 2) Gesture preparation before operation: Before performing the operation action, the operation and maintenance personnel or training personnel adjust the hand shape posture to the hand shape posture that matches the operation action. 3) Collision detection: Collision detection with the virtual object is a prerequisite for completing the operation action. If there is a collision, proceed to the next step to perform the operation action; if there is no collision, continue to move the virtual hand close to the virtual object. Collision detection can also effectively prevent the "penetration" phenomenon between the virtual hand and the virtual object. 4) Determine the operation action rules: Determine the operation action rules in turn. If the rules are met, proceed to the next step; if not, prompt the operator to adjust the hand shape. 5) Operation action completion: If the operation action rules are met, the operated virtual object will respond according to the type of operation action.

[0098] The technical solution of the embodiment of the present invention obtains the interactive operation with the target object, determines the interactive control instruction according to the interactive operation, determines the target screen based on the interactive control instruction and the interactive operation simulation scene, and outputs the target screen to the display device. Based on the above technical solution, by obtaining the interactive operation of the target object, and determining the target screen according to the interactive operation and the interactive operation simulation scene, and then outputting the target screen to the display device, augmented reality technology or virtual reality technology is applied to power system teaching and training, turning physical simulation teaching into pure computer simulation teaching, turning the original relatively simple and boring two-dimensional screen into a three-dimensional screen, and operating personnel can deeply study the structure of equipment such as transformers in an augmented reality or virtual reality environment, understand its working principle and main operating procedures, and experience the entire process of power production such as power transmission and transformation, and transform the original abstract theoretical knowledge learning into practical learning closely related to actual operation, which greatly improves the learning interest of students.

[0099] Embodiment 3

[0100] Figure 5 The structure block diagram of a simulation device for substation interactive operation provided by an embodiment of the present invention. Figure 5 As shown, the device includes: a model acquisition module 510, a virtual software robot construction module 520 and an operation simulation module 530.

[0101] The model acquisition module 510 is used to determine the substation environment simulation model, the substation tool simulation model and the substation equipment simulation model based on the power operation simulation requirements, wherein the environment simulation model, the tool simulation model and the equipment simulation model are pre-established virtual three-dimensional models corresponding to the substation;

[0102] A virtual software robot construction module 520 is used to obtain human body structure information corresponding to the target object, and to establish a virtual software robot body model corresponding to the target object according to the human body structure information;

[0103] The operation simulation module 530 is used to establish an interactive operation simulation scenario corresponding to the power operation simulation requirements based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and perform interactive operation simulation according to the interactive operation simulation scenario.

[0104] On the basis of the above technical solution, the virtual software robot construction module is used to collect multiple high-scale three-dimensional depth images corresponding to the target object based on the somatosensory camera, and determine the human body structure information of the target object according to the multiple high-scale three-dimensional depth images, wherein the human body structure information includes human skeletal structure, skin structure and outer surface structure.

[0105] On the basis of the above technical solution, the virtual software robot construction module is used to control the somatosensory camera to project coded infrared rays to the measurement area when it is detected that the target object is located within the measurement area of ​​the somatosensory camera; collect original images corresponding to the measurement area, and determine the multiple high-scale three-dimensional depth images according to the original images and the coded infrared rays; establish a massive database of multiple high-scale three-dimensional depth images based on the multiple high-scale three-dimensional depth images, so as to losslessly compress, store, restore, reconstruct, and display the multiple high-scale three-dimensional depth images based on the massive database of multiple high-scale three-dimensional depth images.

[0106] On the basis of the above technical solution, the virtual software robot construction module is used to determine the human skeletal structure of the target object based on the human structure information, and determine the matching result between the human skeletal structure and the preset virtual software robot body model; determine the target virtual software robot body model based on the matching result, and establish the virtual software robot body model according to the target virtual software robot body model and the human structure information.

[0107] On the basis of the above technical scheme, the target object includes any at least one of the work supervisor, the work permit holder, the work issuer, the guardian and the operator; the virtual three-dimensional model is used to realize multi-person collaborative work of multiple target objects; the multi-person collaborative work is that multiple target objects join the training in multiple roles in the virtual three-dimensional model, and the interactive operation simulation is completed according to the role interaction and collaboration of the different target objects; the substation equipment simulation model includes a substation primary equipment simulation model and a substation secondary equipment simulation model; the substation equipment simulation model is used to show the internal structure, appearance and key parameters of the substation equipment; the substation tool simulation model includes a safety tool simulation model, a protective equipment simulation model and an instrument simulation model; the substation environment simulation model includes a substation three-dimensional scene simulation model and a substation three-dimensional meteorological simulation model; the substation three-dimensional scene simulation model includes a substation three-dimensional scene simulation model under normal operating conditions and a substation three-dimensional scene simulation model when an accident occurs; the substation three-dimensional meteorological simulation model includes a model library under different meteorological conditions, and a transmission line dynamic model corresponding to the meteorological conditions under different meteorological conditions according to the substation; the meteorological conditions at least include sunny, foggy, rainy, typhoon and lightning.

[0108] On the basis of the above technical scheme, according to the dynamic model of the transmission line corresponding to the meteorological conditions under different meteorological conditions of the substation, it includes: obtaining the static equally divided coordinates of the transmission line in the station corresponding to the substation, and determining the dynamic equally divided coordinates with the static equally divided coordinates according to the wind deviation angle; determining the hanging points and equally divided nodes of the transmission line in the station based on the dynamic equally divided coordinates, and connecting the hanging points and the equally divided nodes to determine the dynamic model of the transmission line.

[0109] On the basis of the above technical solution, the operation simulation module is used to obtain the interactive operation with the target object, determine the interactive control instruction according to the interactive operation; determine the target screen based on the interactive control instruction and the interactive operation simulation scene, and output the target screen to the display device, wherein the display device includes a large 3D arc screen, 3D glasses, an electronic touch sandbox, a somatosensory interactive system, a tablet computer, and a workstation.

[0110] Based on the above technical solution, the operation simulation module is used to collect somatosensory motion data and / or gesture data corresponding to the target object based on a somatosensory camera; and / or, obtain the interactive operation sent by the target object through a control device, wherein the control device includes at least one of a touch gesture recognition, a mouse, a keyboard, a tablet computer, a PC and a large-screen device.

[0111] The technical solution of the embodiment of the present invention determines the substation tool simulation model, the substation environment simulation model and the substation equipment simulation model based on the power operation simulation requirements, and obtains the human body structure information corresponding to the target object, establishes a virtual software robot body model corresponding to the target object according to the human body structure information, and then establishes an interactive operation simulation scene corresponding to the power operation simulation requirements based on the substation tool simulation model, the substation environment simulation model, the substation equipment simulation model and the virtual software robot body model, and performs interactive operation simulation according to the interactive operation simulation scene. Based on the above technical solution, by constructing a virtual software robot body model corresponding to the target object, and then establishing an interactive operation simulation scene based on the virtual software robot body model and the substation tool simulation model, the substation environment simulation model and the substation equipment simulation model, the authenticity and realism of the simulation exercise process are improved, thereby improving the training efficiency of operation and maintenance personnel.

[0112] The simulation device for substation interactive operation provided by the embodiment of the present invention can execute the simulation method for substation interactive operation provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0113] Embodiment 4

[0114] Figure 6 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0115] like Figure 6 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0116] The fidelity and realism of interactive operation simulation depends on the modeling accuracy of each component and model in the power grid. In order to ensure the modeling accuracy of each component and model, and to drive the virtual scene to perform complex interactive actions according to the real-time actions of the user, the electronic device provided by the embodiment of the present invention needs to adopt multi-CPU memory, multi-CPU processor and multi-computer distributed computing and parallel computing technology, such as Figure 7 As shown, Figure 7 Another structural schematic diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. Figure 7 As shown, the electronic device using multi-CPU memory, multi-CPU processor and multi-computer distributed computing and parallel computing technology can realize the real reproduction of the dynamic process of power grid simulation under various operations, abnormalities and even fault conditions, and ensure a fast real-time response speed to various operations or faults.

[0117] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0118] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a simulation method for substation interaction.

[0119] In some embodiments, the simulation method of substation interaction operation may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the simulation method of substation interaction operation described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the simulation method of substation interaction operation in any other appropriate manner (for example, by means of firmware).

[0120] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0122] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0125] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0126] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0127] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A simulation method for substation interactive operation, characterized in that: include: Determine a substation environment simulation model, a substation tool simulation model and a substation equipment simulation model based on power operation simulation requirements, wherein the environment simulation model, the tool simulation model and the equipment simulation model are pre-established virtual three-dimensional models corresponding to the substation; Acquire human body structure information corresponding to the target object, and establish a virtual software robot body model corresponding to the target object according to the human body structure information; An interactive operation simulation scenario corresponding to the power operation simulation requirement is established based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and an interactive operation simulation is performed according to the interactive operation simulation scenario.

2. The method according to claim 1, characterized in that The obtaining of human body structure information corresponding to the target object includes: Based on the somatosensory camera, multiple high-proportion three-dimensional depth images corresponding to the target object are collected, and the human body structure information of the target object is determined according to the multiple high-proportion three-dimensional depth images, wherein the human body structure information includes human body bone structure, skin structure and outer surface structure.

3. The method according to claim 2, characterized in that The method of collecting multiple high-ratio three-dimensional depth images corresponding to the target object based on a body sensing camera includes: When it is detected that the target object is located within the measurement area of ​​the body sensing camera, controlling the body sensing camera to project coded infrared rays toward the measurement area; Acquire an original image corresponding to the measurement area, and determine the multiple high-scale three-dimensional depth images according to the original image and the coded infrared rays; A massive database of multiple high-scale three-dimensional depth images is established based on the multiple high-scale three-dimensional depth images, so as to losslessly compress, store, restore, reconstruct, and display the multiple high-scale three-dimensional depth images based on the massive database of multiple high-scale three-dimensional depth images.

4. The method according to claim 1, characterized in that: The step of establishing a virtual software robot body model corresponding to the target object according to the human body structure information comprises: Determine the human skeletal structure of the target object based on the human body structure information, and determine a matching result between the human skeletal structure and a preset virtual software robot body model; A target virtual software robot body model is determined based on the matching result, and the virtual software robot body model is established according to the target virtual software robot body model and the human body structure information.

5. The method according to claim 1, characterized in that The target object includes any one of a work supervisor, a work permit holder, a work issuer, a guardian, and an operator; the virtual three-dimensional model is used to realize multi-person collaborative work of multiple target objects; The multi-person collaborative operation is that multiple target objects join the training in multiple roles in the virtual three-dimensional model, and the interactive operation simulation is completed according to the role interaction of the different target objects; the substation equipment simulation model includes a substation primary equipment simulation model and a substation secondary equipment simulation model; the substation equipment simulation model is used to display the internal structure, appearance, and key parameters of the substation equipment; the substation tool simulation model includes a safety tool simulation model, a protective equipment simulation model, and an instrument simulation model; the substation environment simulation model includes a substation three-dimensional scene simulation model and a substation three-dimensional meteorological simulation model; the substation three-dimensional scene simulation model includes a substation three-dimensional scene simulation model under normal operating conditions and a substation three-dimensional scene simulation model when an accident occurs; the substation three-dimensional meteorological simulation model includes a model library under different meteorological conditions, and a dynamic model of the transmission line corresponding to the meteorological conditions according to the substation under different meteorological conditions; the meteorological conditions at least include sunny days, foggy days, rainy days, typhoons, and lightning.

6. The method according to claim 5, characterized in that The dynamic model of the transmission line corresponding to the meteorological conditions under different meteorological conditions of the substation includes: Obtaining static equally divided coordinates of the transmission lines within the substation corresponding to the substation, and determining dynamic equally divided coordinates corresponding to the static equally divided coordinates according to the wind deflection angle; The suspension points and the equally divided nodes of the power transmission line within the station are determined based on the dynamic equally divided coordinates, and the suspension points and the equally divided nodes are connected to determine the dynamic model of the power transmission line.

7. The method according to claim 1, characterized in that The performing interactive operation simulation according to the interactive operation simulation scenario includes: Acquire an interactive operation with the target object, and determine an interactive control instruction according to the interactive operation; A target screen is determined based on the interactive control instruction and the interactive operation simulation scenario, and the target screen is output to a display device, wherein the display device includes a large 3D arc screen, 3D glasses, an electronic touch sandbox, a somatosensory interactive system, a tablet computer, and a workstation.

8. The method according to claim 7, characterized in that The acquiring of the interaction operation with the target object includes: Collecting somatosensory motion data and / or gesture data corresponding to the target object based on a somatosensory camera; and / or, Acquire the active control interaction operation sent by the target object through the control device, wherein the control device includes any at least one of touch gesture recognition, mouse, keyboard, tablet computer, PC and large screen device.

9. A simulation device for substation interactive operation, characterized in that: include: A model acquisition module, used to determine a substation environment simulation model, a substation tool simulation model and a substation equipment simulation model based on power operation simulation requirements, wherein the environment simulation model, the tool simulation model and the equipment simulation model are pre-established virtual three-dimensional models corresponding to the substation; A virtual software robot construction module is used to obtain human body structure information corresponding to the target object, and to establish a virtual software robot body model corresponding to the target object according to the human body structure information; An operation simulation module is used to establish an interactive operation simulation scenario corresponding to the power operation simulation requirement based on the substation environment simulation model, the substation tool simulation model, the substation equipment simulation model and the virtual software robot body model, and perform interactive operation simulation according to the interactive operation simulation scenario.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and at least one memory in communication with the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for simulating substation interaction operation according to any one of claims 1 to 8.

Images