Monitoring method and device for substation operation

Abstract

The invention discloses a monitoring method and device for substation operation, and relates to the technical field of electric power, in particular to the technical field of substations. The method comprises the following steps: constructing a digital twinborn model of a transformer substation according to transformer substation data and equipment data; constructing a digital twinborn environment according to the digital twinborn model and a virtual model of historical operating personnel of the transformer substation; in the digital twin environment, monitoring a target operator according to the operation identification network and the equipment operation identification network to obtain a preliminary monitoring result; and determining a target monitoring result according to the preliminary monitoring result and the personnel state recognition network. The technical problem that in the prior art, the monitoring effect is poor due to the fact that video supervision equipment such as a near induction technology and a monitoring ball and a law enforcement instrument is used for transformer substation operation monitoring is solved.

Description

Technical Field

[0001] The present application relates to the field of electric power technology, in particular to the field of substation technology, and more specifically to a monitoring method and device for substation operations. Background Art

[0002] In the power system, substations play a vital role. They are not only the key nodes for primary energy transmission, conversion and distribution, but also the intelligent control center to ensure the safe and stable operation of the power grid. However, during the substation operation monitoring process, due to the large number of power equipment and the complex live conditions, the operation risks and hidden dangers also increase.

[0003] At present, the main reliance is on near-electric induction technology to warn of the spatial distance between personnel, vehicles, tools and live objects to reduce operational risks. Due to the lack of technical means to accurately determine the spatial distance between operators, tools, construction vehicles and live objects, safety accidents such as personnel mistakenly entering live intervals and large machinery accidentally touching live areas often occur. Especially in complex electric field environments where multiple voltage devices are intertwined, such as substations or line crossings, simple near-electric induction technology is prone to false alarms and missed alarms, and cannot meet the needs of accurate judgment.

[0004] In addition, with the promotion and application of smart safety monitoring technology, each work site is generally equipped with video inspection equipment such as surveillance cameras and law enforcement cameras for online video supervision. However, these video inspection equipment are limited by the on-site placement, network communication and video terminal technical parameters, making it difficult to achieve a three-dimensional display of the behavior trajectory of the work site personnel, live parts, safety measures, etc., as well as risk identification.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The present application provides a monitoring method and device for substation operations, so as to at least solve the technical problem that the prior art uses proximity sensing technology and video inspection equipment such as surveillance balls and law enforcement instruments to monitor substation operations, resulting in poor monitoring effects.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a monitoring method for substation operations is provided, including: constructing a digital twin model of the substation based on substation data and equipment data, wherein the substation data at least includes contour data and a panoramic image of the substation, and the equipment data at least includes location information, geometric information of multiple devices in the substation, and interaction information between different devices, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation; constructing a digital twin environment based on the digital twin model and a virtual model of historical operators of the substation, wherein the digital twin environment is used to simulate and analyze the substation; in the digital twin environment, monitoring the target operators according to the operation identification network and the equipment operation identification network to obtain preliminary monitoring results, wherein the operation identification network is used to identify the operation behavior type of the target operators, and the equipment operation identification network is used to identify the operation status of the equipment; determining the target monitoring results based on the preliminary monitoring results and the personnel status identification network, wherein the personnel status identification network is used to identify the status of the target operators.

[0008] Optionally, a digital twin model of the substation is constructed based on the substation data and the equipment data, including: constructing a substation model based on the substation data, wherein the substation model is used to virtually represent the three-dimensional entity information of the overall structure of the substation; constructing an initial equipment model based on the equipment data, wherein the initial equipment model is used to virtually represent the three-dimensional entity information of the equipment in the substation; obtaining association parameters between different equipment in the substation; determining the association relationship between different equipment based on the association parameters, wherein the association relationship is used to represent the interaction relationship between different equipment; determining the target equipment model based on the association relationship and the initial equipment model; and determining the digital twin model based on the location information of the substation equipment, the substation model, and the target equipment model.

[0009] Optionally, before building a digital twin environment based on the digital twin model and the virtual model of the historical operating personnel of the substation, it includes: obtaining the posture of the historical operating personnel; decomposing the posture into N human joints, where N is an integer greater than 1; associating the N human joints according to the human skeleton connection relationship to obtain the virtual model of the historical operating personnel.

[0010] Optionally, a digital twin environment is constructed based on the digital twin model and the virtual model of the historical operators of the substation, including: taking the historical operation data of the substation and the operation video of the historical operators as target data; and constructing the digital twin environment based on the digital twin model, the virtual model and the target data.

[0011] Optionally, the job identification network is obtained through the following steps: acquiring the job behavior data of historical workers; simulating the job behavior in a virtual model based on the job behavior data to generate abnormal behavior of historical workers; generating an abnormal behavior data set based on the abnormal behavior; and training a first neural network based on the abnormal behavior data set to obtain a job identification network.

[0012] Optionally, in the process of training the first neural network according to the abnormal behavior data set to obtain the operation recognition network, the monitoring method for substation operations also includes: determining the video image features of the substation according to the abnormal behavior data set, wherein the video image features are used to describe the motion and posture features of historical operators; inputting the video image features into the target branch to obtain two-dimensional operation posture data, wherein the target branch is used to learn and judge human posture information in the video according to the video image features; mapping the two-dimensional operation posture data in three-dimensional space to obtain three-dimensional operation posture data; and performing classification and recognition based on the three-dimensional operation posture data, wherein the classification and recognition is used to identify the operation status of historical operators.

[0013] Optionally, the equipment operation identification network is obtained through the following steps: simulating the operating status of the equipment in the substation according to the abnormal behavior data set and the digital twin model to obtain the abnormal operation data set of the equipment; training the second neural network according to the abnormal operation data set to obtain the equipment operation identification network.

[0014] Optionally, after mapping the two-dimensional work posture data in three-dimensional space to obtain three-dimensional work posture data, the monitoring method for substation operations also includes: binding the human body joint points in the three-dimensional work posture data with the joint points of historical workers in the virtual model.

[0015] Optionally, the personnel status recognition network is obtained through the following steps: collecting physiological data of the operating personnel, wherein the physiological data at least includes heart rate, blood pressure, and body temperature; training a third neural network based on the physiological data to obtain the personnel status recognition network.

[0016] According to another aspect of the present application, a monitoring device for substation operations is also provided, including: a first construction unit, which constructs a digital twin model of the substation based on substation data and equipment data, wherein the substation data includes at least the outline data and panoramic image of the substation, and the equipment data includes at least the location information, geometric information and interaction information between different devices of multiple devices in the substation, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation; a second construction unit, which constructs a digital twin environment based on the digital twin model and the virtual model of the historical operating personnel of the substation, wherein the digital twin environment is used to simulate and analyze the substation; a first determination unit, which monitors the target operating personnel in the digital twin environment according to the operation identification network and the equipment operation identification network to obtain preliminary monitoring results, wherein the operation identification network is used to identify the operating behavior type of the target operating personnel, and the equipment operation identification network is used to identify the operating status of the equipment; a second determination unit, which determines the target monitoring result based on the preliminary monitoring result and the personnel status identification network, wherein the personnel status identification network is used to identify the status of the target operating personnel.

[0017] In the embodiment of the present application, a digital twin model is constructed by using the relevant data of the substation and the relevant data of the equipment in the substation, and then the digital twin model and the virtual model of the historical operator of the substation are virtually assembled to realize the construction of the digital twin environment. Then, in the digital twin environment, the operating status of the substation operator and the working status of the equipment in the substation are obtained by using the trained operation recognition network and the equipment operation recognition network, and then the trained personnel status recognition network is combined to identify the status of the operator himself. Based on the analysis of the three dimensions of the operation recognition network, the equipment operation recognition network and the personnel status network, the overall operation status of the substation is obtained. Compared with the use of video and other inspection equipment to monitor the substation in the prior art, this application realizes real-time monitoring of the substation based on the digital twin model, and analyzes the status in the substation based on the multi-dimensional recognition network, and finally obtains the monitoring result. The purpose of more comprehensive monitoring and analysis of the substation is achieved, thereby achieving the technical effect of improving the monitoring effect of the substation, and then solving the technical problem of poor monitoring effect caused by using proximity sensing technology and video inspection equipment such as control balls and law enforcement instruments in the prior art to monitor the operation of the substation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a flow chart of an optional monitoring method for substation operations according to an embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of an optional monitoring device for substation operations according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] It should also be noted that the information and data collected by this application are authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation portals for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0024] According to an embodiment of the present application, an embodiment of a monitoring method for substation operations is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0025] It should be noted that an intelligent monitoring system is used as the execution subject of the monitoring method for substation operation in the embodiment of the present application. It is understandable that the monitoring method for substation operation provided in the embodiment of the present application can also be executed by other systems or devices, and the embodiment of the present application does not specifically limit this.

[0026] Figure 1 is a flow chart of an optional monitoring method for substation operation according to an embodiment of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0027] Step S101, constructing a digital twin model of the substation based on the substation data and equipment data.

[0028] In step S101, the substation data includes at least the outline data and panoramic image of the substation, the equipment data includes at least the location information, geometric information of multiple devices in the substation, and the interaction information between different devices, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation.

[0029] Optionally, the intelligent monitoring system constructs a virtual three-dimensional model of the substation based on the substation data and equipment data, and real-time monitoring of the substation can be achieved through the digital twin model.

[0030] Step S102: construct a digital twin environment based on the digital twin model and the virtual model of the historical operators of the substation.

[0031] In step S102, the digital twin environment is used to simulate and analyze the substation.

[0032] Optionally, the intelligent monitoring system builds a digital twin environment based on the digital twin model and the virtual model. The digital twin environment can be used to simulate various scenarios and conditions in the substation and test different strategies and methods.

[0033] Step S103: In the digital twin environment, the target operator is monitored according to the operation identification network and the equipment operation identification network to obtain preliminary monitoring results.

[0034] In step S103, the operation identification network is used to identify the operation behavior type of the target operator, and the equipment operation identification network is used to identify the operation status of the equipment.

[0035] Optionally, the intelligent monitoring system identifies the job type, job content, abnormal job behavior, etc. of the workers in the substation in real time based on the trained job identification network, and identifies the operating status of all equipment in the substation in real time based on the trained equipment operation identification network.

[0036] Step S104, determining the target monitoring result according to the preliminary monitoring result and the personnel status recognition network.

[0037] In step S104, the personnel status recognition network is used to identify the status of the target operator.

[0038] Optionally, the intelligent monitoring system monitors the status of operating personnel in real time based on a trained personnel status recognition network.

[0039] It can be seen from the contents of steps S101 to S104 that in the present application, firstly, a digital twin model of the substation is constructed based on the substation data and the equipment data, wherein the substation data at least includes the outline data and the panoramic image of the substation, and the equipment data at least includes the location information, geometric information and interaction information between different devices of multiple devices in the substation, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation. Secondly, a digital twin environment is constructed based on the digital twin model and the virtual model of the historical operators of the substation, wherein the digital twin environment is used to simulate and analyze the substation. Then, in the digital twin environment, the target operators are monitored according to the operation identification network and the equipment operation identification network to obtain preliminary monitoring results, wherein the operation identification network is used to identify the operation behavior type of the target operators, and the equipment operation identification network is used to identify the operation status of the equipment. Finally, the target monitoring results are determined based on the preliminary monitoring results and the personnel status identification network, wherein the personnel status identification network is used to identify the status of the target operators.

[0040] As can be seen from the above content, in the embodiment of the present application, a digital twin model is constructed by using the relevant data of the substation and the relevant data of the equipment in the substation, and then the digital twin model and the virtual model of the historical operator of the substation are virtually assembled to realize the construction of the digital twin environment, and then the operation status of the substation operator and the working status of the equipment in the substation are obtained by using the trained operation recognition network and the equipment operation recognition network in the digital twin environment, and then the operator's own status is identified by combining the trained personnel status recognition network. The overall operation status of the substation is obtained based on the analysis of the three dimensions of the operation recognition network, the equipment operation recognition network and the personnel status network. Compared with the existing technology using video and other inspection equipment to monitor the substation, this application realizes real-time monitoring of the substation based on the digital twin model, and analyzes the status in the substation based on the multi-dimensional recognition network, and finally obtains the monitoring result. The purpose of more comprehensive monitoring and analysis of the substation is achieved, thereby achieving the technical effect of improving the monitoring effect of the substation, and then solving the technical problem of poor monitoring effect caused by using proximity sensing technology and video inspection equipment such as control balls and law enforcement instruments in the existing technology to monitor the operation of the substation.

[0041] In an optional embodiment, the intelligent monitoring system constructs a substation model based on the substation data, wherein the substation model is used to virtually represent the three-dimensional entity information of the overall structure of the substation, and constructs an initial equipment model based on the equipment data, wherein the initial equipment model is used to virtually represent the three-dimensional entity information of the equipment in the substation, and then obtains the association parameters between different equipment in the substation, and determines the association relationship between different equipment based on the association parameters, wherein the association relationship is used to represent the interaction relationship between different equipment, and determines the target equipment model based on the association relationship and the initial equipment model, and finally determines the digital twin model based on the location information of the substation equipment, the substation model and the target equipment model.

[0042] Optionally, the intelligent monitoring system first constructs a virtual three-dimensional model of the overall structure of the substation based on the contour data and panoramic image of the substation, then constructs an initial equipment virtual three-dimensional model based on the geometric parameters of each distribution equipment in the substation, and then associates each device in the initial equipment virtual three-dimensional model based on the obtained association relationship between the devices to obtain a target equipment model. Finally, according to the position parameters of each device, the target equipment model is placed at the corresponding position in the substation model to obtain a digital twin model of the substation.

[0043] Optionally, the intelligent monitoring system drives the digital twin model through the historical operating data of each distribution equipment in the substation collected by sensors, checks the correctness of the digital twin model, and modifies the physical and functional characteristics that do not meet the requirements, thereby optimizing the digital twin model.

[0044] From the above content, it can be seen that the intelligent monitoring system constructs a digital twin model through substation data, equipment data, the relationship between different equipment and the location information of the equipment. Based on the digital twin model, it realizes a three-dimensional presentation of the overall structure of the substation, equipment distribution, etc., and improves the substation's production safety guarantee capability and operation and maintenance management capability. At the same time, the intelligent monitoring system uses the historical operation data of each distribution equipment in the substation to drive the digital twin model, and continuously checks the correctness of the digital twin model. In this way, it can ensure that the digital twin model is highly consistent with the actual operating status of the substation, thereby improving the accuracy and efficiency of the monitoring system.

[0045] In an optional embodiment, the intelligent monitoring system obtains the posture of the historical operator, decomposes the posture into N human joints, where N is an integer greater than 1, and then associates the N human joints according to the connection relationship of the human skeleton to obtain a virtual model of the historical operator.

[0046] Optionally, the intelligent monitoring system obtains the posture information of the historical operator, decomposes the posture into multiple human joints, and then associates these joints according to the connection relationship of the human skeleton, and finally obtains a virtual model of the historical operator.

[0047] Optionally, the intelligent monitoring system obtains video data when performing different work tasks. This data can come from information collection equipment such as on-site monitoring cameras. The collected video data is then preprocessed, including denoising, deblurring, adjusting brightness and contrast, etc. From the preprocessed video data, a frame is captured at regular time intervals to form a frame sequence. Then, computer vision techniques, such as deep learning and image processing, are used to perform posture detection on each frame of the image, and key information is extracted from the detected posture, such as the head, shoulders, elbows, wrists, waist, knees and ankles. The extracted joint points are associated according to the skeletal connection relationship of the human body. For example, the shoulder joint is connected to the upper arm, forearm and hand joints. The associated joint points and skeletal model are used to construct a three-dimensional virtual model of the historical operator.

[0048] From the above content, it can be seen that the intelligent monitoring system can improve the ability to capture and analyze the posture of operators by constructing a virtual model of substation operators, and can also provide important technical support for substation safety monitoring and risk assessment. At the same time, by building an accurate virtual model, it can better understand and predict the actions of operators, thereby effectively improving the safety management level of substations.

[0049] In an optional embodiment, the intelligent monitoring system uses the historical operation data of the substation and the historical operation videos of the operators as target data, and then builds a digital twin environment based on the digital twin model, the virtual model and the target data.

[0050] Optionally, the digital twin model of the substation and the virtual model of the operator are virtually assembled, and the actual operation data of the substation and the operation video data of the operator are used to drive the digital twin model to realize the construction of a digital twin environment. In this environment, the activities of the operators in the substation and the operation of the equipment can be simulated and analyzed.

[0051] From the above content, it can be seen that the intelligent monitoring system can effectively build a digital twin environment through digital twin models, virtual models of operators and related data. In this environment, substations can be monitored, analyzed, optimized and made decisions in real time, realizing comprehensive monitoring and management of substations and operators, and improving operational efficiency and safety.

[0052] In an optional embodiment, the intelligent monitoring system obtains the work behavior data of historical operators, simulates the work behavior in a virtual model based on the work behavior data, generates abnormal behavior of the historical operators, and then generates an abnormal behavior data set based on the abnormal behavior. Finally, the first neural network is trained based on the abnormal behavior data set to obtain a work recognition network.

[0053] Optionally, actual work behavior data of the operators is obtained, posture decomposition is performed, and work behavior simulation is performed in a virtual model based on the work behavior data to generate abnormal behaviors of several operators. Videos of the above abnormal behaviors are collected through a virtual camera to form an abnormal behavior data set of the operators. Then, the first neural network is trained according to the abnormal behavior data set to obtain a work recognition network. After the training, the work recognition network can recognize the abnormal behavior of the operators, thereby realizing real-time monitoring of the operators' behaviors and automatic detection of abnormal behaviors. When abnormal behavior is detected, the system can issue an alarm or provide feedback to guide the operators to take corresponding measures.

[0054] Optionally, the data collected by the intelligent monitoring system is analyzed to evaluate the performance and accuracy of the job identification network. Based on the analysis results, the job identification network is optimized and adjusted to improve its identification effect. By continuously collecting data, optimizing the job identification network and improving the intelligent monitoring system, more accurate identification and more effective prevention of abnormal behavior of operators can be achieved.

[0055] From the above content, it can be seen that the intelligent monitoring system can effectively identify the abnormal behavior of operators by using the operation identification network, and improve the safety and efficiency of substation operations. At the same time, this method can also provide valuable data support and decision-making basis for the operation and maintenance management of substations.

[0056] In an optional embodiment, the intelligent monitoring system determines the video image features of the substation based on the abnormal behavior data set, wherein the video image features are used to describe the motion and posture features of historical operating personnel, and the video image features are input into the target branch to obtain two-dimensional operating posture data, wherein the target branch is used to learn and judge the human posture information in the video based on the video image features, and then map the two-dimensional operating posture data in three-dimensional space to obtain three-dimensional operating posture data, and finally perform classification and recognition based on the three-dimensional operating posture data, wherein the classification and recognition is used to identify the operating status of historical operating personnel.

[0057] Optionally, the intelligent monitoring system extracts the behavior video of the operator from the abnormal behavior data set, and then the operation recognition network uses VGG19 (Visual Geometry Group Network with 19 layers) to extract video image features, and uses the video image features as input features, and inputs them into the two branches of PAFs (Part Affinity Fields) and PCM (Part confidence map). Each stage outputs a loss function, and each loss function is used to evaluate the prediction results during the training process and optimize them. Then, the joint point coordinates of the two-dimensional posture extracted from the video are mapped to the three-dimensional space to obtain the three-dimensional operation posture, and then the three-dimensional operation posture is classified and recognized based on the support vector machine.

[0058] Optionally, VGG19 is a convolutional neural network for object recognition. Each layer of the neural network uses the output of the previous layer to further extract more complex features until it is complex enough to be used to recognize objects, so each layer can be regarded as an extractor of many local features. In this way, VGG19 can effectively extract features from images and be used for various visual recognition tasks, including object recognition, scene classification, image segmentation, etc. In the scenario of substation operation behavior recognition, VGG19 can help the intelligent monitoring system extract the key features of the operator's behavior and then identify potential abnormal behavior.

[0059] Optionally, PCM is a heat map of key points, which is used to characterize the positions of key points. Assuming that 18 key points of the human body need to be output, PCM will output 19 channels, with the last channel as the background. In theory, there is no effect if the background is not output, but there are two advantages to outputting the background. First, it adds supervision information, which is beneficial to the learning of the network. Second, the background output continues to be used as the input of the next stage, which is beneficial to the next stage to obtain better semantic information. Through PCM, the intelligent monitoring system can more accurately identify the key points of the human body of the operator, thereby realizing in-depth analysis of the operation behavior and detection of abnormal behavior.

[0060] Optionally, PAFs (Part Affinity Fields) are used to describe the affinity between different key points and to link key points to individuals. PAFs quantify the affinity between key points through numerical values. The larger the value, the tighter the connection between the two key points. PAFs not only provide the connection strength between key points, but also provide the direction information of the connection, which helps to determine the actual bone connection order between key points, and can also distinguish different joints belonging to the same person and between different people, thereby realizing the recognition and distinction of individuals.

[0061] Optionally, FPN (Feature Pyramid Network) is added to VGG19. FPN is an effective method for extracting multi-scale features. It allows the network to simultaneously learn semantic information and spatial information at different levels. Combining FPN with VGG19 can improve the ability to extract video image features, especially in tasks involving human posture estimation.

[0062] Optionally, FPN is used to extract video image features, which are then fused with the video image features extracted by VGG19, and finally the fused features are input to the next layer including the PAFs and PCM branches. Specifically, some layers of VGG19 are first selected as the starting point of FPN. For example, FPN chooses to start from the Conv3_4 layer, which means that FPN is built from the fourth sublayer of the third convolution layer of VGG19. For the selected VGG layers (Conv3_4, Conv4_4, Conv5_4), 1x1 convolution is used to reduce or maintain the number of channels to obtain the connection layers M3, M4, and M5. These connection layers will be used as the intermediate feature representations of the feature pyramid. At the very end of VGG19, a fully connected layer FC-4096 (Conv6) is added. The output of this layer will be used as the highest layer M6 of the feature pyramid. In order to make the feature maps of different layers have the same spatial dimension, M4, M5, and M6 are upsampled so that they can be added to the feature maps of the next layer. The upsampled M4, M5, and M6 are added to M3, M4, and M5 respectively. , and obtain the addition layers L3, L4, and L5. These addition layers fuse features at different levels to enhance the expressiveness of the features. 3x3 convolution operations are applied to L3, L4, and L5, respectively, which helps to further extract and refine the features. Feature maps of different scales are output through 3x3 convolution, which are 7x7 (P3), 14x14 (P4), 28x28 (P5), and 56x56 (P6). These prediction layers contain semantic and position information at different levels. Finally, the obtained prediction layers P3, P4, P5, and P6 are fused with the corresponding image features extracted by VGG19. The fused feature maps will contain rich multi-scale features, which are crucial for the subsequent network training and prediction of PAFs and PCM branches. The fused feature maps are provided as input to the next layer of the network containing the two branches of PAFs and PCM for more in-depth feature analysis and human posture estimation.

[0063] From the above content, it can be seen that the intelligent monitoring system can make full use of the deep feature extraction capability of VGG19 and the multi-scale feature fusion capability of FPN through this method of combining FPN and VGG19, thereby improving the accuracy and robustness of human posture estimation, and then effectively identifying the abnormal behavior of operators in substations, improving operational safety and reducing potential risks. This method combines digital twin technology and deep learning to provide a new solution for the intelligent management of substations, and by simulating and identifying abnormal behaviors, it can timely detect and prevent potential dangerous behaviors and ensure the normal operation and safety of substations.

[0064] In an optional embodiment, the intelligent monitoring system simulates the operating status of the equipment in the substation based on the abnormal behavior data set and the digital twin model to obtain the abnormal operation data set of the equipment, and then trains the second neural network based on the abnormal operation data set to obtain the equipment operation identification network.

[0065] Optionally, in a digital twin environment, the intelligent monitoring system simulates the operating status of the substation equipment using a digital twin model based on the abnormal behavior data set of the operators, and generates a corresponding abnormal operation data set, where the abnormal operation data includes but is not limited to the vibration data and sound data of the equipment. The above-mentioned abnormal operation data set is then used to train the equipment operation recognition network.

[0066] Optionally, the intelligent monitoring system selects ResNet (Residual Network) as the basic network structure. ResNet solves the gradient vanishing problem of deep networks by introducing residual learning. A smoothing unit is added after the ResNet classifier to improve the model's learning and generalization capabilities for abnormal operating data. Then, SGD is used as the optimization algorithm to perform end-to-end training of the network. SGD updates network parameters by random sampling, which helps to avoid local minima and speed up training.

[0067] From the above content, it can be seen that through these steps, in the digital twin environment, the intelligent monitoring system can simulate the operating status of substation equipment and use the abnormal behavior data set to train a network model that can identify abnormal operation of equipment, thereby improving the operation and maintenance efficiency and safety of the substation.

[0068] In an optional embodiment, the intelligent monitoring system binds the human joint points in the three-dimensional working posture data with the joint points of the historical workers in the virtual model.

[0069] Optionally, the intelligent monitoring system binds the body joints in the obtained three-dimensional posture data to the joints of the historical operators in the virtual model one by one, so as to synchronize the operating behaviors of the actual operators with those of the operators in the virtual model.

[0070] From the above content, it can be seen that the intelligent monitoring system can synchronize the behaviors of the operators in the virtual model with those in the actual substation by binding the joint points in the actual three-dimensional posture data with the joints in the virtual model one by one. It can more accurately understand and analyze the postures of the operators and provide safer protection for the working environment of the substation.

[0071] In an optional embodiment, the intelligent monitoring system collects physiological data of the operating personnel, wherein the physiological data includes at least heart rate, blood pressure, and body temperature, and then trains a third neural network based on the physiological data to obtain a personnel status recognition network.

[0072] Optionally, the intelligent monitoring system collects the physiological data of the operating personnel, and then trains a multi-layer perceptron based on the physiological data to obtain a personnel status recognition network.

[0073] Optionally, the intelligent monitoring system collects the operating behavior data, physiological data of the operating personnel and the operating data of the substation equipment in real time as input to the operation identification network, the equipment operation identification network and the personnel status identification network, and continuously optimizes the operation identification network, the equipment operation identification network and the personnel status identification network.

[0074] Optionally, the intelligent monitoring system fuses the output results of the operation recognition network, the equipment operation recognition network, and the personnel status recognition network, and inputs them into a support vector machine again to obtain more accurate monitoring results. Specifically, based on the attention mechanism, the outputs of the operation recognition network, the equipment operation recognition network, and the personnel status recognition network are fused, wherein the calculation formulas for the weights of the operation recognition network, the equipment operation recognition network, and the personnel status recognition network are shown in formulas (1)-(3) respectively:

[0075] w1=a1*σ1 2 (1)

[0076] w2=a2*σ2 2 (2)

[0077] w3=a3*σ3 2 (3)

[0078] Among them, a1 is the initial weight of the job recognition network, σ1 2 is the variance of the output of the job recognition network t moments before the current moment; a2 is the initial weight of the job recognition network, σ2 2 is the variance of the output results of the equipment operation recognition network t moments before the current moment; a3 is the initial weight of the personnel status recognition network, σ3 2is the variance of the output results of the personnel status recognition network t moments before the current moment. It should be noted that the initial weights of the above three networks can be obtained by obtaining the relative importance of each network for job monitoring through conventional methods such as hierarchical analysis method.

[0079] From the above content, it can be seen that through the personnel status recognition network, the intelligent monitoring system can effectively monitor the physiological state of the operators, timely discover health problems and risks, and provide operators with a safer and healthier working environment. And constantly optimize the operation recognition network, personnel status recognition network and equipment operation recognition network through real-time data, and fuse the target monitoring results based on the attention mechanism, and then input them into a support vector machine to obtain the final monitoring results. This method can make the monitoring results of the substation more accurate.

[0080] According to an embodiment of the present application, an embodiment of a monitoring device for substation operations is also provided. Figure 2 is a schematic diagram of an optional monitoring device for substation operation according to an embodiment of the present application, such as Figure 2 As shown, the monitoring device for substation operation includes: a first construction unit 201 , a second construction unit 202 , a first determination unit 203 and a second determination unit 204 .

[0081] Optionally, a first construction unit 201 is used to construct a digital twin model of the substation based on the substation data and the equipment data, wherein the substation data includes at least the outline data and the panoramic image of the substation, and the equipment data includes at least the location information, geometric information and interaction information between different devices of multiple devices in the substation, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation; a second construction unit 202 is used to construct a digital twin environment based on the digital twin model and the virtual model of the historical operators of the substation, wherein the digital twin environment is used to simulate and analyze the substation; a first determination unit 203 is used to monitor the target operators in the digital twin environment according to the operation identification network and the equipment operation identification network to obtain preliminary monitoring results, wherein the operation identification network is used to identify the operation behavior type of the target operators, and the equipment operation identification network is used to identify the operation status of the equipment; a second determination unit 204 is used to determine the target monitoring results based on the preliminary monitoring results and the personnel status identification network, wherein the personnel status identification network is used to identify the status of the target operators.

[0082] Optionally, the first construction unit 201 includes: a first construction subunit, a second construction subunit, a first acquisition subunit, a first determination subunit, a second determination subunit, and a third determination subunit. The first construction subunit is used to construct a substation model based on substation data, wherein the substation model is used to virtually represent the three-dimensional entity information of the overall structure of the substation; the second construction subunit is used to construct an initial equipment model based on equipment data, and the initial equipment model is used to virtually represent the three-dimensional entity information of the equipment in the substation; the first acquisition subunit is used to obtain the association parameters between different equipment in the substation; the first determination subunit is used to determine the association relationship between different equipment based on the association parameters, wherein the association relationship is used to represent the interactive relationship between different equipment; the second determination subunit is used to determine the target equipment model based on the association relationship and the initial equipment model; the third determination subunit is used to determine the digital twin model based on the location information of the substation equipment, the substation model, and the target equipment model.

[0083] Optionally, the second construction unit 202 includes: a second acquisition subunit, a first decomposition subunit, and a first processing subunit. The second acquisition subunit is used to acquire the posture of the historical operator; the first decomposition subunit is used to decompose the posture into N human joints, where N is an integer greater than 1; and the first processing subunit is used to associate the N human joints according to the human skeleton connection relationship to obtain the virtual model of the historical operator.

[0084] Optionally, the second construction unit 202 includes: a fourth determination subunit and a third construction subunit. The fourth determination subunit is used to use the historical operation data of the substation and the operation video of the historical operators as the target data; the third construction subunit is used to build the digital twin environment according to the digital twin model, the virtual model and the target data.

[0085] Optionally, the first determination unit 203 includes: a second acquisition subunit, a first generation subunit, a second generation subunit, and a second processing subunit. The second acquisition subunit is used to acquire the operation behavior data of the historical operators; the first generation subunit is used to simulate the operation behavior in the virtual model based on the operation behavior data to generate abnormal behavior of the historical operators; the second generation subunit is used to generate an abnormal behavior data set according to the abnormal behavior; and the second processing subunit is used to train the first neural network according to the abnormal behavior data set to obtain the operation recognition network.

[0086] Optionally, the second processing subunit includes: a first determination module, a first processing module, a second processing module and a third processing module. The first determination module is used to determine the video image features of the substation according to the abnormal behavior data set, wherein the video image features are used to describe the action and posture features of the historical operators; the first processing module is used to input the video image features into the target branch to obtain the data of the two-dimensional operation posture, wherein the target branch is used to learn and judge the human posture information in the video according to the video image features; the second processing module is used to map the two-dimensional operation posture data in the three-dimensional space to obtain the three-dimensional operation posture data; the third processing module is used to perform classification and recognition based on the three-dimensional operation posture data, wherein the classification and recognition is used to identify the operation status of the historical operators.

[0087] Optionally, the first determination unit 203 includes: a third processing subunit and a fourth processing subunit. The third processing subunit is used to simulate the operating state of the equipment in the substation according to the abnormal behavior data set and the digital twin model to obtain the abnormal operation data set of the equipment; the fourth processing subunit is used to train the second neural network according to the abnormal operation data set to obtain the equipment operation recognition network.

[0088] Optionally, the second processing module includes: a first binding submodule, which is used to bind the human body joint points in the three-dimensional working posture data with the joint points of the historical working personnel in the virtual model.

[0089] Optionally, the second determination unit 204 includes: a first acquisition subunit and a fifth processing subunit. The first acquisition subunit is used to collect physiological data of the operator, wherein the physiological data at least includes heart rate, blood pressure, and body temperature; the fifth processing subunit is used to train the third neural network based on the physiological data to obtain a personnel status recognition network.

[0090] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0091] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0094] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0096] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A monitoring method for substation operation, characterized in that: include: A digital twin model of a substation is constructed according to substation data and equipment data, wherein the substation data at least includes outline data and a panoramic image of the substation, and the equipment data at least includes location information, geometric information of multiple devices in the substation, and interaction information between different devices, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation; Building a digital twin environment according to the digital twin model and a virtual model of a historical operator of the substation, wherein the digital twin environment is used to simulate and analyze the substation; In the digital twin environment, the target operator is monitored according to the operation identification network and the equipment operation identification network to obtain preliminary monitoring results, wherein the operation identification network is used to identify the operation behavior type of the target operator, and the equipment operation identification network is used to identify the operation status of the equipment; The target monitoring result is determined according to the preliminary monitoring result and a personnel status recognition network, wherein the personnel status recognition network is used to identify the status of the target operating personnel.

2. The monitoring method for substation operation according to claim 1, characterized in that: The digital twin model of the substation is built based on the substation data and equipment data, including: Constructing a substation model according to the substation data, wherein the substation model is used to virtually represent the three-dimensional entity information of the overall structure of the substation; Building an initial equipment model according to the equipment data, wherein the initial equipment model is used to virtually represent the three-dimensional entity information of the equipment in the substation; Obtaining correlation parameters between different devices in the substation; Determining an association relationship between the different devices according to the association parameter, wherein the association relationship is used to characterize an interaction relationship between the different devices; Determine a target device model according to the association relationship and the initial device model; The digital twin model is determined according to the location information of the equipment in the substation, the substation model and the target equipment model.

3. The monitoring method for substation operation according to claim 1, characterized in that: Before building a digital twin environment according to the digital twin model and the virtual model of the historical operator of the substation, the method includes: Get the posture of historical operators; Decomposing the posture into N human joints, where N is an integer greater than 1; The N human joints are associated according to the human skeleton connection relationship to obtain a virtual model of the historical operator.

4. The monitoring method for substation operation according to claim 1, characterized in that: A digital twin environment is constructed according to the digital twin model and a virtual model of a historical operator of the substation, including: Using the historical operation data of the substation and the operation video of the historical operators as target data; A digital twin environment is constructed according to the digital twin model, the virtual model and the target data.

5. The monitoring method for substation operation according to claim 1, characterized in that: The job identification network is obtained by the following steps: Obtaining the operation behavior data of the historical operators; Simulating the operation behavior in the virtual model based on the operation behavior data to generate abnormal behavior of the historical operator; generating an abnormal behavior data set according to the abnormal behavior; The first neural network is trained according to the abnormal behavior data set to obtain the job recognition network.

6. The monitoring method for substation operation according to claim 5, characterized in that: In the process of training the first neural network according to the abnormal behavior data set to obtain the operation recognition network, the monitoring method for substation operations also includes: Determining video image features of the substation according to the abnormal behavior data set, wherein the video image features are used to describe the action and posture features of the historical operator; Inputting the video image features into a target branch to obtain data of a two-dimensional working posture, wherein the target branch is used to learn and judge human posture information in the video according to the video image features; Mapping the two-dimensional working posture data in three-dimensional space to obtain three-dimensional working posture data; Classification and recognition are performed based on the three-dimensional working posture data, wherein the classification and recognition is used to identify the working status of the historical working personnel.

7. The monitoring method for substation operation according to claim 5, characterized in that: The device operation identification network is obtained by the following steps: Simulating the operating state of the equipment in the substation according to the abnormal behavior data set and the digital twin model to obtain an abnormal operation data set of the equipment; The second neural network is trained according to the abnormal operation data set to obtain the equipment operation recognition network.

8. The monitoring method for substation operation according to claim 6, characterized in that: After mapping the two-dimensional operation posture data in the three-dimensional space to obtain the three-dimensional operation posture data, the monitoring method for substation operation further includes: The human joint points in the three-dimensional working posture data are bound to the joint points of the historical working personnel in the virtual model.

9. The monitoring method for substation operation according to claim 1, characterized in that: The personnel status recognition network is obtained by the following steps: Collecting physiological data of the operator, wherein the physiological data at least includes heart rate, blood pressure, and body temperature; The third neural network is trained based on the physiological data to obtain the personnel status recognition network.

10. A monitoring device for substation operation, characterized in that: include: A first construction unit constructs a digital twin model of the substation according to the substation data and the equipment data, wherein the substation data at least includes the outline data and the panoramic image of the substation, the equipment data at least includes the location information, geometric information of multiple equipment in the substation, and the interaction information between different equipment, and the digital twin model is used to virtually represent the three-dimensional entity information of the substation; A second construction unit constructs a digital twin environment according to the digital twin model and a virtual model of a historical operator of the substation, wherein the digital twin environment is used to simulate and analyze the substation; A first determination unit, in the digital twin environment, monitors the target operator according to the operation identification network and the equipment operation identification network to obtain a preliminary monitoring result, wherein the operation identification network is used to identify the operation behavior type of the target operator, and the equipment operation identification network is used to identify the operation status of the equipment; The second determination unit determines a target monitoring result according to the preliminary monitoring result and a personnel status recognition network, wherein the personnel status recognition network is used to identify the status of the target operator.

Images