System for analyzing potential safety hazards of transformer substation perimeter based on video images

Abstract

The invention provides a system for analyzing potential safety hazards of a transformer substation perimeter based on video images. The system is provided with a monitoring client, an Nginx agency service, a potential safety hazard monitoring application service, an intelligent algorithm analysis service, a video downloading service, a Mysql database, a casting time sequence database, a cache database Redis and supporting hardware. According to the system, intelligent algorithm service is deployed and operated, algorithm analysis service is started, and video downloading and analyzing service and potential safety hazard monitoring alarm are deployed and operated; real-time capturing is carried out on camera monitoring images of the transformer substation perimeter, algorithm analysis is carried out on environmental elements on the captured images, the purpose of automatically detecting potential safety hazard events or potential perimeter safety risks of the transformer substation perimeter is achieved, the detection accuracy and the intelligent level are continuously improved, and the detection efficiency is improved. The method provides more reliable guarantee for safe operation of the transformer substation, and has wide application prospects and important practical significance.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring systems, and in particular to a system for analyzing safety hazards of a transformer substation perimeter based on video images. Background Art

[0002] With the rapid development of the electric power industry, substations, as important bearing nodes of the electric power transmission and distribution network, have great significance for the safety monitoring of their operation; security precautions at the perimeter of substations are the key link to ensure the normal operation of power facilities and prevent illegal intrusions; traditional perimeter security measures, such as personnel inspections and physical barriers, not only have high manpower costs, but also have low efficiency, and are difficult to meet the increasingly complex and changeable security needs; single video surveillance can no longer meet its use requirements. With the development of science and technology and the development of intelligence, it has become particularly important to design an efficient and intelligent perimeter safety hazard monitoring system to become an important part of the electric power industry; for this reason, the applicant proposes a system based on video image analysis of substation perimeter safety hazards according to the use requirements of substation monitoring. When the analysis finds that there are safety hazards in the monitoring area of ​​the substation perimeter, the system will issue a hidden danger alarm information as soon as possible, handle it in time, and ensure the safe operation of the substation. Summary of the invention

[0003] To solve the above technical problems, the present invention proposes a system for analyzing the safety hazards of the substation perimeter based on video images, by setting up a monitoring client, Nginx proxy service, perimeter safety hazard monitoring application service, intelligent algorithm analysis service, video recording download service, Mysql database, Taosi time series database, cache database Redis and supporting hardware; the system deploys and runs the intelligent algorithm service, executes the startup algorithm analysis service, deploys and runs the video recording download parsing service and safety hazard monitoring alarm; realizes real-time capture of the substation perimeter camera monitoring screen, and performs algorithm analysis on the environmental elements on the captured image, so as to achieve the purpose of automatically detecting the substation perimeter safety hazard events or potential perimeter safety risks, continuously improve the accuracy and intelligence level of detection, provide more reliable protection for the safe operation of the substation, and has broad application prospects and important practical significance.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] A system for analyzing the safety hazards of the perimeter of a substation based on video images, characterized in that: the system for analyzing the safety hazards of the perimeter of a substation based on video images is provided with a monitoring client, an Nginx proxy service, a perimeter safety hazard monitoring application service, an intelligent algorithm analysis service, a video recording download service, a Mysql database, a Taosi time series database, a cache database Redis and supporting hardware; the system captures the image of the camera video screen in real time, adopts image preprocessing technology to remove noise from the image, enhance image quality and extract key features, intelligently analyzes environmental elements on the image, analyzes whether there are safety hazards in the perimeter area monitored by the camera of the substation, analyzes the hazard information and the time when the hazard occurs, and then assembles the hazard message and publishes it to the EMQ message bus, the video recording download service and the perimeter safety hazard monitoring system subscribe to the perimeter safety hazard data from the EMQ message bus, the former intercepts the video within the appropriate time range according to the information contained in the subscription data, stores and publishes the message, and the latter subscribes to the message to process and store the hazard data, pushes an alarm message to the client for the hazard determined to be true, and the client will pop up a window to prompt the alarm, voice broadcast the alarm and display the real-time video monitoring screen.

[0006] Furthermore, the specific steps of the internal data flow of the system for analyzing the potential safety hazards of the substation perimeter based on video images are as follows:

[0007] Step 1: Deploy and run the intelligent algorithm service; configure the camera, video point, measurement point algorithm, aggregation and message reporting, and monitoring tasks in the algorithm analysis configuration system;

[0008] Step 2: Execute the start algorithm analysis task; the algorithm analysis service captures the camera screen image in real time, captures the video screen once every 1 second, and uses image preprocessing technology to remove noise, enhance image quality and extract key features, and intelligently analyze the security environment elements on the image. The intelligent algorithm analysis service supports continuous capture and analysis of 8-channel camera signal screens at the same time. The types of perimeter safety hazards analyzed include illegal intrusion by personnel, floating foreign objects and perimeter fire hazards. If these types of safety hazards are found in the substation perimeter, they will be immediately processed and assembled into hazard data messages and published to the EMQ message bus. The data information includes camera information, video point information, captured pictures, hazard location, hazard type, time of hazard discovery and hazard confidence, etc.

[0009] Step 3, deploy and run the video download service; the video download service subscribes to the perimeter safety hazard data from the EMQ message bus, intercepts the video within the appropriate time range according to the information contained in the data and stores it locally on the video download server, so that the client can query the video file, review the video and download the video clip; deploy and run the perimeter safety hazard monitoring service, the perimeter safety hazard monitoring service subscribes to the perimeter safety hazard data from the EMQ message bus, parses the data structure after subscribing to the message, and according to the "confidence" of the safety hazard data, this "confidence" is the system operation configuration parameter, and the parameter value is set to 70%. When the parsed confidence value is greater than or equal to 70%, the perimeter safety alarm data is judged to be true. At this time, the perimeter safety hazard alarm data will be immediately sent to the monitoring client through Websocket, and then continue to process and store it as alarm record data. The real-time data is entered into the Mysql database, and at the same time, a historical alarm record data is entered into the Taosi time series database;

[0010] Step 4: Safety hazard monitoring alarm; when the monitoring client receives the substation perimeter safety hazard alarm data, it will immediately pop up a window to prompt the perimeter safety hazard monitoring alarm information, link the video to display the camera screen of the real-time monitoring perimeter location, and be accompanied by a voice alarm broadcast; the monitoring client system page also provides perimeter hazard record query, which can view the snapshot of the hazard, review the video screen of the relevant time period, download the video, and view the detailed information of the hazard. In this way, the monitoring client operation and maintenance personnel can more clearly understand the surrounding environment of the substation and the potential risks in the system warning power grid, provide decision support for dispatchers, and improve dispatch efficiency and accuracy.

[0011] Furthermore, the steps for using the system for analyzing the potential safety hazards of the substation perimeter based on video images are as follows:

[0012] Step 1: Connect the cameras installed on the perimeter of the substation to the hard disk recorder;

[0013] Step 2: Connect the real-time monitoring of the hard disk recorder and the camera to the perimeter safety hazard monitoring system to obtain the real-time monitoring data of the perimeter of the video surveillance substation;

[0014] Step 3: The intelligent algorithm analysis service captures the camera screen image in real time, analyzes the environmental factors on the image through the intelligent algorithm, and obtains the safety hazard data of the substation perimeter; the intelligent algorithm analysis service supports 8-channel camera video signal screen capture and continuous analysis at the same time. If the substation perimeter safety hazard data is analyzed, the relevant hazard data will be immediately published to the EMQ message bus, and the information will include camera, image, location and confidence parameters;

[0015] Step 4: The video download service subscribes to the perimeter security risk data from the EMQ message bus, extracts the video within the appropriate time range, stores it, and publishes messages for the client to review, query, and download the video;

[0016] Step 5: The perimeter safety hazard monitoring system subscribes to perimeter safety hazard messages and video information messages from the EMQ message bus, and determines whether the perimeter safety alarm data is true based on the confidence value of the safety hazard data. If true, the alarm data is stored and the alarm message is immediately pushed to the monitoring client;

[0017] Step 6. When the monitoring client receives the substation perimeter safety hazard alarm message, a pop-up window will immediately prompt the alarm information and the camera image of the real-time monitoring perimeter location, accompanied by a voice alarm broadcast; the client system provides hazard record query, which can view snapshots of the hazard, review the video images of the relevant time period, download the video, and view detailed information of the hazard.

[0018] Furthermore, the intelligent algorithm analysis service of the system for analyzing the potential safety hazards of the substation perimeter based on video images has a denoising function, and the specific process is as follows:

[0019] (1) Definition of the denoising autoencoder DAE model, the encoder and decoder structures. The encoder compresses the input into the latent space, while the decoder restores it to the input space.

[0020] (2) Forward propagation: the input data is processed by the encoder and decoder, and the restored image is output;

[0021] (3) Simulate and generate random training image data;

[0022] (4) Create noisy data by adding disturbances through Gaussian noise;

[0023] (5) Use DataLoader to create a batch data set to load data;

[0024] (6) Model training: training is performed within 50 epochs, calculating the reconstruction loss and updating the model weights;

[0025] (7) Denoising: During the testing phase, the trained model is used to denoise the noisy data;

[0026] (8) Visualize the noisy data, original data, and denoised data for comparison.

[0027] Furthermore, the operation process of the video recording download service of the system for analyzing the potential safety hazards of the substation perimeter based on video images is as follows:

[0028] (1) Analyze the message to obtain the camera information and time information of the perimeter hidden danger;

[0029] (2) Initialize the NVR connection;

[0030] (3) User registration device information;

[0031] (4) Video file search;

[0032] (5) Download video files according to time;

[0033] (6) After downloading is complete, stop downloading and save the video file;

[0034] (7) User logout device information;

[0035] (8) Release resources and push video information to the EMQ message bus.

[0036] Furthermore, the autoencoder training process of the intelligent algorithm analysis service of the system for analyzing safety hazards of substation perimeter based on video images is specifically as follows:

[0037] (1) After the first autoencoder is trained, the parameters and the results of the intermediate hidden layer are fixed, the output layer and the corresponding weight bias are removed, and then the hidden layer is used as the input of the second autoencoder;

[0038] (2) After the second autoencoder is trained, the parameters and the results of the middle hidden layer are fixed, the output layer and the corresponding weight bias are removed, and then the hidden layer is used as the input of the third autoencoder;

[0039] (3) After the third autoencoder is trained, the parameters and the results of the middle hidden layer are fixed, and the output layer and the corresponding weight bias are removed;

[0040] (4) Add a classifier after the third autoencoder hidden layer for decoding recovery;

[0041] (5) After the three-layer autoencoder training is completed, the last step is the overall reverse tuning training, that is, the entire network is trained using back propagation to fine-tune the parameters.

[0042] Furthermore, the mapping and restoration process of the autoencoder training of the intelligent algorithm analysis service of the system for analyzing safety hazards of substation perimeters based on video images is to represent the training sample x~ as a red cross located near the low-dimensional manifold; the gray circle represents the equally probable damage process C(x~|x); the gray arrow demonstrates how to convert a training sample into a sample that has undergone this damage process; when the denoising autoencoder is trained to minimize the average value of the square error ||g(f(x~))-x||2, the reconstruction g(f(x~)) estimates Ex,x~P~data~(x)C(x~|x)[x|x~]; g(f(x~)) estimates the center of mass of the original point x that may produce x~, so the vector g(f(x~))-x~ approximately points to the nearest point on the manifold; therefore, the autoencoder can learn the vector field g(f(x))-x represented by the green arrow; the vector field estimates the score is the multiplication factor, i.e., the average root mean square reconstruction error

[0043] Furthermore, the autoencoder cost function of the intelligent algorithm analysis service of the system for analyzing safety hazards of substation perimeters based on video images is: The denoising autoencoder is trained to reconstruct a clean data point x from a damaged version xˉ. The denoising autoencoder introduces a damage process C(xˉ|x), and this conditional distribution represents the probability of generating a damaged sample xˉ for a given data sample x; the autoencoder learns to reconstruct the distribution p from the training data pair (x|xˉ) according to the following process: reconstruct (x|xˉ):

[0044] (1) Take a training sample x from the training data;

[0045] (2) Take a damaged sample xˉ from C(xˉ|x=x);

[0046] (3) Use (x|xˉ) as a training sample to estimate the reconstruction distribution of the autoencoder; p reconstruct (x|xˉ)=p decoder (x|h)p reconstruct (x|xˉ)=p decoder (x|h)

[0047] Where: h is the output of the encoder f(x);

[0048] p decoder Defined in terms of the decoding function g(h).

[0049] Usually we can simply negative the log-likelihood -log p decoder(x|h=f(xˉ)) performs approximate minimization based on the gradient method; as long as the encoder is deterministic, the denoising autoencoder is a feedforward neural network and can be trained in exactly the same way as other feedforward neural networks; the denoising autoencoder performs stochastic gradient descent under the expectation shown in the following equation:

[0050] -Exˉpˉ ddata (x)ExˉC(xˉ|x)log p decoder (x|h=f(xˉ))

[0051] In order to make the error between the reconstructed data and the original undamaged data as small as possible, the goal of the denoising autoencoder is to minimize the loss function as follows:

[0052] J DAE (W)=ΣExˉ~qD(xˉ|x)[L(x,y)].

[0053] Furthermore, the parameter calculation formula for updating the encoder and decoder of the intelligent algorithm analysis service of the system for analyzing the potential safety hazards of the substation perimeter based on video images is:

[0054]

[0055] The benefits of this application are:

[0056] 1. Improve monitoring efficiency and accuracy;

[0057] The system based on video image analysis for substation perimeter safety hazards can provide reliable monitoring around the clock: by embedding or deploying intelligent video analysis components in the front-end system, it can realize real-time and uninterrupted analysis of the monitored perimeter area images, which completely changes the previous mode of monitoring and analyzing the monitoring images entirely by staff; higher accuracy of perimeter safety alarms, the front-end system integrates powerful image processing capabilities, and runs mature intelligent algorithm analysis, so that users can more accurately define the characteristics of security threats, effectively reduce false alarms and missed alarms, and reduce the amount of invalid data;

[0058] 2. Improve response speed and security;

[0059] The system for analyzing the safety hazards of the substation perimeter based on video images has a faster system response speed, can more accurately identify the safety factors of the monitoring perimeter hazards, and can prompt monitoring personnel to pay attention to the relevant monitoring images before the security threat occurs, so as to prepare in advance; the system can also enable users to more accurately define the actions to be taken when a specific security threat occurs, and the monitoring system itself can ensure that the crisis handling steps can be accurately executed according to the predetermined plan, effectively preventing delays caused by human factors in the chaos; enhance security, and automatically detect safety hazards of the substation perimeter, such as illegal intrusion of personnel, floating foreign objects blown by strong winds, and fire safety hazards outside the perimeter.

[0060] And alarm to achieve the purpose of real-time monitoring and alarm;

[0061] 3. Greatly optimize resource utilization and reduce costs;

[0062] The system based on video image analysis of substation perimeter safety hazards only intercepts, downloads and stores video recordings for viewing alarm screens, because abnormal behavior alarm videos only account for a very small part of the total. This can greatly reduce the transmission bandwidth and storage pressure of master station systems at all levels, which happen to be the bottlenecks of today's power video monitoring systems. Through video image intelligent algorithm analysis, potential substation perimeter safety hazards can be discovered and warned in a timely manner, thereby reducing equipment damage and maintenance costs caused by faults. At the same time, since the system can automatically identify and alarm, the frequency and intensity of manual inspections are also reduced, greatly reducing labor costs.

[0063] 4. Promote visual monitoring and dispatching of power grids;

[0064] The system for analyzing the safety hazards of substation perimeter based on video images The perimeter safety hazard monitoring system based on video image analysis can provide an intuitive visual interface for the power grid, allowing operation and maintenance personnel to have a clearer understanding of the surrounding environment of the substation; through the intelligent algorithm analysis of video images, the system can automatically judge and warn of potential risks in the power grid, provide decision support for dispatchers, and improve dispatch efficiency and accuracy;

[0065] 5. Improve the intelligence level of substations;

[0066] The system for analyzing safety hazards around substations based on video images is combined with artificial intelligence algorithms and big data technologies. This method can continuously perform machine learning and optimization to achieve a more intelligent management method, better adapt to the needs of substation safety management, and improve the intelligence level of substations. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of the system architecture of the present invention;

[0068] Figure 2 This is a schematic diagram of data flow in the present invention;

[0069] Figure 3 This is a schematic diagram of the configuration of the algorithm analysis service of the present invention;

[0070] Figure 4 This is a schematic diagram of real-time monitoring and alarming of perimeter safety hazards of the present invention;

[0071] Figure 5 This is a schematic diagram of the perimeter safety hazard record of the present invention Figure 1 ;

[0072] Figure 6 This is a schematic diagram of some programming codes of the present invention Figure 1 ;

[0073] Figure 7 This is a schematic diagram of some programming codes of the present invention Figure 2 ;

[0074] Figure 8 This is a schematic diagram of some programming codes of the present invention Figure 3 ;

[0075] Fig. 9 Schematic diagram of the neural network model of the autoencoder of the present invention;

[0076] Fig.10 Schematic diagram of the training process of the autoencoder of the present invention;

[0077] Fig.11 This is a schematic diagram of the network structure of the data training autoencoder of the present invention;

[0078] Fig.12 This is a schematic diagram of the denoising autoencoder training mapping restoration of the present invention;

[0079] Fig.13 This is an example diagram of the calculation of the cost function of the denoising autoencoder of the present invention;

[0080] Fig.14 This is an example diagram of the denoising autoencoder simulation training of the present invention. DETAILED DESCRIPTION

[0081] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments:

[0082] like Figure 1As shown, it is a schematic diagram of the architecture of a system for analyzing the safety hazards of the substation perimeter based on video images. The system for analyzing the safety hazards of the substation perimeter based on video images is provided with a monitoring client, Nginx proxy service, perimeter safety hazard monitoring application service, intelligent algorithm analysis service, video recording download service, Mysql database, Taosi time series database, cache database Redis and supporting hardware; the system captures the image of the camera video screen in real time, adopts image preprocessing technology to remove noise, enhance image quality and extract key features, intelligently analyzes the environmental factors on the image, and analyzes whether the perimeter area monitored by the camera of the substation is safe. Hidden dangers, analyze the hidden danger information and the time when the hidden danger occurs, then assemble the hidden danger message and publish it to the EMQ message bus. The video download service and the perimeter safety hidden danger monitoring system subscribe to the perimeter safety hidden danger data from the EMQ message bus. The former intercepts the video within the appropriate time range according to the information contained in the subscribed data, stores and publishes the message. The latter subscribes to the message to process and store the hidden danger data. For the hidden dangers that are determined to be true, the alarm message is pushed to the client. The client will pop up an alarm, voice broadcast the alarm and display the real-time video monitoring screen; the system for analyzing the perimeter safety hidden dangers of the substation based on the video screen capture can be widely used in the perimeter safety hidden danger monitoring of various substations, including but not limited to:

[0083] (1) High-voltage substation: Real-time safety hazard monitoring and control of the high-voltage equipment area and surrounding environment;

[0084] (2) Unmanned substation: In the case of no one on duty, timely detection and handling of security incidents can be achieved through remote monitoring;

[0085] (3) Important substations: Focus on monitoring and protecting the perimeters of important substations to ensure the safety and reliability of power grid operation;

[0086] With the continuous training and development of image processing algorithms and machine learning technologies, this method will continuously improve the accuracy and intelligence of detection, provide more reliable guarantees for the safe operation of substations, and has broad application prospects and important practical significance.

[0087] like Figure 2 As shown, the specific steps of the internal data flow of the system for analyzing the safety hazards of the substation perimeter based on video images are as follows:

[0088] Step 1: Deploy and run the intelligent algorithm service; configure the camera, video point, measurement point algorithm, aggregation and message reporting, and monitoring tasks in the algorithm analysis configuration system;

[0089] Step 2: Execute the start algorithm analysis task; the algorithm analysis service captures the camera screen image in real time, captures the video screen once every 1 second, and can be configured. By adopting image preprocessing technology, the image is de-noised, the image quality is enhanced, and key features are extracted. The security environment elements on the image are intelligently analyzed. The intelligent algorithm analysis service supports continuous capture and analysis of 8-channel camera signal screens at the same time. The types of perimeter security hazards analyzed include illegal intrusion by personnel, floating foreign objects, and perimeter fire hazards. If these types of security hazards are found in the substation perimeter, they will be immediately processed and assembled into hazard data messages and published to the EMQ message bus. The data information includes camera information, video point information, captured pictures, hazard location, hazard type, time of discovery, and hazard confidence.

[0090] Step 3, deploy and run the video download service; the video download service subscribes to the perimeter safety hazard data from the EMQ message bus, intercepts the video within the appropriate time range according to the information contained in the data and stores it locally on the video download server, so that the client can query the video file, review the video and download the video clip; deploy and run the perimeter safety hazard monitoring service, the perimeter safety hazard monitoring service subscribes to the perimeter safety hazard data from the EMQ message bus, parses the data structure after subscribing to the message, and according to the "confidence" of the safety hazard data, this "confidence" is the system operation configuration parameter, and the parameter value is set to 70%. When the parsed confidence value is greater than or equal to 70%, the perimeter safety alarm data is judged to be true. At this time, the perimeter safety hazard alarm data will be immediately sent to the monitoring client through Websocket, and then continue to process and store it as alarm record data. The real-time data is entered into the Mysql database, and at the same time, a historical alarm record data is entered into the Taosi time series database;

[0091] Step 4: Safety hazard monitoring alarm; when the monitoring client receives the substation perimeter safety hazard alarm data, it will immediately pop up a window to prompt the perimeter safety hazard monitoring alarm information, link the video to display the camera screen of the real-time monitoring perimeter location, and be accompanied by a voice alarm broadcast; the monitoring client system page also provides perimeter hazard record query, which can view the snapshot of the hazard, review the video screen of the relevant time period, download the video, and view the detailed information of the hazard. In this way, the monitoring client operation and maintenance personnel can more clearly understand the surrounding environment of the substation and the potential risks in the system warning power grid, provide decision support for dispatchers, and improve dispatch efficiency and accuracy.

[0092] The specific steps of using the system for analyzing the safety hazards of the substation perimeter based on video images are as follows:

[0093] Step 1: Connect the cameras installed on the perimeter of the substation to the hard disk recorder. The cameras are connected to the hard disk recorder through the RTSP protocol, store video recording information according to the specification requirements, and provide video data for the video download service;

[0094] Step 2: Connect the real-time monitoring of the hard disk recorder and the camera to the perimeter safety hazard monitoring system to obtain the real-time monitoring data of the perimeter of the video surveillance substation;

[0095] Step 3: The intelligent algorithm analysis service captures the camera screen image in real time, analyzes the environmental factors on the image through the intelligent algorithm, and obtains the safety hazard data of the substation perimeter; the intelligent algorithm analysis service supports 8-channel camera video signal screen capture and continuous analysis at the same time. If the substation perimeter safety hazard data is analyzed, the relevant hazard data will be immediately published to the EMQ message bus, and the information will include camera, image, location and confidence parameters;

[0096] Step 4: The video download service subscribes to the perimeter security risk data from the EMQ message bus, extracts the video within the appropriate time range, stores it, and publishes messages for the client to review, query, and download the video;

[0097] Step 5. The perimeter safety hazard monitoring system subscribes to perimeter safety hazard messages and video information messages from the EMQ message bus, and determines whether the perimeter safety alarm data is true based on the confidence value of the safety hazard data. If it is true, the alarm data is stored and the alarm message is immediately pushed to the monitoring client; based on the confidence of the safety hazard data, it determines whether the perimeter safety hazard alarm data is true. The confidence is the system operation parameter configuration, and it is true when the value is >= 70%; if it is true, it is stored as alarm record data, and the perimeter safety hazard alarm message is immediately pushed to the monitoring client through Websocket

[0098] Step 6. When the monitoring client receives the substation perimeter safety hazard alarm message, a pop-up window will immediately prompt the alarm information and the camera image of the real-time monitoring perimeter location, accompanied by a voice alarm broadcast; the client system provides hazard record query, which can view snapshots of the hazard, review the video images of the relevant time period, download the video, and view detailed information of the hazard.

[0099] like Figure 3 As shown, it is a partial screenshot of the image analysis and aggregation system WEB display after the intelligent algorithm analysis service is deployed and run. Here you can configure the algorithm analysis connection camera, video point configuration, measurement point algorithm configuration, aggregation and message reporting configuration and monitoring task configuration. After starting the monitoring task, the page can display the monitoring result information in real time.

[0100] like Figure 4 , Figure 5As shown, a partial screenshot of the perimeter safety hazard monitoring system is shown. This figure illustrates the video polling monitoring screen, hazard record query page, and real-time perimeter safety hazard alarm prompt pop-up window of the perimeter safety hazard system monitored by the client. When the monitoring center personnel see or hear the alarm message, they will promptly notify the security management personnel or relevant departments, thereby providing decision support for the dispatching personnel and improving the dispatching efficiency and accuracy.

[0101] like Figure 6 As shown, the diagram shows part of the code for intelligent algorithm analysis. The intelligent algorithm analysis service of the system for analyzing the potential safety hazards of the substation perimeter based on video images has a denoising function. The specific process is as follows:

[0102] (1) Definition of the denoising autoencoder DAE model, the encoder and decoder structures. The encoder compresses the input into the latent space, while the decoder restores it to the input space.

[0103] (2) Forward propagation: the input data is processed by the encoder and decoder, and the restored image is output;

[0104] (3) Simulate and generate random training image data;

[0105] (4) Create noisy data by adding disturbances through Gaussian noise;

[0106] (5) Use DataLoader to create a batch data set to load data;

[0107] (6) Model training: training is performed within 50 epochs, calculating the reconstruction loss and updating the model weights;

[0108] (7) Denoising: During the testing phase, the trained model is used to denoise the noisy data;

[0109] (8) Visualize the noisy data, original data, and denoised data for comparison.

[0110] like Figure 7 As shown, part of the code of the video download service is shown. After the service subscribes to the perimeter security hazard message from the EMQ message bus, it parses the message data for business processing. The operation process is as follows:

[0111] (1) Analyze the message to obtain the camera information and time information of the perimeter hidden danger;

[0112] (2) Initialize the NVR connection;

[0113] (3) User registration device information;

[0114] (4) Video file search;

[0115] (5) Download video files according to time;

[0116] (6) After downloading is complete, stop downloading and save the video file;

[0117] (7) User logout device information;

[0118] (8) Release resources and push video information to the EMQ message bus.

[0119] like Figure 8 As shown, the code of the perimeter safety hazard monitoring service is shown. After the service subscribes to the perimeter safety hazard message and video information from the EMQ message bus, it parses the message data for business processing; the specific steps include:

[0120] (1) Determine whether the alarm message is true based on the “confidence” of the potential safety hazard data and the operating parameters of the system operation configuration;

[0121] (2) Store real-time alarm data in the MySQL database, store historical alarm data in the TDengine time series database, and store real-time alarm data in the Redis cache to remove duplicate alarm data within a short period of time;

[0122] (3) Send perimeter security hazard alarm data to the monitoring client via Websocket.

[0123] like Fig. 9 As shown in the figure, the neural network model structure of the autoencoder of the intelligent algorithm analysis service is shown. It consists of two parts: the encoder (Ecoder) and the decoder (Decoder). The former is used to encode the input data (Input Data), thereby mapping the input data to a latent space (Latent Space) with a lower dimension to obtain the encoded data (EncodedData), and the latter is used to restore (decode) the encoded data in the latent space into "input data.

[0124] like Fig.10 The figure shows the schematic diagram of the training process of the intelligent algorithm analysis service autoencoder. The specific process is as follows:

[0125] (1) The first autoencoder, 899->400->899, after training, fix the parameters and the results of the middle hidden layer, remove the output layer and the corresponding weight bias, and then use the hidden layer 400 as the input of the second autoencoder;

[0126] (2) The second autoencoder, 400->200->400, after training, fix the parameters and the results of the middle hidden layer, remove the output layer and the corresponding weight bias, and then use the hidden layer 200 as the input of the third autoencoder;

[0127] (3) The third autoencoder, 200->400->200, after training, fix the parameters and the results of the middle hidden layer, and remove the output layer and the corresponding weight bias;

[0128] (4) Add a classifier after the third autoencoder hidden layer for decoding recovery;

[0129] (5) After the three-layer autoencoder training is completed, the last step is the overall reverse tuning training, that is, the entire network is trained using back propagation and the parameters are fine-tuned.

[0130] like Fig.11 As shown in the figure, it is an example diagram of the network structure of the autoencoder for intelligent algorithm analysis service data training. From the figure, we can see that first we have multiple dimensions (n) of input Xn passing through the encoder, and then generate multiple dimensions of compressed feature vectors. The decoder then restores these multiple dimensions of feature vectors to multiple dimensions (n) of X'n, and then outputs them. The purpose of the whole process is only one: the output X'n is infinitely close to the input Xn. For an autoencoder, there will be 3 hyperparameters that need to be set: the number of neurons of the compressed feature vector, that is, the dimension after compression. The fewer the number of neurons, the heavier the compression; the number of layers. The structure of the autoencoder can be set arbitrarily according to our ideas; the number of neurons in each layer.

[0131] like Fig.12 As shown, the mapping and restoration process of the autoencoder training for the intelligent algorithm analysis service of the system for analyzing safety hazards of the substation perimeter based on video images is shown. The training sample x~ is represented as a red cross located near the low-dimensional manifold; the gray circle represents the equally probable damage process C(x~|x); the gray arrow demonstrates how to convert a training sample into a sample that has undergone this damage process; when the denoising autoencoder is trained to minimize the average value of the square error ||g(f(x~))-x||2, the reconstruction g(f(x~)) estimates Ex,x~P~data~(x)C(x~|x)[x|x~]; g(f(x~)) estimates the center of mass of the original point x that may produce x~, so the vector g(f(x~))-x~ approximately points to the nearest point on the manifold; therefore, the autoencoder can learn the vector field g(f(x))-x represented by the green arrow; the vector field estimates the score is the multiplication factor, i.e., the average root mean square reconstruction error

[0132] like Fig.13As shown, the cost function of the autoencoder for the intelligent algorithm analysis service of the system for analyzing safety hazards of substation perimeters based on video images is: The denoising autoencoder is trained to reconstruct a clean data point x from a damaged version xˉ. The denoising autoencoder introduces a damage process C(xˉ|x), and this conditional distribution represents the probability of generating a damaged sample xˉ for a given data sample x; the autoencoder learns to reconstruct the distribution from the training data pair (x|xˉ) according to the following process:

[0133] p reconstruct (x|xˉ):

[0134] (1) Take a training sample x from the training data;

[0135] (2) Take a damaged sample xˉ from C(xˉ|x=x);

[0136] (3) Use (x|xˉ) as a training sample to estimate the reconstruction distribution of the autoencoder; p reconstruct (x|xˉ)=p decoder (x|h)p reconstruct (x|xˉ)=p decoder (x|h)

[0137] Where: h is the output of the encoder f(x);

[0138] p decoder Defined in terms of the decoding function g(h).

[0139] Usually we can simply negative the log-likelihood -log p decoder (x|h=f(xˉ)) performs approximate minimization based on the gradient method; as long as the encoder is deterministic, the denoising autoencoder is a feedforward neural network and can be trained in exactly the same way as other feedforward neural networks; the denoising autoencoder performs stochastic gradient descent under the expectation shown in the following equation:

[0140] -Exˉpˉ ddata (x)ExˉC(xˉ|x)log p decoder (x|h=f(xˉ))

[0141] In order to make the error between the reconstructed data and the original undamaged data as small as possible, the goal of the denoising autoencoder is to minimize the loss function as follows:

[0142] J DAE (W)=ΣExˉ~qD(xˉ|x)[L(x,y)].

[0143] like Fig.14As shown, the parameter calculation formula for updating the encoder and decoder of the intelligent algorithm analysis service of the system for analyzing the safety hazards of the substation perimeter based on video images is as follows:

[0144]

[0145] The above description is only a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent change made based on the technical essence of the present invention still falls within the scope of protection required by the present invention.

Claims

1. A system for analyzing potential safety hazards of substation perimeter based on video images, characterized in that: The system based on video image analysis of substation perimeter safety hazards is provided with a monitoring client, Nginx proxy service, perimeter safety hazard monitoring application service, intelligent algorithm analysis service, video recording download service, Mysql database, Taosi time series database, cache database Redis and supporting hardware; the system captures the image of the camera video screen in real time, adopts image preprocessing technology to remove noise, enhance image quality and extract key features, intelligently analyzes environmental factors on the image, analyzes whether there are safety hazards in the perimeter area monitored by the camera of the substation, analyzes the hazard information and the time when the hazard occurs, and then assembles the hazard message and publishes it to the EMQ message bus, the video recording download service and the perimeter safety hazard monitoring system subscribe to the perimeter safety hazard data from the EMQ message bus, the former intercepts the video within the appropriate time range according to the information contained in the subscription data, stores and publishes the message, and the latter subscribes to the message to process and store the hazard data, and pushes an alarm message to the client for the hazard determined to be true, and the client will pop up a window to prompt the alarm, voice broadcast the alarm and display the real-time video monitoring screen.

2. A system for analyzing potential safety hazards of substation perimeter based on video images according to claim 1, characterized in that: The specific steps of the internal data flow of the system based on video image analysis of substation perimeter safety hazards are as follows: Step 1: Deploy and run the intelligent algorithm service; configure the camera, video point, measurement point algorithm, aggregation and message reporting, and monitoring tasks in the algorithm analysis configuration system; Step 2: Execute the start algorithm analysis task; the algorithm analysis service captures the camera screen image in real time, captures the video screen once every 1 second, and uses image preprocessing technology to remove noise, enhance image quality and extract key features, and intelligently analyze the security environment elements on the image. The intelligent algorithm analysis service supports continuous capture and analysis of 8-channel camera signal screens at the same time. The types of perimeter safety hazards analyzed include illegal intrusion by personnel, floating foreign objects and perimeter fire hazards. If these types of safety hazards are found in the substation perimeter, they will be immediately processed and assembled into hazard data messages and published to the EMQ message bus. The data information includes camera information, video point information, captured pictures, hazard location, hazard type, time of hazard discovery and hazard confidence, etc. Step 3: Deploy and run the video download service. The video download service subscribes to the perimeter security risk data from the EMQ message bus, extracts the video within the appropriate time range according to the information contained in the data, and stores it locally on the video download server, so that the client can query the video file, access the video, and download the video clip. Deploy and run the perimeter safety hazard monitoring service. The perimeter safety hazard monitoring service subscribes to the perimeter safety hazard data from the EMQ message bus. After subscribing to the message, the data structure is parsed. According to the "confidence" of the safety hazard data, this "confidence" is a system operation configuration parameter, and the parameter value is set to 70%. When the parsed confidence value is greater than or equal to 70%, the perimeter safety alarm data is judged to be true. At this time, the perimeter safety hazard alarm data will be immediately sent to the monitoring client through Websocket, and then continue to process and store it as alarm record data. The real-time data is entered into the Mysql database, and at the same time, a historical alarm record data is entered into the Taosi time series database; Step 4: Safety hazard monitoring alarm; when the monitoring client receives the substation perimeter safety hazard alarm data, it will immediately pop up a window to prompt the perimeter safety hazard monitoring alarm information, link the video to display the camera screen of the real-time monitoring perimeter location, and be accompanied by a voice alarm broadcast; the monitoring client system page also provides perimeter hazard record query, which can view the snapshot of the hazard, review the video screen of the relevant time period, download the video, and view the detailed information of the hazard. In this way, the monitoring client operation and maintenance personnel can more clearly understand the surrounding environment of the substation and the potential risks in the system warning power grid, provide decision support for dispatchers, and improve dispatch efficiency and accuracy.

3. The system for analyzing safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The specific steps of using the system based on video image analysis of substation perimeter safety hazards are as follows: Step 1: Connect the cameras installed on the perimeter of the substation to the hard disk recorder; Step 2: Connect the real-time monitoring of the hard disk recorder and the camera to the perimeter safety hazard monitoring system to obtain the real-time monitoring data of the perimeter of the video surveillance substation; Step 3: The intelligent algorithm analysis service captures the camera image in real time, analyzes the environmental factors on the image through the intelligent algorithm, and obtains the safety hazard data of the substation perimeter; The intelligent algorithm analysis service supports capturing and continuously analyzing the video signals of 8 cameras at the same time. If the substation perimeter safety hazard data is analyzed, the relevant hazard data will be immediately published to the EMQ message bus, and the information will include camera, image, location and confidence parameters; Step 4: The video download service subscribes to the perimeter security risk data from the EMQ message bus, extracts the video within the appropriate time range, stores it, and publishes messages for the client to review, query, and download the video; Step 5: The perimeter safety hazard monitoring system subscribes to perimeter safety hazard messages and video information messages from the EMQ message bus, and determines whether the perimeter safety alarm data is true based on the confidence value of the safety hazard data. If true, the alarm data is stored and the alarm message is immediately pushed to the monitoring client; Step 6. When the monitoring client receives the substation perimeter safety hazard alarm message, a pop-up window will immediately prompt the alarm information and the camera image of the real-time monitoring perimeter location, accompanied by a voice alarm broadcast; the client system provides hazard record query, which can view snapshots of the hazard, review the video images of the relevant time period, download the video, and view detailed information of the hazard.

4. The system for analyzing safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The intelligent algorithm analysis service of the system for analyzing the potential safety hazards of the substation perimeter based on video images has a denoising function, and the specific process is as follows: (1) Definition of the denoising autoencoder DAE model, the encoder and decoder structures. The encoder compresses the input into the latent space, while the decoder restores it to the input space. (2) Forward propagation: the input data is processed by the encoder and decoder, and the restored image is output; (3) Simulate and generate random training image data; (4) Create noisy data by adding disturbances through Gaussian noise; (5) Use DataLoader to create a batch data set to load data; (6) Model training: training is performed within 50 epochs, calculating the reconstruction loss and updating the model weights; (7) Denoising: During the testing phase, the trained model is used to denoise the noisy data; (8) Visualize the noisy data, original data, and denoised data for comparison.

5. The system for analyzing potential safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The operation process of the video recording download service of the system for analyzing the potential safety hazards of the substation perimeter based on video images is as follows: (1) Analyze the message to obtain the camera information and time information of the perimeter hidden danger; (2) Initialize the NVR connection; (3) User registration device information; (4) Video file search; (5) Download video files according to time; (6) After downloading is complete, stop downloading and save the video file; (7) User logout device information; (8) Release resources and push video information to the EMQ message bus.

6. The system for analyzing potential safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The specific process of the autoencoder training of the intelligent algorithm analysis service of the system for analyzing the potential safety hazards of the substation perimeter based on video images is as follows: (1) After the first autoencoder is trained, the parameters and the results of the intermediate hidden layer are fixed, the output layer and the corresponding weight bias are removed, and then the hidden layer is used as the input of the second autoencoder; (2) After the second autoencoder is trained, the parameters and the results of the middle hidden layer are fixed, the output layer and the corresponding weight bias are removed, and then the hidden layer is used as the input of the third autoencoder; (3) After the third autoencoder is trained, the parameters and the results of the middle hidden layer are fixed, and the output layer and the corresponding weight bias are removed; (4) Add a classifier after the third autoencoder hidden layer for decoding recovery; (5) After the three-layer autoencoder training is completed, the last step is the overall reverse tuning training, that is, the entire network is trained using back propagation and the parameters are fine-tuned.

7. The system for analyzing potential safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The mapping and restoration process of the autoencoder training of the intelligent algorithm analysis service of the system for analyzing safety hazards of substation perimeters based on video images is to represent the training sample x~ as a red cross located near the low-dimensional manifold; the gray circle represents the equally probable damage process C(x~|x); the gray arrow demonstrates how to convert a training sample into a sample that has undergone this damage process; when the denoising autoencoder is trained to minimize the average value of the square error ||g(f(x~))-x||2, the reconstruction g(f(x~)) estimates Ex,x~P~data~(x)C(x~|x)[x|x~]; g(f(x~)) estimates the center of mass of the original point x that may produce x~, so the vector g(f(x~))-x~ approximately points to the nearest point on the manifold; therefore, the autoencoder can learn the vector field g(f(x))-x represented by the green arrow; the vector field estimates the score is the multiplication factor, i.e., the average RMS reconstruction error.

8. The system for analyzing potential safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The autoencoder cost function of the intelligent algorithm analysis service of the system for analyzing safety hazards of substation perimeters based on video images is: The denoising autoencoder is trained to reconstruct a clean data point x from a damaged version xˉ. The denoising autoencoder introduces a damage process C(xˉ|x), and this conditional distribution represents the probability of generating a damaged sample xˉ given a data sample x; the autoencoder learns to reconstruct the distribution p from the training data pair (x|xˉ) according to the following process: reconstruct (x|xˉ): (1) Take a training sample x from the training data; (2) Take a damaged sample xˉ from C(xˉ|x=x); (3) Use (x|xˉ) as a training sample to estimate the reconstruction distribution of the autoencoder; p reconstruct (x|xˉ)=p decoder (x|h)p reconstruct (x|xˉ)=p decoder (x|h) Where: h is the output of the encoder f(x); p decoder Defined in terms of the decoding function g(h). Usually we can simply negative the log-likelihood -log p decoder (x|h=f(xˉ)) performs approximate minimization based on the gradient method; as long as the encoder is deterministic, the denoising autoencoder is a feedforward neural network and can be trained in exactly the same way as other feedforward neural networks; the denoising autoencoder performs stochastic gradient descent under the expectation shown in the following equation: -Exˉpˉddata(x)ExˉC(xˉ|x)log p decoder (x|h=f(xˉ)) In order to make the error between the reconstructed data and the original undamaged data as small as possible, the goal of the denoising autoencoder is to minimize the loss function as follows: J DAE (W)=ΣE x ˉ~ qD (xˉ|x)[L(x,y)]。 9. The system for analyzing potential safety hazards of substation perimeter based on video images according to claim 1 is characterized by: The parameter calculation formula for updating the encoder and decoder of the intelligent algorithm analysis service of the system for analyzing the potential safety hazards of the substation perimeter based on video images is:

Images