Intelligent verification method for unplanned operations in substations based on artificial intelligence platform
Through the intelligent verification method of unplanned operations based on the artificial intelligence platform, video stream and audio stream data are used to analyze the construction behavior of the substation, identify unplanned operations and issue warnings, solving the safety hazards caused by unplanned operations on the construction site, and improving verification efficiency and security.
Patent Information
- Application Number
- CN202411117647.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-15
AI Technical Summary
Unplanned operations often occur at the substation construction site, resulting in unclear work tasks, irregular operations, and increased safety hazards. Effective monitoring and verification are required to ensure that the construction behavior is carried out in accordance with the plan.
Adopt an intelligent verification method for unplanned operations based on the artificial intelligence platform. By obtaining substation operation plans, collecting video streams and audio stream data in real time, analyzing the matching degree of operation location, time and content with the plan, identifying unplanned operations and issuing warnings, reducing Complex calculations improve verification efficiency.
It realizes efficient monitoring and verification of unplanned operations of substations, reduces the complex calculation of video streams and audio streams, improves verification efficiency, ensures that construction behavior follows the plan, and reduces safety risks.
Smart Images

Figure CN118627871B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of substation monitoring, and in particular to an intelligent verification method for unplanned operations of a substation based on an artificial intelligence platform. Background Art
[0002] In recent years, unplanned operations have occurred frequently at substation construction sites in various places. Usually, in order to catch up with or speed up the construction progress, they deliberately fail to report plans and arrange operations privately. This unplanned construction behavior may lead to unclear work tasks, non-standard operations, increased safety hazards and other problems. It is necessary to monitor and verify the unplanned construction behavior of substations to ensure that the construction behavior of substations in various places is carried out according to plan. Summary of the invention
[0003] In view of the problems existing in the above-mentioned prior art, the present invention provides an intelligent verification method for unplanned operations of substations based on an artificial intelligence platform. The technical solution is as follows:
[0004] In a first aspect, a method for intelligently checking unplanned operations of a substation based on an artificial intelligence platform is provided, comprising the following steps:
[0005] Obtain substation operation plans from the artificial intelligence platform;
[0006] Determine whether there is any operation behavior at the site according to the monitoring data collected in real time at the substation site, record the monitoring data with the operation behavior as the first data, and determine the real-time operation position corresponding to the operation behavior according to the acquisition device ID of the monitoring data of the first data; the monitoring data includes real-time video stream and audio stream;
[0007] If the real-time operation position does not exist in all the operation positions in the operation plan, the operation corresponding to the real-time operation position is recorded as the first unplanned operation;
[0008] If the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is inconsistent with the operation plan, the operation corresponding to the real-time operation position is recorded as the second unplanned operation;
[0009] If the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is consistent with the operation plan, the operation corresponding to the real-time operation position is recorded as the operation to be monitored; the real-time operation content represented by the on-site monitoring data of the operation position of the operation to be monitored is matched with the substation operation plan, and if they do not match, it is determined to be the third unplanned operation;
[0010] Issue abnormal warnings for the first unplanned operation and the third unplanned operation;
[0011] For the second unplanned operation, the difference between the real-time operation time and the planned operation time of the corresponding position is analyzed, the risk of the real-time operation is judged based on the difference, and corresponding different abnormal warnings are issued based on different risk analysis results.
[0012] In some implementations, determining whether there is an operation on site based on monitoring data collected in real time at the substation site includes:
[0013] The frame difference method is used to determine whether there is a dynamic change in the picture on the video stream data;
[0014] Perform a comparative analysis of the spectrum distribution of the real-time audio stream data and the previous audio stream data to determine whether there is a change.
[0015] In some embodiments, the difference between the real-time operation time and the planned operation time of the location corresponding to the second unplanned operation is analyzed, and the risk of the real-time operation is judged based on the difference, including: when the time difference is less than a preset value, converting the second unplanned operation to a monitored operation for processing.
[0016] In some implementations, analyzing the difference between the real-time operation time and the planned operation time of the corresponding position of the second unplanned operation, and judging the risk of the real-time operation based on the difference, includes:
[0017] Obtaining the planned operation content corresponding to the operation position of the second unplanned operation;
[0018] Based on the planned operation time period corresponding to the planned operation content, determine the proportion of the time gap in the planned operation time period; when the proportion is less than a first preset value, convert the second unplanned operation to a monitored operation for processing; when the proportion is greater than the first preset value and less than a second preset value, determine the operation risk according to the safety operation specifications of the planned operation content corresponding to the second unplanned operation; when the proportion is greater than the second preset value, issue an abnormal alarm and stop the second unplanned operation.
[0019] In some embodiments, the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored is matched with the substation operation plan, including:
[0020] Obtain a reference video stream of the operation process corresponding to the operation plan;
[0021] Based on the video stream data of the on-site monitoring data, a frame-by-frame image comparison analysis is performed with the reference operation video stream to analyze the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream. If the consistency is less than the corresponding preset value, it is determined that the operation to be monitored is an unplanned operation.
[0022] In some embodiments, the analyzing the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream includes:
[0023] Acquire multiple key frames in the reference video stream and form a first key frame sequence in chronological order;
[0024] Acquire multiple key frames of the on-site monitoring video stream and form a second key frame sequence in chronological order;
[0025] Based on a single key frame image in the first key frame sequence, matching and acquiring at least one similar image in the second key frame sequence;
[0026] For each similar image obtained, a plurality of image frames that are adjacent in time to the similar image are acquired, and similarities between the adjacent image frames and the single key frame image are calculated to form a similarity distribution corresponding to the adjacent image frames;
[0027] Based on the similarity distribution of adjacent image frames, similar images that meet preset conditions are used as key frames that match the single key frame image in the first key frame sequence;
[0028] The ratio of the number of matching key frames in the second key frame sequence to the number of image frames in the first key frame sequence is used as the consistency degree of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream.
[0029] In some embodiments, the matching of the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored with the substation operation plan further includes:
[0030] Extract characteristic words according to the operation plan, and retrieve preset typical sound files based on the characteristic word information;
[0031] Based on typical sound files as reference sounds, the consistency between the reference sounds and the live audio stream data is analyzed;
[0032] If the consistency between the reference sound and the on-site audio stream data is less than the corresponding preset value, determining that the operation to be monitored is an unplanned operation;
[0033] The analyzing the consistency between the reference sound and the live audio stream data includes:
[0034] Using a feature extraction module of the adversarial analysis model, extracting a first audio feature from a reference sound and extracting a second audio feature from live audio stream data;
[0035] The adversarial analysis module of the adversarial analysis model is used to analyze the first audio feature and the second audio feature to obtain consistency between the reference sound and the live audio stream data.
[0036] In some embodiments, the method for intelligently checking unplanned operations of substations based on an artificial intelligence platform further includes: performing operation safety detection on the unplanned operations,
[0037] The safety detection method includes: inputting the video stream data and audio stream data based on the on-site monitoring data into a preset network model respectively, extracting the video features and the audio features, and performing the safety detection of the operation behavior based on the video features and the audio features.
[0038] In the second aspect, an intelligent verification system for unplanned substation operations based on an artificial intelligence platform is provided, including:
[0039] A data acquisition unit, used to obtain a substation operation plan from an artificial intelligence platform;
[0040] A real-time operation scene analysis unit is used to determine whether there is an operation behavior on the site according to the monitoring data collected in real time at the substation site, record the monitoring data with the operation behavior as the first data, and determine the real-time operation position corresponding to the operation behavior according to the acquisition device ID of the monitoring data of the first data; the monitoring data includes real-time video stream and audio stream;
[0041] A job position detection unit, used for analyzing whether the real-time job position exists in all the job positions in the job plan, and if the real-time job position does not exist in all the job positions in the job plan, recording the job corresponding to the real-time job position as a first unplanned job;
[0042] an operation time detection unit, used for analyzing whether the operation time of the real-time operation position is consistent with the operation plan, and when the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is inconsistent with the operation plan, recording the operation corresponding to the real-time operation position as a second unplanned operation;
[0043] The operation content detection unit is used to record the operation corresponding to the real-time operation position as the operation to be monitored when the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is consistent with the operation plan; match the real-time operation content represented by the on-site monitoring data of the operation position of the operation to be monitored with the substation operation plan, and determine the third unplanned operation in case of mismatch;
[0044] The unplanned operation response unit is used to issue abnormal warnings for the first unplanned operation and the third unplanned operation; analyze the difference between the real-time operation time and the planned operation time of the corresponding position of the second unplanned operation, judge the risk of the real-time operation based on the difference, and issue corresponding different abnormal warnings based on different risk analysis results.
[0045] According to a third aspect, an electronic device is provided, the electronic device comprising:
[0046] processor;
[0047] a memory for storing processor-executable instructions;
[0048] The processor executes the executable instructions to implement the intelligent verification method for unplanned operations of substations as described in the first aspect.
[0049] In a fourth aspect, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method for intelligent verification of unplanned operations of substations as described in the first aspect are implemented.
[0050] The intelligent verification method for unplanned operations of substations based on an artificial intelligence platform of the present invention has the following beneficial effects: the present invention first performs a simple calculation and analysis on a large amount of monitoring data to determine whether there is any operating behavior in the monitoring scene, and determines that there is manual operation in the monitoring data with dynamic images and dynamic voiceprint changes. The first step reduces the complex calculation amount of a large number of video streams and audio streams, and then performs unplanned operation location, unplanned operation time, and unplanned operation content detection, and screens and determines the unplanned operations layer by layer, avoiding direct operation content analysis of a large amount of monitoring data and comparing it with the operation plan, thereby improving the efficiency of unplanned operation verification. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 It is a flow chart of an intelligent verification method for unplanned operations of a substation based on an artificial intelligence platform in an embodiment of the present application;
[0052] Figure 2 It is a flow chart of a method for analyzing video stream data of a monitored operation in an embodiment of the present application;
[0053] Figure 3 It is a flowchart of the audio stream data analysis method for the operation to be monitored in an embodiment of the present application;
[0054] Figure 4 It is a structural diagram of an intelligent verification system for unplanned operations of a substation based on an artificial intelligence platform in an embodiment of the present application. DETAILED DESCRIPTION
[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0056] The embodiment of the present application provides an intelligent verification method for unplanned substation operations based on an artificial intelligence platform, comprising the following steps:
[0057] Step 1, obtaining the substation operation plan from the artificial intelligence platform;
[0058] Step 2: Determine whether there is any operation on the site based on the monitoring data collected in real time at the substation site, record the monitoring data with the operation as the first data, and determine the real-time operation position corresponding to the operation based on the acquisition device ID of the monitoring data of the first data; the monitoring data includes real-time video stream and audio stream;
[0059] Step 3: If the real-time operation position does not exist in all the operation positions in the operation plan, the operation corresponding to the real-time operation position is recorded as the first unplanned operation;
[0060] Step 4: If the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is inconsistent with the operation plan, the operation corresponding to the real-time operation position is recorded as the second unplanned operation;
[0061] Step 5: If the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is consistent with the operation plan, the operation corresponding to the real-time operation position is recorded as the operation to be monitored; the real-time operation content represented by the on-site monitoring data of the operation position of the operation to be monitored is matched with the substation operation plan; if they do not match, it is determined to be the third unplanned operation;
[0062] Step 6, issuing abnormal warnings for the first unplanned operation and the third unplanned operation; for the second unplanned operation, analyzing the difference between the real-time operation time and the planned operation time of the corresponding position, judging the risk of the real-time operation based on the difference, and issuing corresponding different abnormal warnings based on different risk analysis results.
[0063] A large amount of video surveillance data is generated at the substation site. According to the business goal of patrolling 30,000 cameras in one hour, the amount of data generated every day is 720,000 pieces of data, and the amount of data generated in one month is 21.6 million pieces of data. In the embodiment of the present application, in the first step, a large amount of monitoring data is simply calculated and analyzed to determine whether there is an operation behavior in the monitoring scene, for example, whether there is a dynamic change of the picture in the video stream and whether there is a change of the voiceprint to determine whether there is an action. The determination of the presence of dynamic pictures and dynamic voiceprint changes in the monitoring data is due to the existence of manual operations, resulting in dynamic picture changes and voiceprint changes in the monitoring data. According to this first step, a large amount of complex calculations of video streams and audio streams are reduced. In the second step, the operation whose operation location is not in the plan is recorded as the first unplanned operation, and an abnormal warning needs to be issued to stop the operation; in the third step, considering the complexity of the substation operation, for the operation whose operation time is different from the plan, consider whether the time gap is too large. If it is too large, an abnormal warning needs to be issued to stop the operation. Different time gaps will have different consequences, and the response strategy needs to be determined according to the actual situation. In the fourth step, for the operation whose operation location and operation time are in line with the plan, it is recorded as the operation to be monitored, and the operation content is analyzed to see whether it is in line with the plan. If the job to be inspected does not conform to the job plan, it is determined to be an unplanned job and an abnormal warning needs to be issued.
[0064] In one embodiment, the intelligent verification method for unplanned operations of substations also includes: step 7, performing operation safety detection on unplanned operations, and the safety detection method includes: based on the video stream data and audio stream data of the on-site monitoring data, respectively inputting preset network models, extracting video features and audio features, and performing operation behavior safety detection based on the video features and audio features.
[0065] The behavior safety detection can be directly implemented by analyzing video features or audio features based on the existing intelligent detection model for power operations and the detection model for illegal behaviors at power operations. In the embodiment of the present application, the preset network model uses a multi-modal feature cross-fusion method to fuse video features and audio features, and performs operation behavior safety detection based on the fused features. Specifically, it includes:
[0066] The initial video features are used to obtain the matrices Q1, K1, and V1 using the self-attention mechanism of the transform model. The initial audio features are used to obtain the matrices Q2, K2, and V2 using the self-attention mechanism of the transform model. T1 is obtained by multiplying K1 and V1, and T2 is obtained by multiplying K2 and V2. T1 is processed by the softmax function and multiplied with Q2, and T2 is processed by the softmax function and multiplied with Q1 to achieve cross-complementarity of audio features and video features. Furthermore, the result of multiplying T1 and Q2 is added to the initial video features to obtain crossed video features, the result of multiplying T2 and Q1 is added to the initial audio features to obtain crossed audio features, and the crossed video features and the crossed audio features are concatenated to obtain the fusion result of video features and audio features. Based on the fusion result, the classification layer or output layer of the neural network is used to obtain the safety detection result of the operation behavior.
[0067] In one embodiment, the above step 2, determining whether there is any operation behavior on site based on the monitoring data collected in real time at the substation site, includes:
[0068] Step 21, using a frame difference method to determine whether there is a dynamic change in the picture on the video stream data;
[0069] Step 22: Perform a comparative analysis on the spectrum distribution of the real-time audio stream data and the previous audio stream data to determine whether there is a change.
[0070] In the embodiment of the present application, the frame difference method is used to compare and analyze two images of different time sequences, and the frequency distribution is used to compare and analyze the operation conditions of the substation site. In the embodiment of the present application, the analysis of whether there is an operation behavior on the site is performed, and a simple calculation method is used for preliminary analysis to simplify the amount of calculation and improve the efficiency of intelligent verification of unplanned operations. It can be understood that the frame difference method is used to determine whether there is a dynamic change in the picture. When the frame difference value is greater than the preset threshold, it is determined that there is a moving target in the video stream. When the spectrum distribution comparison analysis of the audio stream data is performed, if the spectrum distribution difference is greater than the preset threshold condition, it is determined that the audio stream acquisition scene has added a sound source, which may be various sounds generated by the on-site operation. For example, the frequency amplitude difference of some frequencies in the spectrum distribution is large compared with the previous audio stream data or the newly added frequency components can determine that the real-time audio stream data has changed. For the analysis of whether there is an operation behavior on the site based on the audio stream and video stream data, a single data analysis can be used, such as only using the video stream, or two data joint methods can be used, such as analyzing the audio stream and the video stream separately, and determining that there is an operation behavior on the site when both change.
[0071] In one embodiment, the above step 5, matching the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored with the substation operation plan, includes the following steps:
[0072] Step 51, obtaining a reference video stream of the operation process corresponding to the operation plan;
[0073] Step 52, based on the video stream data of the on-site monitoring data, perform frame-by-frame image comparison analysis with the reference operation video stream, analyze the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream, if the consistency is less than the corresponding preset value, determine that the monitored operation is an unplanned operation.
[0074] In one embodiment, in the above step 52, analyzing the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream includes:
[0075] Step 521, obtaining multiple key frames in the reference video stream, and forming a first key frame sequence in chronological order;
[0076] Step 522, obtaining multiple key frames of the on-site monitoring video stream, and forming a second key frame sequence in chronological order;
[0077] Step 523, based on the single key frame image in the first key frame sequence, matching and acquiring at least one similar image in the second key frame sequence;
[0078] Step 524, for each similar image obtained, a plurality of image frames that are adjacent in time to the similar image are obtained, and similarities between the adjacent image frames and the single key frame image are calculated to form a similarity distribution corresponding to the adjacent image frames;
[0079] Step 525, selecting similar images that meet preset conditions based on the similarity distribution of adjacent image frames as key frames that match the single key frame image in the first key frame sequence;
[0080] Step 526: The ratio of the number of matching key frames in the second key frame sequence to the number of image frames in the first key frame sequence is used as the consistency between the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream.
[0081] Among them, the similarity distribution of adjacent image frames meets the preset conditions, which include: (1) the similarity change process is regular and (2) the similarity variance is less than the preset value. Specifically, the similarity change process is that in the similarity values corresponding to the entire adjacent image frames, according to the time sequence of the image frames, the similarity can be a monotonically increasing trend, or a monotonically increasing trend on the left side of the similar image and a monotonically decreasing trend on the right side of the similar image. For example, the first key frame sequence is ,in are the key frames in the reference video stream respectively, and the second key frame sequence is ,in are the key frames of the on-site monitoring video stream, and for a single key frame image in the first key frame sequence If a similar image is found in the second key frame sequence , then further obtain similar images Multiple image frames adjacent in time... , , , , ..., analyzing the multiple adjacent ..., , , , , ...image frames and single keyframe images If the change of the similarity meets the preset condition, that is, the similarity distribution of the adjacent image frames meets the preset condition, then determine the second key frame sequence. is the same as the first key frame sequence Matching keyframes.
[0082] In one embodiment, the above step 5, matching the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored with the substation operation plan, further includes:
[0083] Step 53, extracting characteristic words according to the operation plan, and retrieving a preset typical sound file based on the characteristic word information;
[0084] Step 54, based on the typical sound file as the reference sound, analyzing the consistency between the reference sound and the live audio stream data;
[0085] Step 55, if the consistency between the reference sound and the on-site audio stream data is less than the corresponding preset value, it is determined that the operation to be monitored is an unplanned operation;
[0086] Wherein, in step 54, analyzing the consistency between the reference sound and the live audio stream data includes:
[0087] Step 541, using the feature extraction module of the adversarial analysis model to extract a first audio feature from the reference sound and a second audio feature from the live audio stream data;
[0088] Step 542: Analyze the first audio feature and the second audio feature using the adversarial analysis module of the adversarial analysis model to obtain consistency between the reference sound and the live audio stream data.
[0089] The adversarial analysis model, during the training process, is based on a large number of sound samples, and each sound sample is annotated with corresponding annotation data. The annotation data can characterize the sound source device, can characterize the sound source scene, etc. During training, two sound samples of the same type of annotation data and one sound sample of different annotation data are used as an input sample group input into the model for training. Based on the extremely large similarity of sample features between samples with the same annotation and the extremely large difference of sample features between samples with different annotations as the objective function, the network model is iteratively trained until the target loss function meets the preset conditions, and the training is completed to obtain the adversarial analysis model. It can be understood that in the embodiment of the present application, the training stage is mainly to obtain the optimal network parameters of the feature extraction module of the adversarial analysis model. After the training is completed, the features extracted by the feature extraction module of the adversarial analysis model can achieve the extraction of the most similar features of the two audio features when the two input audios are consistent, and the extraction of the features with the largest difference between the two audio features when the two input audios are inconsistent. The two sound samples of the same type of labeled data are denoted as A1 and A2, and the sound sample of different labeled data is denoted as A3. The two sound samples A1 and A2 of the same type of labeled data can be collected based on the same sound source device or based on the same sound source scene, and the sound sample A3 of different labeled data can be collected based on a sound source device or sound source scene different from that of the sound samples A1 and A2. The objective function in the training process can be based on Determine, among which, , , Represents the preset weight coefficient, , , Represent the sample characteristics of samples A1, A2, and A3 respectively, Represents the similarity function. After the training is completed, the two audio stream data to be analyzed are input into the adversarial analysis model, and audio feature extraction and consistency analysis are performed in turn.
[0090] It can be understood that the above steps 51-52 can be used alone to judge unplanned operations, and the above steps 53-55 can be used alone to judge unplanned operations. In preferred cases, the judgment results of the above steps 51-52 and the judgment results of the above steps 53-55 can be combined to improve the accuracy of the third unplanned operation judgment result.
[0091] In one embodiment, the above step 6, analyzing the difference between the real-time operation time and the planned operation time of the corresponding position of the second unplanned operation, and judging the risk of the real-time operation based on the difference, includes: when the time difference is less than a preset value, converting the second unplanned operation into a monitored operation for processing. Specifically, judging the risk of the real-time operation based on the difference includes the following steps:
[0092] Step 61, obtaining the planned operation content corresponding to the operation position of the second unplanned operation;
[0093] Step 62, based on the planned operation time period corresponding to the planned operation content, determine the proportion of the time gap to the planned operation time period; when the proportion is less than a first preset value, convert the second unplanned operation to a monitored operation for processing; when the proportion is greater than the first preset value and less than the second preset value, determine the operation risk according to the safety operation specifications of the planned operation content corresponding to the second unplanned operation; when the proportion is greater than the second preset value, issue an abnormal alarm and stop the second unplanned operation.
[0094] In the embodiment of the present application, the risk of the operation is determined based on the difference between the occurrence time of the real-time operation and the planned time. It can be understood that if the real-time start time of the power operation is too early or too late than the planned time, it may affect the execution of other power operations or be affected by other power operations, causing the power operation to have safety risks. When the proportion corresponding to the time difference is within the first preset value and less than the second preset value, it is necessary to determine whether there is an operation risk based on the relevant provisions on time in the relevant specifications of the power operation.
[0095] Among them, the operation risk is determined based on the safe operating specifications of the power operation. Based on the content of the power operation, the relevant fault / accident / violation records can be searched in the historical operation history records of the same type to determine whether the current power operation has the risk of operation failure / accident / violation caused by the operation time difference. The relevant time specifications can also be extracted based on the content of the safety operating specifications to determine whether the power operation has operation risks.
[0096] The embodiment of the present application also provides a substation unplanned operation intelligent verification system based on an artificial intelligence platform, including:
[0097] A data acquisition unit, used to obtain a substation operation plan from an artificial intelligence platform;
[0098] A real-time operation scene analysis unit is used to determine whether there is an operation behavior on the site according to the monitoring data collected in real time at the substation site, record the monitoring data with the operation behavior as the first data, and determine the real-time operation position corresponding to the operation behavior according to the acquisition device ID of the monitoring data of the first data; the monitoring data includes real-time video stream and audio stream;
[0099] A job position detection unit, used for analyzing whether the real-time job position exists in all the job positions in the job plan, and if the real-time job position does not exist in all the job positions in the job plan, recording the job corresponding to the real-time job position as a first unplanned job;
[0100] an operation time detection unit, used for analyzing whether the operation time of the real-time operation position is consistent with the operation plan, and when the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is inconsistent with the operation plan, recording the operation corresponding to the real-time operation position as a second unplanned operation;
[0101] The operation content detection unit is used to record the operation corresponding to the real-time operation position as the operation to be monitored when the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is consistent with the operation plan; match the real-time operation content represented by the on-site monitoring data of the operation position of the operation to be monitored with the substation operation plan, and determine the third unplanned operation in case of mismatch;
[0102] The unplanned operation response unit is used to issue abnormal warnings for the first unplanned operation and the third unplanned operation; analyze the difference between the real-time operation time and the planned operation time of the corresponding position of the second unplanned operation, judge the risk of the real-time operation based on the difference, and issue corresponding different abnormal warnings based on different risk analysis results.
[0103] The intelligent verification system for unplanned operations of substations provided in this embodiment and the embodiment of the method for intelligent verification of unplanned operations of substations provided in the above-mentioned embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here. In some embodiments, the intelligent verification system for unplanned operations of substations provided in this embodiment can be implemented in a combination of software and hardware. As an example, the intelligent verification system for unplanned operations of substations provided in this embodiment can be directly embodied as a combination of software modules executed by a processor. The software module can be located in a storage medium, and the storage medium is located in a memory. The processor reads the executable instructions included in the software module in the memory, and combines the necessary hardware (for example, including a processor and other components connected to the bus) to complete the method for intelligent verification of unplanned operations of substations provided in this embodiment.
[0104] An embodiment of the present application provides an electronic device, the electronic device comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor implements the above-mentioned intelligent verification method for unplanned operations of substations by running the executable instructions. It can be understood that the memory can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The memory in this embodiment is capable of storing data to support the operation of the terminal. Of course, the electronic device may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The electronic device may also include other components for implementing the functions of the device, which will not be repeated here.
[0105] The embodiment of the present application provides a computer-readable storage medium, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the above-mentioned substation unplanned operation intelligent verification method are implemented. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage node.
[0106] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.
Claims
1. An intelligent verification method for unplanned substation operations based on an artificial intelligence platform, characterized in that: The steps include: Obtain substation operation plans from the artificial intelligence platform; Determine whether there is any operation behavior on the site according to the monitoring data collected in real time at the substation site, record the monitoring data with the operation behavior as the first data, and determine the real-time operation position corresponding to the operation behavior according to the collection device ID of the monitoring data of the first data; The monitoring data includes real-time video stream and audio stream; If the real-time operation position does not exist in all the operation positions in the operation plan, the operation corresponding to the real-time operation position is recorded as the first unplanned operation; If the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is inconsistent with the operation plan, the operation corresponding to the real-time operation position is recorded as the second unplanned operation; If the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is consistent with the operation plan, the operation corresponding to the real-time operation position is recorded as the operation to be monitored; Matching the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored with the substation operation plan, and if there is no match, determining it as the third unplanned operation; Issue abnormal warnings for the first unplanned operation and the third unplanned operation; For the second unplanned operation, analyzing the difference between the real-time operation time and the planned operation time of the corresponding position, judging the risk of the real-time operation based on the difference, and issuing corresponding different abnormal warnings based on different risk analysis results; The real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored is matched with the substation operation plan, including: obtaining a reference video stream of the operation process corresponding to the operation plan; performing frame-by-frame image comparison and analysis on the video stream data based on the on-site monitoring data and the reference operation video stream, analyzing the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream, and if the consistency is less than the corresponding preset value, determining that the operation to be monitored is an unplanned operation; The analysis of the consistency between the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream includes: obtaining multiple key frames in the reference video stream to form a first key frame sequence in chronological order; obtaining multiple key frames in the on-site monitoring video stream to form a second key frame sequence in chronological order; based on a single key frame image in the first key frame sequence, matching and obtaining at least one similar image in the second key frame sequence; for each similar image obtained, obtaining multiple image frames that are adjacent in time to the similar image, calculating the similarity between the adjacent image frames and the single key frame image, and forming a similarity distribution corresponding to the adjacent image frames; based on the similarity distribution of adjacent image frames, similar images that meet preset conditions are used as key frames that match the single key frame image in the first key frame sequence; based on the ratio of the number of matching key frames in the second key frame sequence to the number of image frames in the first key frame sequence, the degree of consistency between the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream.
2. The intelligent verification method for unplanned substation operations based on an artificial intelligence platform according to claim 1 is characterized in that: Determining whether there is any operation behavior on site based on the monitoring data collected in real time on site of the substation includes: The frame difference method is used to determine whether there is a dynamic change in the picture on the video stream data; Perform a comparative analysis of the spectrum distribution of the real-time audio stream data and the previous audio stream data to determine whether there is a change.
3. The intelligent verification method for unplanned substation operations based on an artificial intelligence platform according to claim 1 is characterized in that: Analyzing the difference between the real-time operation time and the planned operation time of the corresponding position of the second unplanned operation, and judging the risk of the real-time operation based on the difference, includes: When the time difference is less than a preset value, converting the second unplanned operation into a to-be-monitored operation for processing; The step of analyzing the difference between the real-time operation time and the planned operation time of the corresponding position of the second unplanned operation and judging the risk of the real-time operation based on the difference specifically includes: Obtaining the planned operation content corresponding to the operation position of the second unplanned operation; Based on the planned operation time period corresponding to the planned operation content, determine the proportion of the time gap in the planned operation time period; when the proportion is less than a first preset value, convert the second unplanned operation to a monitored operation for processing; when the proportion is greater than the first preset value and less than a second preset value, determine the operation risk according to the safety operation specifications of the planned operation content corresponding to the second unplanned operation; when the proportion is greater than the second preset value, issue an abnormal alarm and stop the second unplanned operation.
4. The intelligent verification method for unplanned substation operations based on an artificial intelligence platform according to claim 1 is characterized in that: The real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored is matched with the substation operation plan, and further includes: Extract characteristic words according to the operation plan, and retrieve preset typical sound files based on the characteristic word information; Based on typical sound files as reference sounds, the consistency between the reference sounds and the live audio stream data is analyzed; If the consistency between the reference sound and the on-site audio stream data is less than the corresponding preset value, determining that the operation to be monitored is an unplanned operation; The analyzing the consistency between the reference sound and the live audio stream data includes: Using a feature extraction module of the adversarial analysis model, extracting a first audio feature from a reference sound and extracting a second audio feature from live audio stream data; The adversarial analysis module of the adversarial analysis model is used to analyze the first audio feature and the second audio feature to obtain consistency between the reference sound and the live audio stream data.
5. The intelligent verification method for unplanned substation operations based on an artificial intelligence platform according to claim 1 is characterized in that: Also includes: Conduct safety inspections on unplanned operations. The safety detection method includes: inputting the video stream data and audio stream data based on the on-site monitoring data into a preset network model respectively, extracting the video features and the audio features, and performing the safety detection of the operation behavior based on the video features and the audio features.
6. The intelligent verification system for unplanned substation operations based on artificial intelligence platform is characterized by: include: A data acquisition unit, used to obtain a substation operation plan from an artificial intelligence platform; A real-time operation scene analysis unit is used to determine whether there is an operation behavior on the site according to the monitoring data collected in real time at the substation site, record the monitoring data with the operation behavior as the first data, and determine the real-time operation position corresponding to the operation behavior according to the collection device ID of the monitoring data of the first data; The monitoring data includes real-time video stream and audio stream; A job position detection unit, used for analyzing whether the real-time job position exists in all the job positions in the job plan, and if the real-time job position does not exist in all the job positions in the job plan, recording the job corresponding to the real-time job position as a first unplanned job; an operation time detection unit, used for analyzing whether the operation time of the real-time operation position is consistent with the operation plan, and when the real-time operation position exists in all the operation positions in the operation plan and the operation time of the real-time operation position is inconsistent with the operation plan, recording the operation corresponding to the real-time operation position as a second unplanned operation; A job content detection unit, configured to record the job corresponding to the real-time job position as a job to be monitored when the real-time job position exists in all job positions in the job plan and the job time of the real-time job position is consistent with the job plan; Matching the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored with the substation operation plan, and determining the third unplanned operation in case of mismatch; The unplanned operation response unit is used to issue an abnormal warning for the first unplanned operation and the third unplanned operation; for the second unplanned operation, the difference between the real-time operation time and the planned operation time of the corresponding position is analyzed, the risk of the real-time operation is judged based on the difference, and different corresponding abnormal warnings are issued based on different risk analysis results; In the operation content detection unit, the real-time operation content represented by the on-site monitoring data of the operation location of the operation to be monitored is matched with the substation operation plan, including: obtaining a reference video stream of the operation process corresponding to the operation plan; performing frame-by-frame image comparison and analysis with the reference operation video stream based on the video stream data of the on-site monitoring data, analyzing the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream, and if the consistency is less than the corresponding preset value, determining that the operation to be monitored is an unplanned operation; the analysis of the consistency of the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream includes: obtaining multiple key frames in the reference video stream, forming a first key frame sequence in chronological order; obtaining multiple key frames of the on-site monitoring video stream, and obtaining .... key frames are obtained to form a second key frame sequence in chronological order; based on a single key frame image in the first key frame sequence, at least one similar image is matched and obtained in the second key frame sequence; for each similar image obtained, multiple image frames that are adjacent in time to the similar image are obtained, and the similarities between the adjacent image frames and the single key frame image are calculated to form a similarity distribution corresponding to the adjacent image frames; based on the similarity distribution of adjacent image frames, similar images that meet preset conditions are used as key frames that match the single key frame image in the first key frame sequence; based on the ratio of the number of matching key frames in the second key frame sequence to the number of image frames in the first key frame sequence, the degree of consistency between the dynamic process represented by the on-site monitoring video stream and the dynamic process of the reference video stream is determined.
7. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; The processor implements the intelligent verification method for unplanned operations of substations as described in any one of claims 1 to 5 by running the executable instructions.
8. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the intelligent verification method for unplanned operations of a substation as described in any one of claims 1 to 5 are implemented.
