Method and device for determining substation anomaly, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a method and device for determining substation anomaly, electronic equipment and storage medium. The method comprises: acquiring a visible light video stream and an infrared video stream of a substation area; performing frame-by-frame noise reduction processing on the visible light video stream to obtain a set of denoised images; performing weighted fusion processing on the set of denoised images to obtain a reconstructed video stream; wherein the set of denoised images comprises at least one denoised image data; the reconstructed video stream represents the visible light video stream after frame-by-frame noise reduction and weighted fusion processing; inputting the infrared video stream and the reconstructed video stream into an anomaly detection model for processing to obtain an anomaly result output by the anomaly detection model; and issuing an alarm based on the anomaly result. The method is used to accurately detect substation anomalies to ensure safe operation of the substation.

Description

Technical Field

[0001] This application relates to the field of substation technology, and in particular to a method, apparatus, electronic device, and storage medium for determining substation anomalies. Background Technology

[0002] As the core hub of the power system, substations undertake key functions such as voltage transformation, power distribution, and system stability control. Their operational safety is directly related to the stability of the power grid, the reliability of power supply, and the lifespan of equipment.

[0003] In existing technologies, visible light images are typically acquired at preset time intervals, and anomalies are then identified using fixed rules or simple models. However, fixed intervals result in data delays, making it impossible to capture sudden anomalies in real time and to accurately and promptly identify substation anomalies.

[0004] Therefore, there is an urgent need for a solution that can accurately detect abnormal conditions in substations to ensure their safe operation. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for determining substation anomalies, so as to accurately detect substation anomalies and ensure the safe operation of the substation.

[0006] In a first aspect, embodiments of this application provide a method for determining substation anomalies, including:

[0007] Acquire visible light and infrared video streams of the substation area;

[0008] The visible light video stream is subjected to frame-by-frame denoising processing to obtain a set of denoised images; the set of denoised images is then subjected to weighted fusion processing to obtain a reconstructed video stream; wherein the set of denoised images includes at least one denoised image data; the reconstructed video stream represents the visible light video stream after frame-by-frame denoising and weighted fusion processing.

[0009] The infrared video stream and the reconstructed video stream are input into the anomaly detection model for processing to obtain the anomaly result output by the anomaly detection model; and an alarm is issued based on the anomaly result.

[0010] In one possible implementation, the visible light video stream is subjected to frame-by-frame noise reduction processing to obtain a set of denoised images, including:

[0011] The visible light video stream is split frame by frame to obtain frame set data;

[0012] Perform a Fast Fourier Transform on the data of each frame in the frame set to obtain the frequency domain signal;

[0013] The frequency domain signal is filtered based on a preset filtering threshold to obtain filtered frequency domain data; wherein the preset filtering threshold is determined based on the illumination intensity of the substation.

[0014] The filtered frequency domain data is subjected to inverse fast Fourier transform to obtain the denoised image set.

[0015] In one possible implementation, the preset filtering threshold is determined based on the illumination intensity of the substation, including:

[0016] Obtain the illumination intensity of the substation;

[0017] If it is determined that the light intensity is less than a first preset threshold, then the preset filtering threshold is determined to be the first filtering threshold;

[0018] If it is determined that the light intensity is greater than or equal to a first preset threshold and less than or equal to a second preset threshold, then the preset filtering threshold is determined to be the second filtering threshold.

[0019] If the light intensity is determined to be greater than the second preset threshold, then the preset filtering threshold is determined to be the third filtering threshold; wherein the first filtering threshold, the second filtering threshold, and the third filtering threshold decrease sequentially.

[0020] In one possible implementation, the set of denoised images is subjected to weighted fusion processing to obtain a reconstructed video stream, including:

[0021] Based on feature point matching and timestamp calibration, each frame of the denoised image set is subjected to frame synchronization alignment processing to obtain a continuous single-frame image sequence.

[0022] Based on the continuous single-frame image sequence, an adaptive frame weight is determined; wherein, the adaptive frame weight is a fusion weight coefficient adaptively calculated based on the image sharpness and / or motion vector of each frame in the continuous single-frame image sequence.

[0023] The continuous single-frame image sequence is fused according to the adaptive frame weights to obtain the reconstructed video stream.

[0024] In one possible implementation, the continuous single-frame image sequence is fused according to the adaptive frame weights to obtain the reconstructed video stream, including:

[0025] The continuous single-frame image sequence is divided into segments with a preset step size and a preset time window to obtain the image to be processed;

[0026] The image to be processed is weighted and fused based on the adaptive frame weights to obtain the reconstructed video stream.

[0027] In one possible implementation, the infrared video stream and the reconstructed video stream are input into an anomaly detection model for processing to obtain the anomaly result output by the anomaly detection model, including:

[0028] The infrared video stream and the reconstructed video stream are input into the anomaly detection model for processing. The feature extraction module of the anomaly detection model processes the infrared video stream and the reconstructed video stream to obtain the feature video stream.

[0029] The feature video stream is processed based on the anomaly detection model to obtain the target category; and the motion trajectory of the target category is generated based on the target category.

[0030] The anomaly detection model is used to process the target category and the motion trajectory to obtain the anomaly result output by the anomaly detection model.

[0031] In one possible implementation, and based on the target category, generating the motion trajectory of the target category includes:

[0032] For targets belonging to the target category, target tracking is performed in consecutive frames of the feature video stream to obtain the position information of the target in each frame, and the motion trajectory is formed based on the position information.

[0033] Secondly, embodiments of this application provide a device for determining substation anomalies, comprising:

[0034] The acquisition module is used to acquire visible light video streams and infrared video streams of the substation area;

[0035] The processing module is used to perform frame-by-frame noise reduction on the visible light video stream to obtain a set of denoised images; and to perform weighted fusion processing on the set of denoised images to obtain a reconstructed video stream; wherein the set of denoised images includes at least one denoised image data; and the reconstructed video stream represents the visible light video stream after frame-by-frame noise reduction and weighted fusion processing.

[0036] The detection module is used to input the infrared video stream and the reconstructed video stream into the anomaly detection model for processing, obtain the anomaly result output by the anomaly detection model, and issue an alarm based on the anomaly result.

[0037] In one possible implementation, the processing module includes:

[0038] The visible light video stream is split frame by frame to obtain frame set data;

[0039] Perform a Fast Fourier Transform on the data of each frame in the frame set to obtain the frequency domain signal;

[0040] The frequency domain signal is filtered based on a preset filtering threshold to obtain filtered frequency domain data; wherein the preset filtering threshold is determined based on the illumination intensity of the substation.

[0041] The filtered frequency domain data is subjected to inverse fast Fourier transform to obtain the denoised image set.

[0042] In one possible implementation, the preset filtering threshold is determined based on the illumination intensity of the substation, including:

[0043] Obtain the illumination intensity of the substation;

[0044] If it is determined that the light intensity is less than a first preset threshold, then the preset filtering threshold is determined to be the first filtering threshold;

[0045] If it is determined that the light intensity is greater than or equal to a first preset threshold and less than or equal to a second preset threshold, then the preset filtering threshold is determined to be the second filtering threshold.

[0046] If the light intensity is determined to be greater than the second preset threshold, then the preset filtering threshold is determined to be the third filtering threshold; wherein the first filtering threshold, the second filtering threshold, and the third filtering threshold decrease sequentially.

[0047] In one possible implementation, the processing module includes:

[0048] Based on feature point matching and timestamp calibration, each frame of the denoised image set is subjected to frame synchronization alignment processing to obtain a continuous single-frame image sequence.

[0049] Based on the continuous single-frame image sequence, an adaptive frame weight is determined; wherein, the adaptive frame weight is a fusion weight coefficient adaptively calculated based on the image sharpness and / or motion vector of each frame in the continuous single-frame image sequence.

[0050] The continuous single-frame image sequence is fused according to the adaptive frame weights to obtain the reconstructed video stream.

[0051] In one possible implementation, the continuous single-frame image sequence is fused according to the adaptive frame weights to obtain the reconstructed video stream, including:

[0052] The continuous single-frame image sequence is divided into segments with a preset step size and a preset time window to obtain the image to be processed;

[0053] The image to be processed is weighted and fused based on the adaptive frame weights to obtain the reconstructed video stream.

[0054] In one possible implementation, the detection module includes:

[0055] The infrared video stream and the reconstructed video stream are input into the anomaly detection model for processing. The feature extraction module of the anomaly detection model processes the infrared video stream and the reconstructed video stream to obtain the feature video stream.

[0056] The feature video stream is processed based on the anomaly detection model to obtain the target category; and the motion trajectory of the target category is generated based on the target category.

[0057] The anomaly detection model is used to process the target category and the motion trajectory to obtain the anomaly result output by the anomaly detection model.

[0058] In one possible implementation, and based on the target category, generating the motion trajectory of the target category includes:

[0059] For targets belonging to the target category, target tracking is performed in consecutive frames of the feature video stream to obtain the position information of the target in each frame, and the motion trajectory is formed based on the position information.

[0060] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0061] The memory stores computer-executed instructions;

[0062] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0063] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0064] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0065] This application provides a method, apparatus, electronic device, and storage medium for determining substation anomalies. By simultaneously acquiring visible light video streams and infrared video streams of the substation area, the visible light video stream is first denoised frame by frame to obtain a set of denoised images. Then, image reconstruction is completed using weighted fusion to generate a reconstructed video stream. Subsequently, the infrared video stream and the reconstructed video stream are jointly fed into an anomaly detection model for calculation. The anomaly judgment result output by the model triggers an alarm. This method achieves the effect of taking into account both visible light image details and infrared temperature characteristics, reducing environmental noise interference, improving the accuracy of substation equipment anomaly identification, reducing missed and false alarms, and timely detecting hidden faults such as equipment overheating and external damage, and providing rapid early warning. Attached Figure Description

[0066] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0067] Figure 1 A flowchart illustrating a method for determining substation anomalies provided in this application embodiment. Figure 1 ;

[0068] Figure 2 A flowchart illustrating a method for determining substation anomalies provided in this application embodiment. Figure 2 ;

[0069] Figure 3 A schematic diagram of the structure of a substation anomaly determination device provided in this application embodiment. Figure 1 ;

[0070] Figure 4 A schematic diagram of the structure of a substation anomaly determination device provided in this application embodiment. Figure 2 ;

[0071] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0072] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application.

[0074] Substation anomaly monitoring falls under the technical field of intelligent operation and maintenance and video surveillance analysis in power systems. It primarily addresses business needs such as sensing the operating status of equipment within transmission and substations, identifying on-site personnel activities, and providing early warnings of sudden risk events. These scenarios are typically deployed in key locations such as main transformer areas, switchyards, busbar areas, cable trench entrances and exits, perimeter passages, and internal roads. Visible light cameras capture images of equipment appearance, structural outlines, and personnel activities, while infrared imaging devices collect information on surface heat distribution and temperature rise changes. The relevant video data is then transmitted to the substation monitoring host, security platform, or integrated operation and maintenance center for analysis and processing.

[0075] As the core hub of the power system, substations undertake critical functions such as voltage transformation, power distribution, and system stability control. Their operational safety directly affects the stability of the power grid, the reliability of power supply, and the service life of transformer equipment. Under complex and ever-changing natural environments and potential human interference, substations require 24 / 7, high-precision monitoring methods to perceive their operational status in real time. Any missed detection or misjudgment could lead to equipment damage or even large-scale power outages. Therefore, continuous and accurate anomaly monitoring of substations is of paramount importance.

[0076] Current safety monitoring of substation operations typically relies on monitoring systems acquiring visible light images at preset time intervals, followed by analysis of these images using fixed rules or simple morphological detection models to determine the presence of anomalies. However, this method has significant shortcomings in practical applications: Firstly, the fixed-interval image acquisition mechanism introduces inherent data delays, making it impossible to capture sudden anomalies immediately, and the resulting large number of redundant images greatly increases the processing burden. Secondly, relying solely on visible light images is highly susceptible to interference from lighting conditions, rain, fog, and strong light; image quality deteriorates sharply at night or in low light conditions, and it cannot effectively identify invisible anomalies such as internal equipment overheating. While infrared images can reflect temperature distribution, they lack detailed features, and traditional models struggle to integrate and utilize multimodal information, resulting in high false alarm and missed detection rates in complex scenarios, severely restricting the real-time performance and accuracy of substation safety monitoring.

[0077] Therefore, this application provides a method for determining substation anomalies, which can solve the above-mentioned problems.

[0078] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0079] Figure 1 A flowchart illustrating a method for determining substation anomalies provided in this application embodiment. Figure 1 ,like Figure 1 As shown, the method includes:

[0080] S101. Acquire the visible light video stream and infrared video stream of the substation area.

[0081] For example, the visible light video stream is used to provide visible light image information of the substation site as the basic input for frame-by-frame noise reduction, fusion and subsequent anomaly detection. Specifically, it can be formed by continuous acquisition by visible light camera devices deployed in locations such as the main transformer area, switchyard, busbar area, cable trench entrances and exits, perimeter passages and roads within the station.

[0082] Infrared video streams are used to provide information on thermal radiation or temperature at the substation site. Specifically, they can be acquired by an infrared imaging device that is set up in conjunction with a visible light camera device, and are used to reflect the heat distribution on the equipment surface, local temperature rise, and the status of hot spots.

[0083] The substation area can be understood as the monitoring range within the station that requires abnormal monitoring. It can be a single key equipment area or the entire station area covered by multiple monitoring points.

[0084] Acquiring visible light video streams and infrared video streams is not limited to simply receiving two video data streams, but includes video acquisition, time information recording, transmission access, buffer organization, and synchronous preparation processes before the input processing module.

[0085] For example, the executing entity can be an in-station monitoring host, a security analysis server, an edge computing unit in an integrated operation and maintenance platform, or an embedded intelligent analysis device deployed near the camera. The image resolution output by the visible light camera can be set to 1920×1080, 1280×720, or other resolutions that match the on-site bandwidth conditions, and the frame rate can be set to 25 frames per second, 30 frames per second, or a continuous acquisition frequency set according to the real-time requirements of the station; the thermal imaging video output by the infrared camera can use the same or different resolution as the visible light video, and include a temperature matrix, thermal radiation value, or a calibrated pseudo-color temperature image in the video frames.

[0086] To ensure that the subsequent anomaly detection model can perform joint analysis on the two types of videos, in one possible embodiment, the visible light camera and the infrared camera are mounted on the same pan-tilt unit or adjacent fixed brackets, and are synchronously triggered for acquisition using a unified clock source, so that the scenes acquired at the same time maintain correspondence in spatial location and temporal order.

[0087] In another possible implementation, after the two types of camera devices independently acquire video, the processing unit establishes a correspondence based on the timestamp, acquisition sequence number, or synchronization identifier in the network video stream, matching visible light frames and infrared frames within adjacent time windows. Based on the above analysis, it can be seen that through synchronous acquisition or post-synchronous matching, the two modal data can form corresponding video pairs before entering subsequent processing, thereby avoiding misalignment between infrared thermal anomalies and visible light target positions.

[0088] Furthermore, the video acquisition process can also include pre-checks on network status and image validity. For example, after receiving the video stream, the processing unit can detect whether the video stream is interrupted, whether the frame sequence is continuous, whether the timestamp bounces back, and whether the image contains completely black frames, completely white frames, or obviously distorted frames. When short-term jitter, sudden drops in bitrate, or packet loss are detected in the visible light video stream, abnormal frames can be marked and entered into low-weight processing in the subsequent noise reduction and fusion process. When short-term missing frames occur in the infrared video stream, the most recent valid infrared frame can be temporarily stored to make up for one processing cycle to maintain the stability of the model input structure. This step establishes a reliable data foundation for subsequent frame-by-frame noise reduction and dual-modal anomaly identification. Especially in substations with complex operating conditions such as day-night switching, backlight, rain, fog, and electromagnetic interference, establishing video inputs with consistent time and clear sources first can reduce the risk of misjudgment caused by input mismatch in subsequent processing links.

[0089] S102. Perform frame-by-frame noise reduction on the visible light video stream to obtain a set of denoised images; perform weighted fusion processing on the set of denoised images to obtain a reconstructed video stream.

[0090] The denoised image set includes at least one denoised image data; the reconstructed video stream represents the visible light video stream after frame-by-frame denoising and weighted fusion processing.

[0091] For example, the denoised image set is used to carry the result of frame-by-frame denoising of the visible light video stream, providing input for subsequent weighted fusion. It is essentially a data set formed by processing consecutive frames of the visible light video stream frame by frame. Each denoised image data corresponds to a frame in the original visible light video stream or a base frame in a group of processing units.

[0092] The reconstructed video stream is used to characterize the visible light video stream after frame-by-frame noise reduction and weighted fusion processing. It is an important input for subsequent anomaly detection models. It not only preserves the target contours, device edges, personnel poses and scene structure information in the original video, but also suppresses the interference caused by random noise, transient blur and local distortion through cross-frame fusion.

[0093] For example, the processing unit first decomposes the acquired visible light video stream into consecutive image frames in chronological order, and assigns a frame number, acquisition timestamp, and monitoring point identifier to each frame. For single-frame image data, color space conversion and basic normalization processing can be performed first, such as converting the RGB (Red Green Blue) image into a representation with separate luminance and chrominance components, so as to process luminance noise more accurately. Then, noise reduction processing is performed on each frame.

[0094] In one possible embodiment, frame-by-frame noise reduction is implemented using frequency domain processing. This involves performing a Fast Fourier Transform (FFT) on the current frame image to convert the spatial domain image into a frequency domain representation. A filtering threshold is set based on the high-frequency noise distribution in the spectrum, suppressing frequency components exceeding the noise threshold range. Then, an Inverse Fast Fourier Transform (IFT) is used to restore the image to the image domain, resulting in the denoised single-frame image. The filtering threshold can be adaptively determined based on the image's local variance, brightness level, edge strength, and sensor noise estimates. For example, under low-light conditions at night, the random noise introduced by high gain in the image is stronger, so the high-frequency suppression strength can be appropriately increased; during the day, when device edges are clear and noise is low, the filtering strength can be reduced to avoid excessive smoothing of details. If the spectrum F(u,v) is processed by the frequency domain filtering function H(u,v), the output spectrum G(u,v) can be expressed as G(u,v) = F(u,v) × H(u,v), where F(u,v) represents the spectral components of the original frame in the frequency domain, H(u,v) represents the filtering coefficients constructed based on noise characteristics, and G(u,v) represents the filtered spectrum. This expression aims to achieve a balance between noise suppression and preservation of structural details by adjusting the retention ratio of different frequency components.

[0095] After frame-by-frame denoising of all frames to be processed, a set of denoised images is obtained. This set can be stored in a buffer, memory queue, or circular buffer structure and organized into a window sequence in chronological order. For example, N consecutive denoised images can be selected as a fusion window, where N can be 3, 5, or 7 to balance real-time performance and efficiency in utilizing continuous information. Subsequently, the processing unit performs weighted fusion processing on the denoised image set. The weighted fusion processing is not a simple averaging; instead, the images in the set are first aligned to frame synchronization to correct for spatial displacement caused by camera vibrations, wind tremors, or slight gimbal shifts. Then, the corresponding weights are determined based on the clarity, noise level, structural similarity to adjacent frames, and temporal proximity of each image.

[0096] For example, the weight of the current center frame can be appropriately increased to maintain temporal currentness, while the weights of the preceding and following neighboring frames are used to compensate for information loss in the center frame due to momentary blurring or local occlusion. The fused target frame I reconstruction can be expressed as I reconstruction = Σwi × Ii, where Ii represents the i-th denoised image data, wi represents the corresponding weight, and each weight satisfies Σwi = 1. This formula reflects the contribution ratio of different denoised frames to the final reconstruction result. By controlling the size of wi, clear, stable image information consistent with the current scene can occupy a higher proportion, thereby obtaining a reconstructed video stream with better continuity and lower noise.

[0097] In one possible embodiment, the weights can be determined jointly based on image gradient energy, edge fidelity, signal-to-noise ratio estimate, and inter-frame motion consistency.

[0098] In another possible implementation, the weights can also be combined with a time-order decay factor setting so that the denoised frames that are closer to the current time have a higher contribution.

[0099] The reconstructed video stream formed in this way corresponds to the original visible light video stream on the time axis, which can maintain the continuous expression of the video and significantly reduce the impact of single-frame random noise, slight blur and local flicker on subsequent recognition.

[0100] By first denoising frame by frame and then performing weighted fusion, not only can the image quality of a single frame be improved, but also the redundant information between adjacent frames can be used to repair the effective details in the disturbed frames. This enhances the device outline, foreign object boundary, human figure and motion trajectory clues before they are input into the anomaly detection model, thereby providing a more stable and distinguishable visible light representation for subsequent bimodal anomaly judgment.

[0101] S103. Input the infrared video stream and the reconstructed video stream into the anomaly detection model for processing to obtain the anomaly result output by the anomaly detection model; and issue an alarm based on the anomaly result.

[0102] For example, the anomaly detection model is used to jointly process the infrared video stream and the reconstructed video stream and output anomaly results. It can be deployed as a deep learning model, a fusion recognition network, or a comprehensive analysis model formed by connecting target detection, target tracking and anomaly discrimination modules.

[0103] Anomaly results are used to represent the anomaly judgment conclusions obtained by the model after processing the input video stream. They can reflect whether there are equipment thermal anomalies, connector overheating, foreign objects hanging, personnel accidentally entering, illegal intrusion, wandering in the area, or other risk states that need attention.

[0104] Alarms are abnormal notification actions that are output to the outside after an abnormal result is generated. They can be issued through audio-visual devices, platform messages, dispatch terminal pop-ups, mobile push notifications, or control linkage signals.

[0105] For example, the temporally corresponding infrared video frame sequence and the reconstructed video frame sequence are first fed into the input of the anomaly detection model.

[0106] In one possible embodiment, the infrared video stream and the reconstructed video stream are respectively fed into two input channels of the model, and the model extracts their respective feature representations at the front end. The visible light channel extracts device contours, target boundaries, personnel postures, scene textures, and motion appearance information, while the infrared channel extracts temperature distribution, hot spot locations, heat diffusion trends, and hot spot intensity information. Subsequently, feature fusion is performed in the intermediate layer or decision layer to form a unified feature video stream representation.

[0107] In another possible implementation, the infrared frames and reconstructed video frames can be spatially aligned and region mapped first, projecting the infrared hotspots onto the corresponding positions in the visible light image, and then inputting them into the joint coding network for end-to-end recognition. After obtaining the fused features, the anomaly detection model can continue to perform target detection, identifying personnel, equipment, vehicles, insulators, wire connection points, fence boundaries, and foreign object categories in the image; then, it combines continuous frames to perform target tracking, generating the motion trajectory of each target on the time axis.

[0108] The target category characterizes the entity to which the currently detected object belongs, while the motion trajectory characterizes the target's position changes, direction of movement, speed changes, and stationary state over a period of time. The model ultimately outputs anomaly results based on the target category, thermal characteristics, motion trajectory, and their matching with preset rules. For example, when the infrared channel detects that the temperature at the main transformer connection point is consistently higher than the historical baseline and the hotspot area matches the location of the equipment connection point in the reconstructed video, it can be determined as an equipment overheating anomaly; when the reconstructed video identifies personnel crossing a fence into a restricted area, and the infrared video confirms the presence of a human heat source movement trajectory, it can be determined as an illegal intrusion or accidental entry anomaly; when the reconstructed video identifies newly attached objects on a conductor, and the infrared does not show a clear human heat source, it can be determined as a foreign object sloughing risk.

[0109] After obtaining abnormal results, the processing unit generates alarms based on the anomaly category, confidence level, duration, and risk level.

[0110] For example, when an abnormal result meets preset triggering conditions, such as an abnormal confidence level exceeding a threshold, a continuous frame count reaching a set value, or the same abnormality recurring within a continuous time window, the system can issue an alarm and record the alarm time, monitoring point, abnormal screenshot, infrared heat map, target category, and trajectory information. The alarm information can be uploaded to the on-site monitoring platform and the remote operation and maintenance center, and associated with video playback, PTZ linkage, access control linkage, or inspection work order systems, enabling operation and maintenance personnel to quickly locate on-site risks.

[0111] By inputting the reconstructed, more stable visible light video stream and the original infrared video stream with clear thermal information into the anomaly detection model, the model can simultaneously utilize visible contour features and thermal anomaly features for joint discrimination. Compared to analyzing either modality data alone, this approach is more effective in improving the accuracy of equipment anomaly identification, small target foreign object detection, and personnel activity assessment in complex backgrounds. Furthermore, directly triggering alarms based on anomaly results can shorten the latency from video acquisition to risk alerts, improving real-time response capabilities and alarm effectiveness in unattended or minimally staffed substations.

[0112] This application provides a method for determining substation anomalies. By simultaneously acquiring visible light video streams and infrared video streams of the substation area, the visible light video stream is first denoised frame by frame to obtain a set of denoised images. Then, image reconstruction is completed using weighted fusion to generate a reconstructed video stream. Subsequently, the infrared video stream and the reconstructed video stream are jointly fed into an anomaly detection model for calculation. The anomaly judgment result output by the model triggers an alarm. This method achieves the effect of taking into account both visible light image details and infrared temperature characteristics, reducing environmental noise interference, improving the accuracy of substation equipment anomaly identification, reducing missed and false alarms, and enabling timely detection and rapid early warning of hidden faults such as equipment overheating and external damage.

[0113] Figure 2 A flowchart illustrating a method for determining substation anomalies provided in this application embodiment. Figure 2 ,like Figure 2 As shown, in this embodiment... Figure 1 Based on the embodiments, a method for determining substation anomalies is described in detail, the method including:

[0114] S201. Acquire the visible light video stream and infrared video stream of the substation area.

[0115] For example, this step can be referred to step S101, and will not be described again.

[0116] S202. Perform frame-by-frame splitting on the visible light video stream to obtain frame set data; perform Fast Fourier Transform on the data of each frame in the frame set data to obtain frequency domain signal; filter the frequency domain signal based on a preset filtering threshold to obtain filtered frequency domain data; perform Inverse Fast Fourier Transform on the filtered frequency domain data to obtain a denoised image set.

[0117] The preset filtering threshold is determined based on the light intensity of the substation.

[0118] For example, the Fast Fourier Transform is used to transform a single frame of an image from the spatial domain to the frequency domain in order to distinguish image details, random noise, and periodic interference in a frequency representation.

[0119] Frequency domain signals are used to characterize the amplitude and phase information of each frequency component, which facilitates subsequent suppression of noise components according to thresholds.

[0120] The preset filtering threshold is used to limit the frequency range to be retained or weakened. Its value is adjusted according to the light intensity of the substation, so as to adapt to the noise difference under strong light during the day, weak light at night or backlight conditions.

[0121] The inverse fast Fourier transform is used to restore the filtered frequency domain data to image domain data, so as to form a denoised single-frame image and constitute a set of denoised images.

[0122] In actual processing, the visible light video stream is read sequentially by the monitoring host or edge computing unit and split into a continuous set of frame data. Each frame in the frame set maintains a temporal correspondence consistent with the original video stream. Each frame image can first undergo grayscale normalization or size unification to improve the consistency of frequency domain operations, and then a Fast Fourier Transform (FFT) is performed to obtain the corresponding frequency domain signal. Subsequently, a filtering threshold is selected based on the current light intensity of the substation to suppress high-frequency noise components or components with amplitudes below the threshold in the frequency domain signal, resulting in filtered frequency domain data. Then, an Inverse Fast Fourier Transform (IFFT) is performed on the filtered frequency domain data to restore a denoised image set; each image in this set corresponds to a frame in the original video stream and can be directly used for subsequent weighted fusion and anomaly detection.

[0123] The above processing enables frequency domain denoising of visible light video before it enters the subsequent recognition stage. This reduces the impact of low-light noise, jitter noise, and scattering interference on image edges and target contours, allowing the denoised image set to retain more effective texture information. Since the filtering threshold is adaptively determined with illumination intensity, the denoising intensity can match the imaging quality of the substation under different time periods and weather conditions, thereby reducing the risk of over-smoothing and loss of detail. The resulting denoised image set exhibits high stability, facilitating subsequent generation of reconstructed video streams and improving the accuracy and reliability of anomaly detection results.

[0124] In one example, the illumination intensity of a substation is obtained; if the illumination intensity is determined to be less than a first preset threshold, then a preset filtering threshold is determined as the first filtering threshold; if the illumination intensity is determined to be greater than or equal to the first preset threshold and less than or equal to a second preset threshold, then a preset filtering threshold is determined as the second filtering threshold; if the illumination intensity is determined to be greater than the second preset threshold, then a preset filtering threshold is determined as the third filtering threshold; wherein, the first filtering threshold, the second filtering threshold, and the third filtering threshold decrease sequentially.

[0125] For example, the illuminance of a substation is used to characterize the current ambient lighting level. It can be collected in real-time by a illuminance sensor installed within the substation, or the corresponding illuminance information can be output by an environmental acquisition device. A first preset threshold and a second preset threshold are used to divide different lighting conditions into low-light, medium-light, and high-light intervals to adapt the filtering intensity. The first filtering threshold, the second filtering threshold, and the third filtering threshold correspond to filtering thresholds under different lighting conditions. The first filtering threshold is used for low-light scenarios, the second filtering threshold for medium-light scenarios, and the third filtering threshold for high-light scenarios, with the thresholds decreasing sequentially to match the changes in image noise distribution when the light intensity is enhanced.

[0126] In practical implementation, after acquiring the current illumination intensity data of the substation, the system compares this data with a first preset threshold and a second preset threshold, and reads the corresponding preset filtering threshold from a pre-stored threshold mapping table accordingly. When the ground illumination is low, noise in the dark areas of the image is usually more pronounced, so the system uses a higher first filtering threshold to enhance frequency domain noise suppression. When the illumination is in the middle range, the system uses a second filtering threshold to balance noise suppression and detail preservation. When the illumination is high, the system uses a third filtering threshold to reduce the impact of filtering on edges and textures. The threshold determination result is further used to filter the frequency domain signal of the visible light video stream, thereby outputting a video image suitable for subsequent reconstruction.

[0127] In one possible implementation, when the light intensity is less than 5 lux in a low-light nighttime scene, the preset filtering threshold is increased by 30%; when the light intensity is between 5 lux and 100 lux in a rain / fog / strong light scene, the preset filtering threshold is adjusted to the middle value; and when the light intensity is greater than 100 lux in a normal lighting scene, the preset filtering threshold is decreased by 15%.

[0128] By linking the filtering threshold to the actual illumination intensity of the substation, the noise reduction parameters can be automatically adjusted according to environmental changes, avoiding the problems of insufficient noise suppression under low illumination or excessive smoothing under high illumination when the threshold is fixed. Since the preset filtering threshold corresponds one-to-one with the illumination state, the filtering process is more adaptable to different time periods, different weather conditions, and local occlusion conditions within the substation. This improves the stability of the denoised image set and provides more consistent input data for subsequent weighted fusion and anomaly detection, thereby improving the accuracy of substation anomaly identification and the reliability of alarms.

[0129] S203. Based on feature point matching and timestamp calibration, perform frame synchronization alignment processing on each frame of the denoised image set to obtain a continuous single-frame image sequence; determine the adaptive frame weight based on the continuous single-frame image sequence; perform fusion processing on the continuous single-frame image sequence according to the adaptive frame weight to obtain the reconstructed video stream.

[0130] The adaptive frame weight is a fusion weight coefficient that is adaptively calculated based on the image sharpness and / or motion vectors of each frame in a continuous single-frame image sequence.

[0131] For example, feature point matching is used to extract corner points, texture points or edge points in adjacent frames and establish inter-frame correspondence, while timestamp calibration is used to unify images with deviations in acquisition time to the same temporal reference.

[0132] A continuous single-frame image sequence refers to a set of single-frame images arranged in the sampling order after being aligned by time and content.

[0133] Adaptive frame weights are used to characterize the contribution of each frame to the fusion process, image sharpness is used to reflect the degree of image detail preservation, and motion vectors are used to reflect the inter-frame displacement changes. Both are used together to adjust the fusion weights.

[0134] For example, the denoised image set can be output by the pre-denoising processing module. The processor first reads the acquisition timestamp of each frame image and corrects the timing deviation through interpolation correction or unified clock source. Then, it performs feature point extraction, matching and registration on adjacent frames to map the same device area or the same target contour to a consistent position, forming a continuous single-frame image sequence.

[0135] Subsequently, a sharpness index is calculated based on the gradient magnitude, edge intensity, or local contrast of each frame. This index is then combined with optical flow estimation, block matching displacement, or target tracking results to obtain motion vectors, which are used to generate corresponding fusion weight coefficients. Frames with high sharpness and good motion consistency are assigned larger weights; frames with blurriness, jitter, or abrupt shifts are assigned smaller weights to reduce the impact of noisy frames on the final result. The fusion process can be implemented using weighted averaging, sliding window stacking, or multi-frame resampling. The output reconstructed video stream retains continuous temporal information and enhances effective details.

[0136] This method eliminates inter-frame misalignment through alignment processing, and then dynamically allocates weights based on sharpness and motion information. This ensures that the denoised images no longer participate in subsequent analysis as isolated frames, but instead form a continuous and stable video representation, thereby improving the sharpness, consistency, and target identifiability of the reconstructed video stream. Because the fusion weights can automatically adjust according to changes in on-site motion and image quality, good video reconstruction results can still be maintained even in substation scenarios with day-night variations, wind and sand disturbances, or localized blurring.

[0137] After adopting this implementation method, the effective edges and target contours in the reconstructed video stream are preserved, the interference of jittery frames and blurred frames on the overall picture is suppressed, the input quality received by the subsequent anomaly detection model is more stable, which helps to improve the accuracy of identifying equipment anomalies, foreign objects and human activities, and reduce the probability of false alarms and false negatives in complex environments.

[0138] In one example, a sequence of consecutive single-frame images is divided into images to be processed using a preset step size and a preset time window; the images to be processed are then weighted and fused based on adaptive frame weights to obtain a reconstructed video stream.

[0139] For example, a continuous single-frame image sequence refers to a set of single-frame images output consecutively after frame synchronization and alignment, with its temporal order corresponding to the original video capture order. A preset step size controls the sliding interval of the continuous single-frame image sequence on the time axis, and a preset time window limits the coverage of frames participating in each fusion. Together, these two elements can divide the continuous image sequence into multiple local image segments to form the image to be processed. The image to be processed is the direct input object for subsequent weighted fusion and typically includes several adjacent frame images within the current window.

[0140] For example, a buffer can be set up in the image processing host. A continuous sequence of single-frame images is written to the buffer according to timestamps, and the starting position of the window is determined according to a preset step size. Then, corresponding frame segments are extracted according to the preset time window to obtain the image to be processed. The adaptive frame weight can be calculated based on the image sharpness, motion vector, or local texture stability of each frame. When a frame has jitter, blur, or occlusion, its corresponding weight will decrease accordingly, while the weight of image frames with stable images and relatively complete detail retention will increase. Based on this weight, weighted superposition, normalization, or pixel-level fusion operations are performed on the image to be processed to output the reconstructed video stream. This reconstructed video stream can be further written to the video buffer by the central processing unit for subsequent anomaly detection. In practical applications, other models of this processing module can also be selected; this embodiment does not limit this.

[0141] This implementation uses window-based segmentation and sliding fusion to allow information to accumulate continuously between adjacent frames. Adaptive frame weights are used to suppress interference from low-quality image frames, thereby improving the stability and continuity of the reconstructed video. Because the fusion process highlights effective target contours in clear frames and weakens the influence of noisy frames, the reconstructed video stream is more conducive to subsequent models recognizing changes in device appearance, personnel activity, and thermal anomalies.

[0142] By adopting this implementation method, the availability of video information can be improved under the complex lighting, jitter and local noise interference conditions of substations, the impact of single frame distortion on detection results can be reduced, and the smoothness and consistency of continuous image expression can be enhanced, thereby improving the accuracy of anomaly identification and the reliability of alarms.

[0143] S204. Input the infrared video stream and the reconstructed video stream into the anomaly detection model for processing. The feature extraction module of the anomaly detection model processes the infrared video stream and the reconstructed video stream to obtain the feature video stream. The feature video stream is processed based on the anomaly detection model to obtain the target category. Based on the target category, the motion trajectory of the target category is generated. The target category and motion trajectory are processed based on the anomaly detection model to obtain the anomaly result output by the anomaly detection model.

[0144] For example, the feature video stream is used to represent the video feature representation obtained after joint processing by the feature extraction module of the anomaly detection model. It carries infrared thermal distribution information and target contour information in the reconstructed video stream, serving as the input basis for subsequent category recognition.

[0145] The feature extraction module extracts and fuses features from the input video to extract salient regions, texture changes, temperature changes, and temporal correlation information of the target, thereby forming an intermediate representation that the model can use for discrimination. The target category characterizes the type of object identified by the model, such as equipment parts, people, vehicles, or foreign objects. The motion trajectory describes the positional change path of the target corresponding to the target category in consecutive frames, reflecting its direction of movement, speed changes, and stationary areas, and providing temporal basis for anomaly detection.

[0146] For example, the infrared video stream and the reconstructed video stream can be fed into the feature extraction module through the dual input channels of the anomaly detection model. Inside the model, complementary features in the two types of video streams are extracted and the feature video stream is output through convolutional feature encoding, temporal correlation modeling and cross-modal feature fusion.

[0147] In another implementation, the infrared video stream and the reconstructed video stream can be concatenated and then input into the model to uniformly complete feature extraction. After classifying the feature video stream, the model generates a motion trajectory by combining the target position change information in consecutive frames. The trajectory can be composed of the inter-frame displacement of the target detection box center point, Kalman filter tracking results, or other target tracking results. Subsequently, the anomaly detection model makes a judgment based on the combination relationship between the target category and the motion trajectory. When the target category matches a preset abnormal object, or when the motion trajectory exhibits characteristics such as crossing boundaries, stopping, reversing, clustering, or abnormal approach, an anomaly result is output.

[0148] In one possible implementation, the infrared video stream and the reconstructed video stream are input into an anomaly detection model for processing. A feature extraction module based on the anomaly detection model processes the infrared video stream and the reconstructed video stream to obtain a feature video stream. The feature video stream is then partitioned based on the anomaly detection model to obtain target categories. Understandably, targets in different regions of the feature video stream are detected based on the partitioning results. For example, the highest level of target detection is performed on the primary regions of the feature video stream, identifying personnel, open flames, large tools, safety equipment (helmets, insulated gloves), work clothes, hand tools, etc., with a model confidence threshold of 0.6, and detection is performed on every frame of the feature video stream. Similarly, the highest level of target detection is performed on the secondary regions of the feature video stream, identifying personnel, open flames, large tools, safety equipment (helmets, insulated gloves), work clothes, hand tools, etc., with a model confidence threshold of 0.75, and detection is performed on every two frames of the feature video stream. The highest level of target detection is performed on the third-level regions in the feature video stream to identify people, open flames, large tools, etc. in the third-level regions. The confidence threshold of the model is set to 0.85, and the detection is performed on every three frames of the feature video stream.

[0149] Based on the identified target, the target's trajectory is determined; that is, whether the target is stationary or passing through. If the target is determined to be stationary, the target's actions are identified based on the standard skeleton point model in the anomaly detection model to determine the target's behavior. If the target is determined to be passing through, the target's actions are identified based on the simplified skeleton point model in the anomaly detection model to determine the target's behavior. Based on the target and its behavior, an anomaly result is obtained.

[0150] In one possible implementation, target dwell time can be categorized into working dwell time, observation dwell time, and abnormal dwell time. If a target is detected carrying compliant tools, has a corresponding work permit authorization, behaves correctly, and has low trajectory entropy, then the target is identified as working dwell time. If a target is detected without work permit authorization or carrying no tools, does not engage in any illegal actions, and only observes the equipment, and has medium trajectory entropy, then the target is identified as observing dwell time. If a target is detected without work permit authorization, carrying illegal tools, or exhibiting suspicious actions (touching equipment wiring, looking around, avoiding cameras), and has high trajectory entropy, then the target is identified as abnormal dwell time. If the target is identified as working dwell time, the corresponding dwell time factor is low, and a dwell time factor of 1 can be selected. If the target is identified as observing dwell time, the corresponding dwell time factor is medium, and a dwell time factor of 1.5 can be selected. If the target is identified as abnormal dwell time, the corresponding dwell time factor is high, and a dwell time factor of 1.8 can be selected.

[0151] In one possible implementation, anomalies in the substation are determined based on the anomaly risk module in the anomaly detection model. The anomaly risk value is determined based on area weight, behavior weight, task factor, and dwell factor; understandably, the anomaly risk value = area weight × behavior weight × task factor × dwell factor. Wherein, area weight (1.0-2.5): set based on equipment voltage level and operating status; behavior weight (1.0-3.0): from a preset behavior risk database (e.g., "climbing live equipment" weight 3.0); task factor (0.5-1.5): associated with the work order system, calculating personnel authorization and tool compliance; dwell factor (1.0-1.8): dynamically calculated based on dwell time and the degree of anomalousness of dwell behavior.

[0152] This approach enables collaborative analysis of infrared thermal information and reconstructed visible light information within the model, reducing misjudgments caused by single-modal recognition. Furthermore, it enhances the ability to identify abnormal behavior through joint category and trajectory determination. Because the motion trajectory incorporates continuous frame temporal constraints, the model can compensate for short-term occlusion, local blurring, and instantaneous jitter, thereby improving the stability, accuracy, and alarm reliability of anomaly detection in complex substation scenarios.

[0153] In one example, target tracking is performed on targets belonging to the target category in consecutive frames of the feature video stream to obtain the target's position information in each frame, and a motion trajectory is formed based on the position information.

[0154] Target tracking refers to maintaining the association with the same target continuously within consecutive frames of a feature video stream, ensuring that the target's positional changes over time can be continuously recorded. Positional information is used to characterize the target's spatial location in each frame. This can be achieved using either the target's center point coordinates or the target's bounding box coordinates. The bounding box coordinates more completely reflect the area occupied by the target in the image, facilitating subsequent trajectory fitting. In practical applications, this positional information can also be supplemented by recording the target's scale, aspect ratio, or inter-frame displacement; however, this embodiment does not limit this.

[0155] In implementation, the feature video stream can be input into the target tracking module. After initial localization of targets belonging to the target category, matching is performed between adjacent consecutive frames based on target appearance features, motion consistency, and spatial proximity to determine the correspondence of the same target in different frames. The target tracking module outputs the target's position data in each frame and writes this position data into a trajectory buffer. The trajectory buffer saves the position information of each frame in chronological order to form a continuous position sequence. When the target experiences short-term occlusion or local blurring in a frame, trajectory continuity can be maintained based on the correlation results of previous and subsequent frames to reduce the probability of target loss. In practical applications, the target tracking module can also employ other tracking algorithms, which are not limited in this embodiment.

[0156] When generating a motion trajectory based on location information, the location information in each frame can be connected sequentially in time to obtain the target's motion path in the feature video stream. When the location information uses bounding boxes, the center point of the bounding box can also be taken as the trajectory node, thereby generating a smoother trajectory curve. This motion trajectory can be used to characterize the target's direction of movement, speed of movement, and area of ​​stay within a preset time period, and further provide temporal behavioral features for anomaly detection models, enabling the model to combine the target category and motion trajectory to determine whether the target has abnormally intruded, moved beyond its boundaries, or stayed without authorization.

[0157] In this way, the motion trajectory corresponding to the target category can be continuously constructed and stably updated in the feature video stream, so that anomaly detection no longer depends on the target result of a single frame, but can be comprehensively judged by combining temporal displacement changes, thereby improving the continuity and accuracy of target behavior recognition in complex scenes and reducing the probability of misjudgment caused by single-frame false detection, occlusion or image jitter.

[0158] This application provides a method for determining substation anomalies. It involves synchronously acquiring visible light and infrared dual-modal video streams of the substation area. First, the visible light video stream is split frame by frame to obtain frame set data, which is then converted to a frequency domain signal via Fast Fourier Transform (FFT). Noise components are filtered out using a preset filtering threshold, and then the image set is restored using Inverse Fast Fourier Transform (IFT). Subsequently, frame synchronization and alignment are achieved through feature point matching and timestamp calibration. Adaptive frame weights are calculated based on a continuous sequence of single-frame images, and weighted fusion is used to generate a reconstructed video stream. Finally, the infrared video stream and the reconstructed visible light video stream are input into an anomaly detection model, sequentially performing feature extraction, target category identification, and motion trajectory generation. The method outputs anomaly results through joint analysis of target category and motion trajectory. This approach effectively solves the technical problems of high visible light video noise interference in complex substation environments, low fusion accuracy due to asynchronous timing of dual-modal data, and high false alarm and missed detection rates in single-modal anomaly detection. It significantly improves the accuracy and real-time performance of substation anomaly detection. Furthermore, the adaptive frame weight fusion preserves the effective information of key frames, enhancing the reliability of anomaly target identification and trajectory tracking.

[0159] Figure 3 A schematic diagram of the structure of a substation anomaly determination device provided in this application embodiment. Figure 1 ,like Figure 3 As shown, the substation anomaly determination device 30 provided in this embodiment includes:

[0160] Acquisition module 301 is used to acquire visible light video streams and infrared video streams of the substation area;

[0161] Processing module 302 is used to perform frame-by-frame denoising on the visible light video stream to obtain a set of denoised images; and to perform weighted fusion on the set of denoised images to obtain a reconstructed video stream; wherein the set of denoised images includes at least one denoised image data; and the reconstructed video stream represents the visible light video stream after frame-by-frame denoising and weighted fusion.

[0162] The detection module 303 is used to input the infrared video stream and the reconstructed video stream into the anomaly detection model for processing, obtain the anomaly result output by the anomaly detection model, and issue an alarm based on the anomaly result.

[0163] This embodiment provides a device for determining substation anomalies, which can execute the method provided in the above-described method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0164] Figure 4 A schematic diagram of the structure of a substation anomaly determination device provided in this application embodiment. Figure 2 ,like Figure 4 As shown, the substation anomaly determination device 40 provided in this embodiment includes:

[0165] Acquisition module 401 is used to acquire visible light video streams and infrared video streams of the substation area;

[0166] Processing module 402 is used to perform frame-by-frame denoising on the visible light video stream to obtain a set of denoised images; and to perform weighted fusion on the set of denoised images to obtain a reconstructed video stream; wherein the set of denoised images includes at least one denoised image data; and the reconstructed video stream represents the visible light video stream after frame-by-frame denoising and weighted fusion.

[0167] The detection module 403 is used to input the infrared video stream and the reconstructed video stream into the anomaly detection model for processing, obtain the anomaly result output by the anomaly detection model, and issue an alarm based on the anomaly result.

[0168] In one possible implementation, the processing module 402 includes:

[0169] The visible light video stream is split frame by frame to obtain frame set data;

[0170] Perform a Fast Fourier Transform on the data of each frame in the frame set to obtain the frequency domain signal;

[0171] Based on a preset filtering threshold, the frequency domain signal is filtered to obtain filtered frequency domain data; where the preset filtering threshold is determined based on the illumination intensity of the substation.

[0172] The filtered frequency domain data is processed by inverse fast Fourier transform to obtain a set of denoised images.

[0173] In one possible implementation, the preset filtering threshold is determined based on the illumination intensity of the substation, including:

[0174] Obtain the light intensity of the substation;

[0175] If it is determined that the light intensity is less than the first preset threshold, then the preset filtering threshold is determined to be the first filtering threshold;

[0176] If it is determined that the light intensity is greater than or equal to the first preset threshold and less than or equal to the second preset threshold, then the preset filtering threshold is determined to be the second filtering threshold.

[0177] If the light intensity is determined to be greater than the second preset threshold, then the preset filtering threshold is determined to be the third filtering threshold; wherein, the first filtering threshold, the second filtering threshold, and the third filtering threshold decrease sequentially.

[0178] In one possible implementation, the processing module 402 includes:

[0179] Based on feature point matching and timestamp calibration, frame synchronization alignment is performed on each frame of the denoised image set to obtain a continuous single-frame image sequence.

[0180] Based on a continuous sequence of single-frame images, adaptive frame weights are determined; wherein, the adaptive frame weights are fusion weight coefficients adaptively calculated based on the image sharpness and / or motion vectors of each frame in the continuous sequence of single-frame images.

[0181] The reconstructed video stream is obtained by fusing consecutive single-frame image sequences according to adaptive frame weights.

[0182] In one possible implementation, the sequence of consecutive single-frame images is fused according to adaptive frame weights to obtain a reconstructed video stream, including:

[0183] The continuous single-frame image sequence is divided into segments with a preset step size and a preset time window to obtain the image to be processed;

[0184] The reconstructed video stream is obtained by weighted fusion of the images to be processed based on adaptive frame weights.

[0185] In one possible implementation, the detection module 403 includes:

[0186] The infrared video stream and the reconstructed video stream are input into the anomaly detection model for processing. The feature extraction module based on the anomaly detection model processes the infrared video stream and the reconstructed video stream to obtain the feature video stream.

[0187] The feature video stream is processed based on an anomaly detection model to obtain the target category; and the motion trajectory of the target category is generated based on the target category.

[0188] The anomaly detection model is used to process the target category and motion trajectory to obtain the anomaly results output by the anomaly detection model.

[0189] In one possible implementation, and based on the target category, generating the motion trajectory of the target category includes:

[0190] For targets belonging to the target category, target tracking is performed in consecutive frames of the feature video stream to obtain the target's position information in each frame, and a motion trajectory is formed based on the position information.

[0191] This embodiment provides a device for determining substation anomalies, which can execute the method provided in the above-described method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0192] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the electronic device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0193] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0194] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0195] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0196] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0197] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0198] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0199] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0200] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0201] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0202] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0203] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0204] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0205] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0207] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for determining substation anomalies, characterized in that, include: Acquire visible light and infrared video streams of the substation area; The visible light video stream is subjected to frame-by-frame noise reduction processing to obtain a set of noise-reduced images; The denoised image set is subjected to weighted fusion processing to obtain a reconstructed video stream; wherein the denoised image set includes at least one denoised image data; the reconstructed video stream represents a visible light video stream after frame-by-frame denoising and weighted fusion processing; The infrared video stream and the reconstructed video stream are input into the anomaly detection model for processing to obtain the anomaly result output by the anomaly detection model; and an alarm is issued based on the anomaly result.

2. The method according to claim 1, characterized in that, The visible light video stream is subjected to frame-by-frame noise reduction processing to obtain a set of denoised images, including: The visible light video stream is split frame by frame to obtain frame set data; Perform a Fast Fourier Transform on the data of each frame in the frame set to obtain the frequency domain signal; The frequency domain signal is filtered based on a preset filtering threshold to obtain filtered frequency domain data; wherein the preset filtering threshold is determined based on the illumination intensity of the substation. The filtered frequency domain data is subjected to inverse fast Fourier transform to obtain the denoised image set.

3. The method according to claim 2, characterized in that, The preset filtering threshold is determined based on the light intensity of the substation, including: Obtain the illumination intensity of the substation; If it is determined that the light intensity is less than a first preset threshold, then the preset filtering threshold is determined to be the first filtering threshold; If it is determined that the light intensity is greater than or equal to a first preset threshold and less than or equal to a second preset threshold, then the preset filtering threshold is determined to be the second filtering threshold. If the light intensity is determined to be greater than the second preset threshold, then the preset filtering threshold is determined to be the third filtering threshold; wherein the first filtering threshold, the second filtering threshold, and the third filtering threshold decrease sequentially.

4. The method according to claim 1, characterized in that, The denoised image set is subjected to weighted fusion processing to obtain a reconstructed video stream, including: Based on feature point matching and timestamp calibration, each frame of the denoised image set is subjected to frame synchronization alignment processing to obtain a continuous single-frame image sequence. Based on the continuous single-frame image sequence, an adaptive frame weight is determined; wherein, the adaptive frame weight is a fusion weight coefficient adaptively calculated based on the image sharpness and / or motion vector of each frame in the continuous single-frame image sequence. The continuous single-frame image sequence is fused according to the adaptive frame weights to obtain the reconstructed video stream.

5. The method according to claim 4, characterized in that, The reconstructed video stream is obtained by fusing the consecutive single-frame image sequence according to the adaptive frame weights, including: The continuous single-frame image sequence is divided into segments with a preset step size and a preset time window to obtain the image to be processed; The image to be processed is weighted and fused based on the adaptive frame weights to obtain the reconstructed video stream.

6. The method according to any one of claims 1-5, characterized in that, The infrared video stream and the reconstructed video stream are input into an anomaly detection model for processing to obtain the anomaly results output by the anomaly detection model, including: The infrared video stream and the reconstructed video stream are input into the anomaly detection model for processing. The feature extraction module of the anomaly detection model processes the infrared video stream and the reconstructed video stream to obtain the feature video stream. The feature video stream is processed based on the anomaly detection model to obtain the target category; and the motion trajectory of the target category is generated based on the target category. The anomaly detection model is used to process the target category and the motion trajectory to obtain the anomaly result output by the anomaly detection model.

7. The method according to claim 6, characterized in that, And based on the target category, generate the motion trajectory of the target category, including: For targets belonging to the target category, target tracking is performed in consecutive frames of the feature video stream to obtain the position information of the target in each frame, and the motion trajectory is formed based on the position information.

8. A device for determining substation anomalies, characterized in that, include: The acquisition module is used to acquire visible light video streams and infrared video streams of the substation area; The processing module is used to perform frame-by-frame noise reduction processing on the visible light video stream to obtain a set of noise-reduced images; The denoised image set is subjected to weighted fusion processing to obtain a reconstructed video stream; wherein the denoised image set includes at least one denoised image data; the reconstructed video stream represents a visible light video stream after frame-by-frame denoising and weighted fusion processing; The detection module is used to input the infrared video stream and the reconstructed video stream into the anomaly detection model for processing, obtain the anomaly result output by the anomaly detection model, and issue an alarm based on the anomaly result.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

Images