A machine vision-based substation equipment anomaly detection method and system

Abstract

The application provides a substation equipment anomaly detection method and system based on machine vision, which comprises the following steps: detecting image sequences of each equipment in a complete operation cycle; detecting an equipment motion area and an equipment static area in each frame of image, performing time sequence alignment processing on the equipment motion area and a cycle operation reference template of the corresponding equipment, and calculating a time sequence deviation between the motion feature and the reference template; scheduling all cameras covering the target equipment in the field of view to simultaneously detect the corresponding equipment images, respectively calculating the feature deviation values of the same equipment in each equipment image, and generating a multi-view feature deviation vector; according to the physical connection relationship and the energy transmission path of the substation equipment, a corresponding equipment anomaly propagation directed graph is constructed, and the multi-view feature deviation vector is matched with the equipment anomaly propagation directed graph to determine whether the target equipment is normal according to the matching result. The application can greatly improve the detection efficiency.

Description

Technical Field

[0001] This invention relates to the field of substation inspection technology, and in particular to a method and system for detecting substation equipment anomalies based on machine vision. Background Technology

[0002] With the rapid advancement of smart grid construction, machine vision-based substation equipment anomaly detection technology has gradually replaced traditional manual inspections, becoming a core means of ensuring the safe and stable operation of substations. However, the persistently high false alarm rate remains a key bottleneck restricting the large-scale application of this technology. According to industry statistics, even the most advanced substation visual inspection systems currently available still experience a false alarm rate as high as 5%-10% in complex operating environments, with some densely populated substations generating hundreds of invalid alarms daily. This large number of invalid alarms leads to severe "alarm fatigue" among maintenance personnel, significantly increasing their workload and potentially causing truly critical equipment anomalies to be overlooked, thus creating significant safety hazards.

[0003] To address the aforementioned issues, existing technologies primarily focus on optimizing the false alarm rate from a single dimension: some solutions improve detection accuracy by modifying the feature extraction structure of the target detection network, but cannot effectively distinguish between real equipment defects and environmental interference such as shadows, reflections, and stains; some solutions use simple multi-frame temporal analysis to filter out instantaneous interference such as birds and fallen leaves, but are completely ineffective against persistent static interference; some solutions introduce general large models for secondary analysis, but lack specific knowledge of substation equipment topology and defect characteristics, resulting in a still high false alarm rate; and some solutions reduce false alarms by lowering detection sensitivity and increasing alarm thresholds, but inevitably lead to a significant increase in the missed detection rate.

[0004] In summary, existing technologies generally lack a systematic false alarm filtering mechanism for specific substation scenarios, failing to effectively integrate equipment spatial topology, defect physical characteristics, temporal evolution patterns, and multimodal information. Single-dimensional filtering methods can only address specific types of interference and cannot cope with the complex and ever-changing interference factors in substations, such as lighting, weather, and background, resulting in low system detection efficiency and difficulty in meeting the actual operational needs of unattended smart substations. Therefore, developing a multi-dimensional collaborative false alarm filtering method for substation visual anomaly detection has become an urgent technical problem to be solved in this field. Summary of the Invention

[0005] Therefore, the purpose of this invention is to provide a machine vision-based method and system for detecting anomalies in substation equipment, in order to solve the problem that the existing technology lacks a systematic false alarm filtering mechanism for specific substation scenarios, cannot cope with the complex and ever-changing interference factors in substations, and thus results in low system detection efficiency.

[0006] The first aspect of the present invention proposes: A machine vision-based method for detecting anomalies in substation equipment, wherein the method includes: Based on the installation location, field of view, and calibration parameters of the substation monitoring cameras, a spatial mapping relationship of the overlapping areas of different camera fields of view is constructed, and based on the spatial mapping relationship, the image sequence of each device within a complete operating cycle is detected. Based on the image sequence, the periodic motion features and appearance change features of each device are detected to generate a device periodic operation reference template. The image sequences are processed in parallel to detect the device motion region and device static region in each frame of the image. The device motion region is time-aligned with the corresponding device periodic operation reference template to calculate the time deviation between the motion features and the reference template. When the timing deviation of any surveillance camera exceeds the preset deviation threshold, the corresponding target device is detected and a multi-camera collaborative observation command is triggered to schedule all cameras whose field of view covers the target device to simultaneously detect the corresponding device image, and calculate the feature deviation value of the same device in each device image to generate a multi-view feature deviation vector. Based on the physical connection relationship and energy transfer path of the substation equipment, a corresponding directed graph of equipment anomaly propagation is constructed. The multi-view feature deviation vector is then matched with the directed graph of equipment anomaly propagation to determine whether the target equipment is normal based on the matching result.

[0007] The beneficial effects of this invention are as follows: This solution accurately extracts image sequences of the complete operating cycle of each device in a substation by constructing a spatial mapping relationship of overlapping areas of different monitoring camera fields of view, thus solving the problem of incomplete equipment status observation caused by the limited field of view of a single camera. Based on the periodic movement and appearance change characteristics of the equipment, a periodic operation benchmark template is generated. Combined with parallel image sequence processing and time-series alignment to calculate time-series deviation, this significantly improves the processing efficiency of single-frame detection and filters out false alarms caused by instantaneous interference through the time-series dimension. When a time-series deviation exceeds the standard, multi-camera collaborative observation is triggered to generate a multi-view feature deviation vector. Multi-view cross-validation is then used to further eliminate single-camera imaging interference. Finally, a directed graph of anomaly propagation constructed from the physical connection relationship of the equipment and the energy transfer path is used for matching and judgment, verifying the authenticity of the anomaly from a physical mechanism perspective. This forms a systematic false alarm filtering mechanism covering space, time, multiple views, and physical mechanisms, effectively addressing the complex and ever-changing interference factors in substations and significantly improving the accuracy of equipment anomaly detection and the overall operating efficiency of the system.

[0008] Furthermore, the step of performing time-series alignment processing between the device motion region and the corresponding device periodic operation reference template to calculate the time-series deviation between the motion characteristics and the reference template includes: Dense optical computation is performed on the moving area of ​​the equipment to obtain a one-dimensional velocity time-series curve of the core moving component of the equipment. The velocity signal in the one-dimensional velocity time-series curve is converted into a complex analytic signal by Hilbert transform to solve for the continuous instantaneous phase and instantaneous frequency, and generate phase-frequency joint time-series characteristics. The phase-frequency joint timing feature is used as the input signal, the standard phase-frequency curve pre-stored in the periodic operation reference template is used as the reference signal, and the output phase is dynamically adjusted by a proportional-integral controller to keep the input signal and the reference signal in phase synchronization and output the corresponding phase error signal. Based on a preset phase jump detection window, the phase error signal is dynamically calculated and processed to generate the corresponding timing deviation.

[0009] Furthermore, the step of dynamically calculating and processing the phase error signal based on a preset phase jump detection window to generate the corresponding timing deviation includes: The phase error signal is decomposed into several modal function components and several residual trend components by means of an adaptive variational mode decomposition algorithm. Based on the multiple relationship between the center frequency of each component and the rated operating frequency of the equipment and the energy ratio, several effective modal components are selected accordingly. Hilbert transform is performed on each of the effective modal components to detect the corresponding instantaneous amplitude spectrum and instantaneous phase spectrum. Based on the instantaneous amplitude spectrum and instantaneous phase spectrum, the amplitude fluctuation coefficient, phase variance entropy and phase synchronization index between modes of each effective modal component within the detection window are calculated to construct a three-dimensional modal feature vector. The Mahalanobis distance between the three-dimensional modal feature vector and the baseline modal feature distribution under normal operating conditions is calculated to generate the corresponding time series deviation.

[0010] Furthermore, the step of scheduling all cameras covering the target device to simultaneously detect the corresponding device images, and calculating the feature deviation value of the same device in each of the device images to generate a multi-view feature deviation vector includes: Based on the intrinsic and extrinsic parameter matrices of each camera and the pre-stored 3D point cloud model of the target device, a local view projection matrix for each camera is generated, and each device image is mapped to a unified device body coordinate system through the local view projection matrix to generate a view-normalized image. The viewpoint-normalized image is segmented into several non-overlapping sub-regions, and the gradient histogram features and local binary pattern features of each sub-region are detected accordingly, and then concatenated to obtain the sub-region feature vector. The cosine distance between each sub-region feature vector and the corresponding sub-region reference feature vector in the periodic operation reference template is calculated to generate the original sub-region deviation value. The original sub-region deviation values ​​are then weighted and fused to generate the corresponding multi-view feature deviation vector.

[0011] Furthermore, the step of weighted fusion processing of the original deviation values ​​of each sub-region to generate the corresponding multi-view feature deviation vector includes: The projection error covariance matrix of each camera in the device body coordinate system is calculated by the error propagation law, and the projection error covariance of the geometric center point of each sub-region on the image plane of each camera is calculated based on the projection error covariance matrix. The projection error covariance is used as the observation weight coefficient for each sub-region. For each sub-region, the original deviation value of the sub-region corresponding to all cameras and its corresponding observation weight coefficient are multiplied and summed to calculate the initial fusion deviation value of the sub-region. The initial fusion deviation values ​​of each sub-region are integrated to generate the corresponding multi-view feature deviation vector.

[0012] Furthermore, the step of constructing a corresponding directed graph of equipment anomaly propagation based on the physical connection relationships and energy transfer paths of the substation equipment includes: Based on the electrical wiring relationships, mechanical connection relationships, and heat conduction paths between the substation equipment, electrical connection edges, mechanical connection edges, and thermal connection edges are constructed respectively, wherein the direction of each edge is consistent with the direction of corresponding energy transfer; For the electrical connection edge, the electrical propagation coefficient is calculated based on the rated voltage, rated current and line impedance of the devices at both ends of the connection. For the mechanical connection edge, the mechanical propagation coefficient is calculated based on the material stiffness, damping coefficient and contact area of ​​the connecting components. For the thermal connection edge, the thermal propagation coefficient is calculated based on the thermal conductivity, cross-sectional area and length of the connecting medium. Each propagation coefficient is used as a weight attribute of the corresponding edge. Based on the electrical connection edge, the mechanical connection edge, and the thermal connection edge, respectively construct electrical anomaly propagation subgraphs, mechanical anomaly propagation subgraphs, and thermal anomaly propagation subgraphs, and merge the subgraphs to construct the corresponding directed graph of equipment anomaly propagation.

[0013] Furthermore, the step of matching the multi-view feature deviation vector with the directed graph of device anomaly propagation to determine whether the target device is normal based on the matching result includes: Each element in the multi-view feature deviation vector is mapped to the observation degree of the corresponding node in the directed graph of device anomaly propagation, and a node observation vector is constructed based on each observation degree. A set of normal operation data is detected in the normal operation history database of the target device, and a normal observation vector is constructed based on the set of normal operation data. The Euclidean distance between the node observation vector and the normal observation vector is calculated, and it is determined whether the Euclidean distance meets the preset requirements. If the Euclidean distance meets the preset requirements, the target device is determined to be in a normal state.

[0014] The second aspect of the present invention proposes: A machine vision-based substation equipment anomaly detection system, wherein the system includes: The module is used to construct a spatial mapping relationship of the overlapping areas of different camera fields of view based on the installation position, field of view angle and calibration parameters of the substation monitoring camera, and to detect the image sequence of each device in a complete operating cycle based on the spatial mapping relationship. The processing module is used to detect the periodic motion features and appearance change features of each device according to the image sequence, so as to generate a device periodic operation reference template, and to perform parallel processing on each image sequence to detect the device motion area and device static area in each frame image, and to perform time-series alignment processing between the device motion area and the corresponding device periodic operation reference template to calculate the time-series deviation between the motion features and the reference template. The calculation module is used to detect the corresponding target device when the timing deviation of any monitoring camera exceeds a preset deviation threshold, and trigger a multi-camera collaborative observation command to schedule all cameras covering the target device to simultaneously detect the corresponding device image, and calculate the feature deviation value of the same device in each device image to generate a multi-view feature deviation vector. The judgment module is used to construct a corresponding directed graph of equipment anomaly propagation based on the physical connection relationship and energy transfer path of the substation equipment, and to match the multi-view feature deviation vector with the directed graph of equipment anomaly propagation to determine whether the target equipment is normal based on the matching result.

[0015] Furthermore, the processing module is specifically used for: Dense optical computation is performed on the moving area of ​​the equipment to obtain a one-dimensional velocity time-series curve of the core moving component of the equipment. The velocity signal in the one-dimensional velocity time-series curve is converted into a complex analytic signal by Hilbert transform to solve for the continuous instantaneous phase and instantaneous frequency, and generate phase-frequency joint time-series characteristics. The phase-frequency joint timing feature is used as the input signal, the standard phase-frequency curve pre-stored in the periodic operation reference template is used as the reference signal, and the output phase is dynamically adjusted by a proportional-integral controller to keep the input signal and the reference signal in phase synchronization and output the corresponding phase error signal. Based on a preset phase jump detection window, the phase error signal is dynamically calculated and processed to generate the corresponding timing deviation.

[0016] Furthermore, the processing module is specifically used for: The phase error signal is decomposed into several modal function components and several residual trend components by means of an adaptive variational mode decomposition algorithm. Based on the multiple relationship between the center frequency of each component and the rated operating frequency of the equipment and the energy ratio, several effective modal components are selected accordingly. Hilbert transform is performed on each of the effective modal components to detect the corresponding instantaneous amplitude spectrum and instantaneous phase spectrum. Based on the instantaneous amplitude spectrum and instantaneous phase spectrum, the amplitude fluctuation coefficient, phase variance entropy and phase synchronization index between modes of each effective modal component within the detection window are calculated to construct a three-dimensional modal feature vector. The Mahalanobis distance between the three-dimensional modal feature vector and the baseline modal feature distribution under normal operating conditions is calculated to generate the corresponding time series deviation.

[0017] Furthermore, the calculation module is specifically used for: Based on the intrinsic and extrinsic parameter matrices of each camera and the pre-stored 3D point cloud model of the target device, a local view projection matrix for each camera is generated, and each device image is mapped to a unified device body coordinate system through the local view projection matrix to generate a view-normalized image. The viewpoint-normalized image is segmented into several non-overlapping sub-regions, and the gradient histogram features and local binary pattern features of each sub-region are detected accordingly, and then concatenated to obtain the sub-region feature vector. The cosine distance between each sub-region feature vector and the corresponding sub-region reference feature vector in the periodic operation reference template is calculated to generate the original sub-region deviation value. The original sub-region deviation values ​​are then weighted and fused to generate the corresponding multi-view feature deviation vector.

[0018] Furthermore, the calculation module is specifically used for: The projection error covariance matrix of each camera in the device body coordinate system is calculated by the error propagation law, and the projection error covariance of the geometric center point of each sub-region on the image plane of each camera is calculated based on the projection error covariance matrix. The projection error covariance is used as the observation weight coefficient for each sub-region. For each sub-region, the original deviation value of the sub-region corresponding to all cameras and its corresponding observation weight coefficient are multiplied and summed to calculate the initial fusion deviation value of the sub-region. The initial fusion deviation values ​​of each sub-region are integrated to generate the corresponding multi-view feature deviation vector.

[0019] Furthermore, the determination module is specifically used for: Based on the electrical wiring relationships, mechanical connection relationships, and heat conduction paths between the substation equipment, electrical connection edges, mechanical connection edges, and thermal connection edges are constructed respectively, wherein the direction of each edge is consistent with the direction of corresponding energy transfer; For the electrical connection edge, the electrical propagation coefficient is calculated based on the rated voltage, rated current and line impedance of the devices at both ends of the connection. For the mechanical connection edge, the mechanical propagation coefficient is calculated based on the material stiffness, damping coefficient and contact area of ​​the connecting components. For the thermal connection edge, the thermal propagation coefficient is calculated based on the thermal conductivity, cross-sectional area and length of the connecting medium. Each propagation coefficient is used as a weight attribute of the corresponding edge. Based on the electrical connection edge, the mechanical connection edge, and the thermal connection edge, respectively construct electrical anomaly propagation subgraphs, mechanical anomaly propagation subgraphs, and thermal anomaly propagation subgraphs, and merge the subgraphs to construct the corresponding directed graph of equipment anomaly propagation.

[0020] Furthermore, the determination module is specifically used for: Each element in the multi-view feature deviation vector is mapped to the observation degree of the corresponding node in the directed graph of device anomaly propagation, and a node observation vector is constructed based on each observation degree. A set of normal operation data is detected in the normal operation history database of the target device, and a normal observation vector is constructed based on the set of normal operation data. The Euclidean distance between the node observation vector and the normal observation vector is calculated, and it is determined whether the Euclidean distance meets the preset requirements. If the Euclidean distance meets the preset requirements, the target device is determined to be in a normal state.

[0021] The third aspect of the present invention proposes: A computer includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the machine vision-based substation equipment anomaly detection method as described above.

[0022] The fourth aspect of the present invention proposes: A readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the machine vision-based substation equipment anomaly detection method as described above.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] Figure 1 A flowchart of a substation equipment anomaly detection method based on machine vision provided in the first embodiment of the present invention; Figure 2 The structural block diagram of the substation equipment anomaly detection system based on machine vision provided in the third embodiment of the present invention is shown.

[0025] The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0026] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0027] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0029] Please seeFigure 1 The image shows a substation equipment anomaly detection method based on machine vision provided in the first embodiment of the present invention. The substation equipment anomaly detection method based on machine vision provided in this embodiment can effectively cope with the complex and ever-changing interference factors in substations, and significantly improve the accuracy of equipment anomaly detection and the overall system operating efficiency.

[0030] Specifically, this embodiment provides: A machine vision-based method for detecting anomalies in substation equipment, wherein the method includes: Step S10: Based on the installation location, field of view, and calibration parameters of the substation monitoring cameras, a spatial mapping relationship of the overlapping areas of different camera fields of view is constructed, and based on the spatial mapping relationship, the image sequence of each device in the complete operating cycle is detected. It should be noted that substations typically deploy dozens to hundreds of monitoring cameras, with significant overlap in the fields of view of different cameras. The same device may be observed simultaneously by multiple cameras. This step uses intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (position, attitude) obtained from camera calibration to calculate the mapping relationship between any pixel within the overlapping area and the image planes of different cameras, achieving spatial correlation of observation results from different cameras. Based on this, according to the device's installation location and the camera's field of view, all image sequences for each device throughout its complete operating cycle are automatically extracted, ensuring coverage of all operating states of the device and providing a complete data foundation for subsequent benchmark template construction and anomaly detection.

[0031] Step S20: Based on the image sequence, periodic motion features and appearance change features of each device are detected to generate a device periodic operation reference template. The image sequences are processed in parallel to detect the device motion region and device static region in each frame image. The device motion region is time-aligned with the corresponding device periodic operation reference template to calculate the time deviation between the motion features and the reference template. It should be noted that many devices within a substation (such as transformer cooling fans, oil pumps, circuit breaker energy storage motors, and disconnector drive mechanisms) exhibit fixed periodic motion patterns. Under normal circumstances, the phase, frequency, and amplitude of their motion remain highly stable. Early anomalies (such as bearing wear, missing gear teeth, and motor aging) will initially manifest as slight deviations in the motion timing, while changes in appearance often lag behind these changes. This step analyzes the historical image sequences of the equipment to extract its periodic motion and static appearance features, generating a periodic operation benchmark template containing standard motion patterns and appearance information. During real-time detection, background subtraction and optical flow methods are used to separate the moving and static regions of the equipment in each frame of the image. Only the moving regions undergo time-series alignment processing, significantly reducing computational load. By comparing real-time motion features with the benchmark template, the time-series deviation is calculated. When the time-series deviation exceeds a preset threshold, the equipment is determined to have a potential anomaly, triggering subsequent multi-camera collaborative observation. This initial screening mechanism can complete the anomaly screening of all equipment in the substation within milliseconds, while ensuring a high detection rate of early anomalies.

[0032] Step S30: When the timing deviation of any monitoring camera exceeds the preset deviation threshold, the corresponding target device is detected and a multi-camera collaborative observation command is triggered to schedule all cameras covering the target device to simultaneously detect the corresponding device image, and calculate the feature deviation value of the same device in each device image to generate a multi-view feature deviation vector. It should be noted that single-camera detection is susceptible to environmental interference, such as overexposure due to direct sunlight, incomplete observation due to equipment component obstruction, and feature distortion due to angular deviation, all of which can lead to false alarms. In this step, when a potential anomaly is initially detected, all cameras covering the target equipment's field of view are immediately deployed to simultaneously acquire multi-view images of the target equipment. Through viewpoint normalization and feature extraction, the feature deviation value of the equipment at each viewpoint is calculated. The deviation values ​​of all viewpoints are then merged into a unified multi-view feature deviation vector, comprehensively reflecting the abnormal performance of the equipment at different angles, effectively eliminating interference from single cameras, and significantly reducing the false alarm rate.

[0033] Step S40: Based on the physical connection relationship and energy transfer path of the substation equipment, construct a corresponding directed graph of equipment anomaly propagation, and match the multi-view feature deviation vector with the directed graph of equipment anomaly propagation to determine whether the target equipment is normal based on the matching result.

[0034] It's important to note that a substation is a tightly coupled and complex system. Equipment malfunctions can propagate to adjacent equipment along electrical lines, mechanical connections, and heat conduction paths. For example, a rise in transformer oil temperature might be caused by a cooling fan malfunction, not a problem with the transformer itself; abnormal bus voltage can lead to abnormal operating parameters in all outgoing equipment. This step constructs a directed graph of equipment malfunction propagation based on the substation's electrical wiring diagram, equipment installation diagram, and heat conduction characteristics, clarifying the direction and extent of malfunction propagation. Multi-view feature deviation vectors are matched with the propagation graph to analyze the propagation path and causal relationships, ultimately determining whether the malfunction of the target equipment is due to a fault in the equipment itself or a secondary malfunction caused by upstream propagation. This provides maintenance personnel with accurate malfunction location and handling recommendations.

[0035] Second Embodiment Furthermore, the step of performing time-series alignment processing between the device motion region and the corresponding device periodic operation reference template to calculate the time-series deviation between the motion characteristics and the reference template includes: Dense optical computation is performed on the moving area of ​​the equipment to obtain a one-dimensional velocity time-series curve of the core moving component of the equipment. The velocity signal in the one-dimensional velocity time-series curve is converted into a complex analytic signal by Hilbert transform to solve for the continuous instantaneous phase and instantaneous frequency, and generate phase-frequency joint time-series characteristics. The phase-frequency joint timing feature is used as the input signal, the standard phase-frequency curve pre-stored in the periodic operation reference template is used as the reference signal, and the output phase is dynamically adjusted by a proportional-integral controller to keep the input signal and the reference signal in phase synchronization and output the corresponding phase error signal. Based on a preset phase jump detection window, the phase error signal is dynamically calculated and processed to generate the corresponding timing deviation.

[0036] It's important to note that traditional frame difference or block matching methods only provide coarse motion information and cannot capture minute speed changes. Dense optical flow (DIF) can calculate the motion vector of each pixel within a moving region. By averaging the optical flow vectors of core moving components (such as fan blades or oil pump shafts), a high-precision one-dimensional velocity timing curve is obtained. Based on this, a Hilbert transform is introduced to convert the real-valued velocity signal into a complex analytic signal, thereby calculating the continuous instantaneous phase and frequency. Compared to amplitude features, phase features are more sensitive to minute motion changes: for example, a 1% decrease in cooling fan speed results in an almost undetectable amplitude change, but the phase deviation accumulates over time and can be clearly captured. The generated phase-frequency joint timing features comprehensively characterize the dynamic properties of the device's motion, providing a highly sensitive feature foundation for subsequent timing alignment and deviation calculation.

[0037] The operating frequency of equipment may experience minor normal fluctuations due to factors such as grid voltage fluctuations and load changes. Directly comparing the phase difference could misinterpret these normal fluctuations as abnormalities. This step employs a proportional-integral (PI) controller to construct a phase synchronization closed-loop system: the real-time input phase is compared with a reference standard phase to obtain the phase error; the PI controller dynamically adjusts the output phase based on the phase error, ensuring that the input signal always tracks the phase changes of the reference signal. When the equipment is operating normally, the phase error will fluctuate within a small range near zero; when the equipment malfunctions, the motion pattern changes, the PI controller cannot fully track the phase changes, and the phase error gradually increases, forming a significant deviation signal. This synchronization mechanism effectively distinguishes between normal frequency fluctuations and abnormal phase shifts, significantly improving the detection's anti-interference capability.

[0038] Phase error signals contain high-frequency noise, random fluctuations, and abnormal deviations, requiring smoothing and statistical analysis using a sliding window. This step sets a phase jump detection window of a preset length, calculating the mean, variance, and maximum deviation of the phase error within each window; the timing deviation value is dynamically updated using a sliding window. When the timing deviation exceeds a preset threshold, it indicates a significant abnormal change in the device's motion pattern, triggering subsequent multi-camera collaborative observation.

[0039] Furthermore, the step of dynamically calculating and processing the phase error signal based on a preset phase jump detection window to generate the corresponding timing deviation includes: The phase error signal is decomposed into several modal function components and several residual trend components by means of an adaptive variational mode decomposition algorithm. Based on the multiple relationship between the center frequency of each component and the rated operating frequency of the equipment and the energy ratio, several effective modal components are selected accordingly. Hilbert transform is performed on each of the effective modal components to detect the corresponding instantaneous amplitude spectrum and instantaneous phase spectrum. Based on the instantaneous amplitude spectrum and instantaneous phase spectrum, the amplitude fluctuation coefficient, phase variance entropy and phase synchronization index between modes of each effective modal component within the detection window are calculated to construct a three-dimensional modal feature vector. The Mahalanobis distance between the three-dimensional modal feature vector and the baseline modal feature distribution under normal operating conditions is calculated to generate the corresponding time series deviation.

[0040] It's important to note that the phase error signal is a typical non-stationary signal, containing components of different frequencies: high-frequency components mainly consist of electromagnetic interference and image noise, while low-frequency components primarily reflect the slow trend caused by equipment aging. Only components related to the equipment's rated operating frequency and its harmonics are the effective signals reflecting abnormal equipment motion. Adaptive Variational Mode Decomposition (AVMD) is an adaptive multi-scale signal decomposition method that automatically determines the number of modes to be decomposed based on the signal characteristics, avoiding the mode aliasing problem of traditional empirical mode decomposition. This step uses AVMD to decompose the phase error signal, obtaining several mode function components with different center frequencies and a residual trend component. Then, based on whether the center frequency of the component is an integer multiple of the equipment's rated operating frequency and the component's energy proportion, the effective mode components related to equipment motion are selected, while noise and trend components are eliminated, providing a clean signal foundation for subsequent feature extraction.

[0041] A single mean phase error cannot fully characterize the complex characteristics of anomalies; a comprehensive analysis from three dimensions—amplitude, phase, and intermodal synchronicity—is necessary. This step performs a Hilbert transform on each effective modal component again to obtain its instantaneous amplitude and phase spectra. Based on these, three core features are calculated: the amplitude fluctuation coefficient, reflecting the stability of the modal amplitude (anomalies increase amplitude fluctuation); the phase variance entropy, reflecting the dispersion of the phase distribution (anomalies cause phase disorder and increase entropy); and the intermodal phase synchronization index, reflecting the phase synchronization between different modes (equipment failures disrupt the motion synchronization between components, leading to a decrease in the synchronization index). Combining these three features into a three-dimensional modal feature vector provides a more comprehensive and detailed characterization of the anomaly characteristics of the phase error signal.

[0042] Compared to traditional Euclidean distance, Mahalanobis distance considers the correlation between features and the variance differences across dimensions, enabling a more accurate measurement of the anomaly degree of multidimensional features. This step establishes a multivariate Gaussian distribution of baseline modal features based on a large amount of historical data from normal equipment operation. The Mahalanobis distance between the current three-dimensional modal feature vector and this distribution is calculated. A larger Mahalanobis distance indicates a greater deviation of the current features from the normal state and a larger temporal deviation. This deviation calculation method based on statistical distribution can adapt to the operating characteristics of different devices, exhibiting stronger versatility and robustness.

[0043] Furthermore, the step of scheduling all cameras covering the target device to simultaneously detect the corresponding device images, and calculating the feature deviation value of the same device in each of the device images to generate a multi-view feature deviation vector includes: Based on the intrinsic and extrinsic parameter matrices of each camera and the pre-stored 3D point cloud model of the target device, a local view projection matrix for each camera is generated, and each device image is mapped to a unified device body coordinate system through the local view projection matrix to generate a view-normalized image. The viewpoint-normalized image is segmented into several non-overlapping sub-regions, and the gradient histogram features and local binary pattern features of each sub-region are detected accordingly, and then concatenated to obtain the sub-region feature vector. The cosine distance between each sub-region feature vector and the corresponding sub-region reference feature vector in the periodic operation reference template is calculated to generate the original sub-region deviation value. The original sub-region deviation values ​​are then weighted and fused to generate the corresponding multi-view feature deviation vector.

[0044] It's important to note that different cameras, with varying installation positions and orientations, will produce images of the same device with different perspective distortions, sizes, and angles, making direct feature extraction unsuitable for comparison. This step utilizes a pre-generated high-precision 3D point cloud model of the target device, combined with the intrinsic and extrinsic parameters of each camera, to calculate the local projection matrix from that camera's perspective. The projection matrix is ​​then used to back-project each pixel in the 2D image onto the 3D device's body coordinate system, and then reproject it onto the frontal view plane, generating a perspective-normalized image. After normalization, images of the same device captured by all cameras have the same size, angle, and perspective, eliminating the influence of perspective differences and laying the foundation for subsequent feature comparison.

[0045] The probability and degree of impact of anomalies vary in different parts of the equipment. For example, the contact parts of a circuit breaker and the bushing parts of a transformer are critical areas, while minor corrosion of the outer casing has a smaller impact. This step divides the viewpoint-normalized image into several non-overlapping sub-regions according to the structure of the equipment, with each sub-region corresponding to a specific component of the equipment. Gradient histogram (HOG) features and Local Binary Pattern (LBP) features are extracted for each sub-region: HOG features are sensitive to edge and shape changes and are suitable for detecting anomalies such as deformation and displacement of the equipment; LBP features are sensitive to texture changes and are suitable for detecting appearance anomalies such as corrosion, oil leakage, and damage of the equipment. The two types of features are concatenated into a sub-region feature vector, which can comprehensively capture the appearance and structural changes of the sub-region.

[0046] Cosine distance is a commonly used metric for measuring the similarity between two feature vectors. A larger distance indicates a greater difference between the two features and a higher degree of anomaly. This step calculates the cosine distance between the feature vector of each sub-region and the baseline feature vector from each viewpoint, obtaining the original deviation value for the sub-region. Then, the deviation values ​​of the sub-regions from all viewpoints are weighted and fused to obtain the final deviation value for each sub-region. The final deviation values ​​of all sub-regions are then arranged in order to generate a multi-view feature deviation vector. This vector not only reflects the overall degree of anomaly of the equipment but also locates the specific location where the anomaly occurred, providing refined information for subsequent anomaly analysis.

[0047] Furthermore, the step of weighted fusion processing of the original deviation values ​​of each sub-region to generate the corresponding multi-view feature deviation vector includes: The projection error covariance matrix of each camera in the device body coordinate system is calculated by the error propagation law, and the projection error covariance of the geometric center point of each sub-region on the image plane of each camera is calculated based on the projection error covariance matrix. The projection error covariance is used as the observation weight coefficient for each sub-region. For each sub-region, the original deviation value of the sub-region corresponding to all cameras and its corresponding observation weight coefficient are multiplied and summed to calculate the initial fusion deviation value of the sub-region. The initial fusion deviation values ​​of each sub-region are integrated to generate the corresponding multi-view feature deviation vector.

[0048] It should be noted that there are significant differences in observation accuracy among different cameras: cameras closer to the device, with higher resolution, and at more precise angles exhibit higher observation accuracy and smaller projection errors; conversely, the farther away from the device, the lower the observation accuracy and the larger the projection error. This step, based on the camera's intrinsic and extrinsic parameter errors and the accuracy of the 3D point cloud model, calculates the 3D projection error covariance matrix of each camera in the device's body coordinate system using the error propagation law. Then, the geometric center point of each sub-region is projected onto the image plane of each camera, and the corresponding 2D projection error covariance is calculated. The magnitude of the projection error covariance directly reflects the camera's observation accuracy for that sub-region: the smaller the covariance, the higher the observation accuracy and the higher the reliability.

[0049] Traditional averaging fusion methods do not consider the differences in observation accuracy between different cameras, leading to the lower overall fusion accuracy due to the observations from low-precision cameras. This step uses the reciprocal of the projection error covariance as the observation weight coefficient: higher observation accuracy results in a smaller projection error covariance and a larger weight; lower observation accuracy results in a larger projection error covariance and a smaller weight. For each sub-region, the initial fusion bias value for the sub-region is obtained by multiplying the original bias values ​​of all cameras by their corresponding weight coefficients and summing the results. This adaptive weighted fusion method fully utilizes the observations from high-precision cameras, suppresses interference from low-precision cameras, and significantly improves the accuracy of the fusion bias.

[0050] This step is the final stage in generating the multi-view feature bias vector. It arranges the initial fusion bias values ​​of all sub-regions according to the structural order of the devices to generate the final multi-view feature bias vector. This vector integrates the observation results from all cameras covering the target device, with each element corresponding to the anomaly degree of a sub-region of the device. This eliminates the observation errors of a single camera while preserving refined information about local anomalies, providing accurate input for subsequent anomaly propagation map matching.

[0051] Furthermore, the step of constructing a corresponding directed graph of equipment anomaly propagation based on the physical connection relationships and energy transfer paths of the substation equipment includes: Based on the electrical wiring relationships, mechanical connection relationships, and heat conduction paths between the substation equipment, electrical connection edges, mechanical connection edges, and thermal connection edges are constructed respectively, wherein the direction of each edge is consistent with the direction of corresponding energy transfer; For the electrical connection edge, the electrical propagation coefficient is calculated based on the rated voltage, rated current and line impedance of the devices at both ends of the connection. For the mechanical connection edge, the mechanical propagation coefficient is calculated based on the material stiffness, damping coefficient and contact area of ​​the connecting components. For the thermal connection edge, the thermal propagation coefficient is calculated based on the thermal conductivity, cross-sectional area and length of the connecting medium. Each propagation coefficient is used as a weight attribute of the corresponding edge. Based on the electrical connection edge, the mechanical connection edge, and the thermal connection edge, respectively construct electrical anomaly propagation subgraphs, mechanical anomaly propagation subgraphs, and thermal anomaly propagation subgraphs, and merge the subgraphs to construct the corresponding directed graph of equipment anomaly propagation.

[0052] It's important to note that anomalies in substation equipment propagate primarily through three paths: electrical paths, where anomalies flow along electrical lines from the power source to the load side (e.g., a busbar fault can cause all outgoing equipment to lose power); mechanical paths, where anomalies propagate through mechanical connections (e.g., transformer vibrations can be transmitted to adjacent equipment via the foundation); and thermal paths, where anomalies propagate through heat conduction and radiation (e.g., reactor heating can cause surrounding equipment temperatures to rise). This step constructs electrical connection edges based on the substation's electrical wiring diagram, aligning with the current direction; constructs mechanical connection edges based on the equipment's installation layout diagram, aligning with the vibration transmission direction; and constructs thermal connection edges based on the equipment's spatial location and thermal conductivity characteristics, pointing from high-temperature equipment to low-temperature equipment. These three connection edges comprehensively cover all possible propagation paths of anomalies.

[0053] A larger propagation coefficient indicates that the anomaly propagates more easily along that path, resulting in a greater impact on downstream equipment. This step calculates the propagation coefficient using corresponding physical formulas for different types of connection edges: the electrical propagation coefficient is inversely proportional to the line impedance; the smaller the impedance, the easier it is for electrical anomalies to propagate. The mechanical propagation coefficient is directly proportional to the connection stiffness and inversely proportional to the damping; the greater the stiffness and the smaller the damping, the easier the vibration propagates. The thermal propagation coefficient is directly proportional to the thermal conductivity and cross-sectional area of ​​the medium, and inversely proportional to the length; the higher the thermal conductivity, the larger the cross-sectional area, and the shorter the length, the easier the heat conduction. Using the propagation coefficient as the edge weight allows the directed graph of anomaly propagation to not only reflect the propagation direction but also quantify the propagation intensity.

[0054] During the merging process, for multiple edges connecting the same pair of devices, all edges and their weights are retained to reflect the propagation characteristics of different types of anomalies. The final generated directed graph of device anomaly propagation, with substation devices as nodes, three types of propagation edges as connections, and propagation coefficients as weights, comprehensively and quantitatively characterizes the propagation patterns of anomalies within the substation, providing a physical mechanism basis for subsequent anomaly reasoning and source tracing.

[0055] Furthermore, the step of matching the multi-view feature deviation vector with the directed graph of device anomaly propagation to determine whether the target device is normal based on the matching result includes: Each element in the multi-view feature deviation vector is mapped to the observation degree of the corresponding node in the directed graph of device anomaly propagation, and a node observation vector is constructed based on each observation degree. A set of normal operation data is detected in the normal operation history database of the target device, and a normal observation vector is constructed based on the set of normal operation data. The Euclidean distance between the node observation vector and the normal observation vector is calculated, and it is determined whether the Euclidean distance meets the preset requirements. If the Euclidean distance meets the preset requirements, the target device is determined to be in a normal state.

[0056] It should be noted that each element in the multi-view feature deviation vector corresponds to the anomaly level of a sub-region of the device. The deviation values ​​of all sub-regions of the same device are weighted and summed to obtain the overall anomaly observation degree of the device; the higher the observation degree, the more obvious the anomaly performance of the device. The observation degrees of all devices are arranged in node order to generate a node observation vector, which reflects the anomaly observation results of all devices in the entire station at the current time.

[0057] This step is crucial for establishing the normal baseline. Based on historical data from the long-term normal operation of the target equipment, the mean and variance of the observations for each device are calculated, and a normal observation vector and normal observation distribution are constructed. The normal observation vector represents the observation level of the equipment under normal conditions and serves as the benchmark for identifying anomalies.

[0058] When the Euclidean distance exceeds a preset threshold, it indicates an abnormal equipment status. Based on this, further analysis is performed using a directed graph of equipment anomaly propagation: if the target equipment has a high observability and all its upstream equipment has low observability, it is determined to be an anomaly in the target equipment itself; if the upstream equipment of the target equipment has an even higher observability and a larger propagation coefficient, it is determined to be a secondary anomaly caused by the propagation of the upstream anomaly. This determination method, which combines physical propagation laws, can effectively distinguish between intrinsic anomalies and propagation anomalies, significantly reducing the false alarm rate, while achieving precise location of the anomaly source, providing a scientific basis for the operation and maintenance of substations.

[0059] Please see Figure 2 The third embodiment of the present invention provides: A machine vision-based substation equipment anomaly detection system, wherein the system includes: The module is used to construct a spatial mapping relationship of the overlapping areas of different camera fields of view based on the installation position, field of view angle and calibration parameters of the substation monitoring camera, and to detect the image sequence of each device in a complete operating cycle based on the spatial mapping relationship. The processing module is used to detect the periodic motion features and appearance change features of each device according to the image sequence, so as to generate a device periodic operation reference template, and to perform parallel processing on each image sequence to detect the device motion area and device static area in each frame image, and to perform time-series alignment processing between the device motion area and the corresponding device periodic operation reference template to calculate the time-series deviation between the motion features and the reference template. The calculation module is used to detect the corresponding target device when the timing deviation of any monitoring camera exceeds a preset deviation threshold, and trigger a multi-camera collaborative observation command to schedule all cameras covering the target device to simultaneously detect the corresponding device image, and calculate the feature deviation value of the same device in each device image to generate a multi-view feature deviation vector. The judgment module is used to construct a corresponding directed graph of equipment anomaly propagation based on the physical connection relationship and energy transfer path of the substation equipment, and to match the multi-view feature deviation vector with the directed graph of equipment anomaly propagation to determine whether the target equipment is normal based on the matching result.

[0060] Furthermore, the processing module is specifically used for: Dense optical computation is performed on the moving area of ​​the equipment to obtain a one-dimensional velocity time-series curve of the core moving component of the equipment. The velocity signal in the one-dimensional velocity time-series curve is converted into a complex analytic signal by Hilbert transform to solve for the continuous instantaneous phase and instantaneous frequency, and generate phase-frequency joint time-series characteristics. The phase-frequency joint timing feature is used as the input signal, the standard phase-frequency curve pre-stored in the periodic operation reference template is used as the reference signal, and the output phase is dynamically adjusted by a proportional-integral controller to keep the input signal and the reference signal in phase synchronization and output the corresponding phase error signal. Based on a preset phase jump detection window, the phase error signal is dynamically calculated and processed to generate the corresponding timing deviation.

[0061] Furthermore, the processing module is specifically used for: The phase error signal is decomposed into several modal function components and several residual trend components by means of an adaptive variational mode decomposition algorithm. Based on the multiple relationship between the center frequency of each component and the rated operating frequency of the equipment and the energy ratio, several effective modal components are selected accordingly. Hilbert transform is performed on each of the effective modal components to detect the corresponding instantaneous amplitude spectrum and instantaneous phase spectrum. Based on the instantaneous amplitude spectrum and instantaneous phase spectrum, the amplitude fluctuation coefficient, phase variance entropy and phase synchronization index between modes of each effective modal component within the detection window are calculated to construct a three-dimensional modal feature vector. The Mahalanobis distance between the three-dimensional modal feature vector and the baseline modal feature distribution under normal operating conditions is calculated to generate the corresponding time series deviation.

[0062] Furthermore, the calculation module is specifically used for: Based on the intrinsic and extrinsic parameter matrices of each camera and the pre-stored 3D point cloud model of the target device, a local view projection matrix for each camera is generated, and each device image is mapped to a unified device body coordinate system through the local view projection matrix to generate a view-normalized image. The viewpoint-normalized image is segmented into several non-overlapping sub-regions, and the gradient histogram features and local binary pattern features of each sub-region are detected accordingly, and then concatenated to obtain the sub-region feature vector. The cosine distance between each sub-region feature vector and the corresponding sub-region reference feature vector in the periodic operation reference template is calculated to generate the original sub-region deviation value. The original sub-region deviation values ​​are then weighted and fused to generate the corresponding multi-view feature deviation vector.

[0063] Furthermore, the calculation module is specifically used for: The projection error covariance matrix of each camera in the device body coordinate system is calculated by the error propagation law, and the projection error covariance of the geometric center point of each sub-region on the image plane of each camera is calculated based on the projection error covariance matrix. The projection error covariance is used as the observation weight coefficient for each sub-region. For each sub-region, the original deviation value of the sub-region corresponding to all cameras and its corresponding observation weight coefficient are multiplied and summed to calculate the initial fusion deviation value of the sub-region. The initial fusion deviation values ​​of each sub-region are integrated to generate the corresponding multi-view feature deviation vector.

[0064] Furthermore, the determination module is specifically used for: Based on the electrical wiring relationships, mechanical connection relationships, and heat conduction paths between the substation equipment, electrical connection edges, mechanical connection edges, and thermal connection edges are constructed respectively, wherein the direction of each edge is consistent with the direction of corresponding energy transfer; For the electrical connection edge, the electrical propagation coefficient is calculated based on the rated voltage, rated current and line impedance of the devices at both ends of the connection. For the mechanical connection edge, the mechanical propagation coefficient is calculated based on the material stiffness, damping coefficient and contact area of ​​the connecting components. For the thermal connection edge, the thermal propagation coefficient is calculated based on the thermal conductivity, cross-sectional area and length of the connecting medium. Each propagation coefficient is used as a weight attribute of the corresponding edge. Based on the electrical connection edge, the mechanical connection edge, and the thermal connection edge, respectively construct electrical anomaly propagation subgraphs, mechanical anomaly propagation subgraphs, and thermal anomaly propagation subgraphs, and merge the subgraphs to construct the corresponding directed graph of equipment anomaly propagation.

[0065] Furthermore, the determination module is specifically used for: Each element in the multi-view feature deviation vector is mapped to the observation degree of the corresponding node in the directed graph of device anomaly propagation, and a node observation vector is constructed based on each observation degree. A set of normal operation data is detected in the normal operation history database of the target device, and a normal observation vector is constructed based on the set of normal operation data. The Euclidean distance between the node observation vector and the normal observation vector is calculated, and it is determined whether the Euclidean distance meets the preset requirements. If the Euclidean distance meets the preset requirements, the target device is determined to be in a normal state.

[0066] The fourth embodiment of the present invention provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the substation equipment anomaly detection method based on machine vision as described above.

[0067] The fifth embodiment of the present invention provides a readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the substation equipment anomaly detection method based on machine vision as described above.

[0068] In summary, the machine vision-based substation equipment anomaly detection method and system provided by the above embodiments of the present invention can effectively cope with the complex and ever-changing interference factors in substations, and significantly improve the accuracy of equipment anomaly detection and the overall operating efficiency of the system.

[0069] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0070] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0071] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0072] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0073] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0074] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for detecting anomalies in substation equipment based on machine vision, characterized in that, The method includes: Based on the installation location, field of view, and calibration parameters of the substation monitoring cameras, a spatial mapping relationship of the overlapping areas of different camera fields of view is constructed, and based on the spatial mapping relationship, the image sequence of each device within a complete operating cycle is detected. Based on the image sequence, the periodic motion features and appearance change features of each device are detected to generate a device periodic operation reference template. The image sequences are processed in parallel to detect the device motion region and device static region in each frame of the image. The device motion region is time-aligned with the corresponding device periodic operation reference template to calculate the time deviation between the motion features and the reference template. When the timing deviation of any surveillance camera exceeds the preset deviation threshold, the corresponding target device is detected and a multi-camera collaborative observation command is triggered to schedule all cameras whose field of view covers the target device to simultaneously detect the corresponding device image, and calculate the feature deviation value of the same device in each device image to generate a multi-view feature deviation vector. Based on the physical connection relationship and energy transfer path of the substation equipment, a corresponding directed graph of equipment anomaly propagation is constructed. The multi-view feature deviation vector is then matched with the directed graph of equipment anomaly propagation to determine whether the target equipment is normal based on the matching result.

2. The substation equipment anomaly detection method based on machine vision according to claim 1, characterized in that, The step of performing time-series alignment processing between the motion region of the device and the corresponding periodic operation reference template of the device to calculate the time-series deviation between the motion characteristics and the reference template includes: Dense optical computation is performed on the moving area of ​​the equipment to obtain a one-dimensional velocity time-series curve of the core moving component of the equipment. The velocity signal in the one-dimensional velocity time-series curve is converted into a complex analytic signal by Hilbert transform to solve for the continuous instantaneous phase and instantaneous frequency, and generate phase-frequency joint time-series characteristics. The phase-frequency joint timing feature is used as the input signal, the standard phase-frequency curve pre-stored in the periodic operation reference template is used as the reference signal, and the output phase is dynamically adjusted by a proportional-integral controller to keep the input signal and the reference signal in phase synchronization and output the corresponding phase error signal. Based on a preset phase jump detection window, the phase error signal is dynamically calculated and processed to generate the corresponding timing deviation.

3. The substation equipment anomaly detection method based on machine vision according to claim 2, characterized in that, The step of dynamically calculating and processing the phase error signal based on a preset phase jump detection window to generate the corresponding timing deviation includes: The phase error signal is decomposed into several modal function components and several residual trend components by means of an adaptive variational mode decomposition algorithm. Based on the multiple relationship between the center frequency of each component and the rated operating frequency of the equipment and the energy ratio, several effective modal components are selected accordingly. Hilbert transform is performed on each of the effective modal components to detect the corresponding instantaneous amplitude spectrum and instantaneous phase spectrum. Based on the instantaneous amplitude spectrum and instantaneous phase spectrum, the amplitude fluctuation coefficient, phase variance entropy and phase synchronization index between modes of each effective modal component within the detection window are calculated to construct a three-dimensional modal feature vector. The Mahalanobis distance between the three-dimensional modal feature vector and the baseline modal feature distribution under normal operating conditions is calculated to generate the corresponding time series deviation.

4. The substation equipment anomaly detection method based on machine vision according to claim 1, characterized in that, The step of scheduling all cameras covering the target device to simultaneously detect the corresponding device images, and calculating the feature deviation value of the same device in each of the device images to generate a multi-view feature deviation vector includes: Based on the intrinsic and extrinsic parameter matrices of each camera and the pre-stored 3D point cloud model of the target device, a local view projection matrix for each camera is generated, and each device image is mapped to a unified device body coordinate system through the local view projection matrix to generate a view-normalized image. The viewpoint-normalized image is segmented into several non-overlapping sub-regions, and the gradient histogram features and local binary pattern features of each sub-region are detected accordingly, and then concatenated to obtain the sub-region feature vector. The cosine distance between each sub-region feature vector and the corresponding sub-region reference feature vector in the periodic operation reference template is calculated to generate the original deviation value of the sub-region. The original deviation values ​​of each sub-region are then weighted and fused to generate the corresponding multi-view feature deviation vector.

5. The substation equipment anomaly detection method based on machine vision according to claim 4, characterized in that, The step of weighted fusion of the original deviation values ​​of each sub-region to generate the corresponding multi-view feature deviation vector includes: The projection error covariance matrix of each camera in the device body coordinate system is calculated by the error propagation law, and the projection error covariance of the geometric center point of each sub-region on the image plane of each camera is calculated based on the projection error covariance matrix. The projection error covariance is used as the observation weight coefficient for each sub-region. For each sub-region, the original deviation value of the sub-region corresponding to all cameras and its corresponding observation weight coefficient are multiplied and summed to calculate the initial fusion deviation value of the sub-region. The initial fusion deviation values ​​of each sub-region are integrated to generate the corresponding multi-view feature deviation vector.

6. The substation equipment anomaly detection method based on machine vision according to claim 1, characterized in that, The step of constructing a corresponding directed graph of equipment anomaly propagation based on the physical connection relationships and energy transfer paths of substation equipment includes: Electrical connection edges, mechanical connection edges, and thermal connection edges are constructed based on the electrical wiring relationships, mechanical connection relationships, and heat conduction paths between the substation equipment, wherein the direction of each edge is consistent with the direction of corresponding energy transfer; For the electrical connection edge, the electrical propagation coefficient is calculated based on the rated voltage, rated current and line impedance of the devices at both ends of the connection. For the mechanical connection edge, the mechanical propagation coefficient is calculated based on the material stiffness, damping coefficient and contact area of ​​the connecting components. For the thermal connection edge, the thermal propagation coefficient is calculated based on the thermal conductivity, cross-sectional area and length of the connecting medium. Each propagation coefficient is used as a weight attribute of the corresponding edge. Based on the electrical connection edge, the mechanical connection edge, and the thermal connection edge, respectively construct electrical anomaly propagation subgraphs, mechanical anomaly propagation subgraphs, and thermal anomaly propagation subgraphs, and merge the subgraphs to construct the corresponding directed graph of equipment anomaly propagation.

7. The substation equipment anomaly detection method based on machine vision according to claim 1, characterized in that, The step of matching the multi-view feature deviation vector with the directed graph of device anomaly propagation to determine whether the target device is normal based on the matching result includes: Each element in the multi-view feature deviation vector is mapped to the observation degree of the corresponding node in the directed graph of device anomaly propagation, and a node observation vector is constructed based on each observation degree. A set of normal operation data is detected in the normal operation history database of the target device, and a normal observation vector is constructed based on the set of normal operation data. The Euclidean distance between the node observation vector and the normal observation vector is calculated, and it is determined whether the Euclidean distance meets the preset requirements. If the Euclidean distance meets the preset requirements, the target device is determined to be in a normal state.

8. A substation equipment anomaly detection system based on machine vision, characterized in that, The system includes: The module is used to construct a spatial mapping relationship of the overlapping areas of different camera fields of view based on the installation position, field of view angle and calibration parameters of the substation monitoring camera, and to detect the image sequence of each device in a complete operating cycle based on the spatial mapping relationship. The processing module is used to detect the periodic motion features and appearance change features of each device according to the image sequence, so as to generate a device periodic operation reference template, and to perform parallel processing on each image sequence to detect the device motion area and device static area in each frame image, and to perform time-series alignment processing between the device motion area and the corresponding device periodic operation reference template to calculate the time-series deviation between the motion features and the reference template. The calculation module is used to detect the corresponding target device when the timing deviation of any monitoring camera exceeds a preset deviation threshold, and trigger a multi-camera collaborative observation command to schedule all cameras covering the target device to simultaneously detect the corresponding device image, and calculate the feature deviation value of the same device in each device image to generate a multi-view feature deviation vector. The judgment module is used to construct a corresponding directed graph of equipment anomaly propagation based on the physical connection relationship and energy transfer path of the substation equipment, and to match the multi-view feature deviation vector with the directed graph of equipment anomaly propagation to determine whether the target equipment is normal based on the matching result.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the substation equipment anomaly detection method based on machine vision as described in any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the machine vision-based substation equipment anomaly detection method as described in any one of claims 1 to 7.

Images