A power equipment fault diagnosis method and system based on dynamic visual perception

Abstract

The application provides a power equipment fault diagnosis method and system based on dynamic visual perception, and belongs to the technical field of online monitoring of power equipment. By fusing the space-time gradient information of asynchronous event flow, the pixel-level motion vector in a high-speed scene can be accurately estimated. Based on this, the low-frame-rate image is motion compensated and reconstructed, effectively overcoming the motion blur problem caused by long exposure time of a traditional camera in a high-speed scene, and finally generating a high-frame-rate, high-definition image sequence, which provides a high-quality visual data basis for subsequent accurate analysis. The method does not rely on image information alone, but creatively extracts multi-dimensional space-time features from the reconstructed high-definition image and the continuous confidence of the event flow. This multi-source, multi-modal feature fusion can more comprehensively represent the equipment operating state, especially capture transient abnormalities and weak signs that are difficult to record by traditional images, thereby significantly improving the richness and robustness of feature expression.

Description

Technical Field

[0001] This invention belongs to the field of online monitoring technology for power equipment, specifically relating to a method and system for diagnosing power equipment faults based on dynamic visual perception. Background Technology

[0002] With the large-scale grid connection of renewable energy sources such as wind power and photovoltaics, the operational complexity and uncertainty of power systems have increased significantly, placing unprecedented demands on the monitoring of the operational status and fault early warning of key equipment such as generators and transformers. Traditional online monitoring solutions widely employ visible light or infrared imaging technology, the core of which is a frame-synchronous sampling and output imaging mode. However, when capturing transient, high-speed, and large dynamic range equipment status changes such as arc discharge, high-frequency vibration of components, and local overheating and cooling, this mode has the following inherent limitations: The contradiction between temporal resolution and transient capture: The fixed frame rate of traditional video (usually ≤100 fps) leads to a "temporal blind spot" between adjacent frames. Many critical fault signs in power equipment (such as microsecond-level local arcs and specific frequency vibrations caused by mechanical loosening) change faster than the sampling interval, making them easily missed between frames and resulting in missed detections.

[0003] Imbalance between data efficiency and processing burden: To compensate for insufficient temporal resolution, the video frame rate needs to be significantly increased. This directly leads to an exponential increase in data volume, putting enormous pressure on the system's transmission bandwidth, storage space, and the real-time performance of backend processing algorithms. This contradiction is particularly prominent when deployed in multiple locations.

[0004] Imaging dynamic range is difficult to match the scene: There are often extreme light contrasts at the power equipment monitoring site, such as the coexistence of a strong electric arc and the dark equipment itself. Traditional cameras have limited dynamic range (usually <60 dB), and are prone to local overexposure or underexposure in scenes with both strong and low light, thus losing critical detail information.

[0005] The multimodal information fusion layer is shallow: Although existing monitoring systems may integrate multiple sensors (such as visible light, infrared, and sound), their fusion is mostly limited to data layer splicing or decision layer voting. They lack a mechanism to effectively correlate and complement heterogeneous data such as asynchronous event streams and synchronous images at the underlying physical model level, thus failing to fully leverage the synergistic advantages of multi-source information.

[0006] To address these challenges, event cameras have emerged as a novel biomimetic vision sensor. Each pixel operates independently and asynchronously, outputting an event containing pixel location, precise timestamp, and polarity of the change only when a logarithmic change in brightness is detected exceeding a threshold. This mechanism endows it with inherent advantages such as microsecond-level temporal resolution, high dynamic range (>120 dB), and low data bandwidth, theoretically making it ideal for capturing the fast, high-contrast dynamic processes of electrical equipment.

[0007] However, directly introducing event cameras into the field of power equipment monitoring still faces a series of technical bottlenecks from data to application: Inherent limitations of event data: The event stream contains a large number of false events caused by sensor thermal noise, background lighting flicker, etc., resulting in a low signal-to-noise ratio. Furthermore, events only represent edge information of brightness changes, lacking absolute grayscale or temperature information of the scene, making the data itself extremely sparse and incomplete.

[0008] The fundamental obstacle to heterogeneous modal fusion lies in the essential differences between event data and traditional image / video data in terms of data format and spatiotemporal representation. Mainstream computer vision algorithms and neural network models are designed for synchronous frame data and cannot directly process asynchronous event streams, necessitating new data representation and fusion architectures.

[0009] Domain-Specific Diagnostic Knowledge Gap: Current research largely focuses on the basic algorithms of event cameras themselves (such as reconstruction and recognition), lacking feature extraction and diagnostic models for specific fault modes of power equipment (such as mechanical wear, insulation degradation, and poor contact). How to define and extract discriminative features strongly correlated with the physical failure mechanism of equipment from the fused dynamic visual information, and to construct a robust and interpretable fault diagnosis model, remains a research gap that has not yet been systematically explored.

[0010] In summary, existing technologies suffer from time blind spots and limited dynamic range in perception capabilities, and face challenges in data processing such as bandwidth constraints and superficial data fusion. Emerging event perception technologies are further hampered by data deficiencies, modal gaps, and a lack of domain knowledge. Therefore, developing a method for dynamic visual perception and intelligent diagnosis of power equipment that can deeply integrate multimodal data such as asynchronous events and synchronous images, and achieve high precision, low latency, and strong robustness, has become an urgent and crucial research direction in this field. Summary of the Invention

[0011] The purpose of this invention is to overcome the problems of noise interference, difficulty in heterogeneous data fusion, and inaccurate fault feature extraction in the prior art, and to provide a power equipment fault diagnosis method and system based on dynamic visual perception.

[0012] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for fault diagnosis of power equipment based on dynamic visual perception, comprising the following steps: Acquire low frame rate images and asynchronous event streams of power equipment, and preprocess the asynchronous event streams to obtain the continuous confidence of the asynchronous event streams on a spatiotemporal grid; The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information. The gradient information is then fused with the low frame rate image to estimate the pixel motion vector field in the high-speed dynamic scene. Based on the pixel motion vector field, motion compensation is performed on low frame rate images, and the continuous confidence is fused to reconstruct high frame rate and high-definition images without motion blur. Multi-dimensional spatiotemporal features are extracted from the high-definition image and the continuous confidence on the spatiotemporal grid. The extracted multi-dimensional spatiotemporal features are calculated through a probabilistic model to obtain a probability map that characterizes the probability of fault occurrence at each pixel location. The probability map is mapped to a heat map to obtain the fault diagnosis results.

[0013] The method for acquiring low frame rate images and asynchronous event streams of power equipment, and preprocessing the asynchronous event streams to obtain the continuous confidence scores of the asynchronous event streams on a spatiotemporal grid is as follows: Acquire low frame rate images and asynchronous event streams of power equipment, and align the low frame rate images and asynchronous event streams to a unified time base based on their respective timestamps; Event filtering is performed on the asynchronous event stream by introducing a spatiotemporal double-weighted Gaussian kernel and an edge-enhanced Sigmoid function, transforming the asynchronous event stream into continuous confidence scores defined on a spatiotemporal grid.

[0014] The event filtering of the asynchronous event stream, by introducing a spatiotemporal double-weighted Gaussian kernel and an edge-enhancing Sigmoid function, transforms the asynchronous event stream into a continuous confidence score defined on a spatiotemporal grid. The formula is as follows:

[0015] in, The confidence level of the filtered event; These represent the pixel plane x-coordinate, pixel plane y-coordinate, and event timestamp, respectively. Defines the spatial location of the event on the sensor array. It defines the precise moment when the event occurred; This is a primitive asynchronous event; by The spatiotemporal cube neighborhood centered on the center; For spatial Gaussian weight kernels; It is a time-exponentially decaying weight kernel; It is a Laplace edge response for event streams, used to enhance the spatiotemporal continuity of real events.

[0016] The method for calculating the spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid using the central difference method to obtain gradient information, and then fusing the gradient information with the low frame rate image to estimate the pixel motion vector field in a high-speed dynamic scene is as follows: The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information, which is then stored in a cache. Read the low frame rate images of the current time and the previous time, and calculate the spatial image gradient components and temporal gradient components of the low frame rate images. Read the gradient vector field corresponding to the current image block from the cache, and calculate the first gradient variance of the low frame rate image within the image block and the second gradient variance of the continuous confidence on the corresponding spatiotemporal grid. The spatiotemporal regularization weights for cross-modal fusion are calculated based on the first gradient variance and the second gradient variance. Based on the aforementioned spatiotemporal regularization weights, as well as the spatial and temporal gradient components of the low frame rate image, the energy function is constructed using the iterative reweighted least squares method to obtain the final motion vector field.

[0017] In the step of calculating the spatiotemporal regularization weights for cross-modal fusion based on the first gradient variance and the second gradient variance, the formula for calculating the spatiotemporal regularization weights for cross-modal fusion is as follows:

[0018] in, The acquired image data of the power equipment is low frame rate image; Image scale The image gradient below; For spatiotemporal regularization weights; Let V be the gradient variance at each scale.

[0019] The method for reconstructing a high-frame-rate, high-resolution image without motion blur by performing motion compensation on a low frame-rate image based on the pixel motion vector field and fusing the continuous confidence score is as follows: Based on low frame rate images and pixel motion vector fields, construct respectively with Image pyramids and sports field pyramids at various scales; At each scale Based on the motion field at this scale, motion compensation is performed on the low frame rate image at the corresponding scale to obtain the motion-compensated image; The motion-compensated images are weighted and fused according to their corresponding scale fusion weights, and then reconstructed scale by scale using upsampling and cross-scale residual connection methods to obtain the final high-resolution, high-frame-rate high-definition images.

[0020] At each scale Based on the motion field at this scale, motion compensation is performed on the low frame rate image at the corresponding scale to obtain the motion-compensated image. Based on the preset event-motion joint modulation function, the scale is calculated. Spatial modulation map on;

[0021] in, For event-motion joint modulation function, It is the mean of the local motion vector. Standard deviation is a measure of consistency. To perform Gaussian smoothing on the amplitude of motion velocity.

[0022] The specific formula for the step of weighted fusion of the motion-compensated image according to its corresponding scale fusion weight, and reconstructing the final high-resolution, high-frame-rate high-definition image scale by scale through upsampling and cross-scale residual connection is as follows:

[0023] in, For reconstructed high-resolution images; In order to scale Motion compensation operator on; For the current reconstruction moment With the most recent image frame moment The time difference.

[0024] In the step of extracting multi-dimensional spatiotemporal features from the high-definition image and the continuous confidence scores on the spatiotemporal grid, and calculating the extracted multi-dimensional spatiotemporal features using a probabilistic model to obtain a probability map representing the probability of fault occurrence at each pixel location, the probabilistic model is as follows:

[0025] in, This is a fault probability diagram. As a standardized feature, b is the feature weight, and b is the bias term. It is a radially based core.

[0026] Secondly, the present invention provides a power equipment fault diagnosis system based on dynamic visual perception, comprising: The data acquisition and preprocessing module is used to acquire low frame rate images and asynchronous event streams of power equipment, and preprocess the asynchronous event streams to obtain the continuous confidence of the asynchronous event streams on the spatiotemporal grid. The motion vector field generation module is used to calculate the spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid using the central difference method, obtain gradient information, and fuse the gradient information with the low frame rate image to estimate the pixel motion vector field in the high-speed dynamic scene. The high-definition image generation module is used to perform motion compensation on low frame rate images based on the pixel motion vector field, and fuse the continuous confidence to reconstruct high frame rate high-definition images without motion blur. The calculation module is used to extract multi-dimensional spatiotemporal features from the high-definition image and the continuous confidence on the spatiotemporal grid, and to calculate the extracted multi-dimensional spatiotemporal features through a probabilistic model to obtain a probability map that characterizes the probability of fault occurrence at each pixel location. The result generation module is used to map the probability map into a heat map to obtain the fault diagnosis result.

[0027] Compared with the prior art, the present invention has the following beneficial effects: By fusing spatiotemporal gradient information from asynchronous event streams, pixel-level motion vectors in high-speed scenes can be accurately estimated. Based on this, motion compensation and reconstruction of low frame rate images are performed, effectively overcoming the motion blur problem caused by long exposure times in high-speed scenes with traditional cameras. This ultimately generates high-frame-rate, high-resolution images, providing a high-quality visual data foundation for subsequent precise analysis. This method does not solely rely on image information but creatively extracts multi-dimensional spatiotemporal features from both the reconstructed high-resolution images and the continuous confidence of the event stream. This multi-source, multi-modal feature fusion can more comprehensively characterize the device's operating status, especially capturing transient anomalies and subtle signs that are difficult to record with traditional images, thus significantly improving the richness and robustness of feature representation. Attached Figure Description

[0028] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0029] To further understand the content of this invention, the invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments are merely illustrative and not limiting of the invention.

[0030] Example 1 A method for fault diagnosis of power equipment based on dynamic visual perception includes the following steps: S1: Acquire low frame rate images and asynchronous event streams of power equipment, and preprocess the asynchronous event streams to obtain the continuous confidence of the asynchronous event streams on the spatiotemporal grid; S2: The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information, and the gradient information is fused with the low frame rate image to estimate the pixel motion vector field in the high-speed dynamic scene. S3: Based on the pixel motion vector field, perform motion compensation on the low frame rate image and fuse the continuous confidence to reconstruct a high frame rate and high-definition image without motion blur; S4: Extract multi-dimensional spatiotemporal features from the high-definition image and the continuous confidence on the spatiotemporal grid, and calculate the extracted multi-dimensional spatiotemporal features through a probabilistic model to obtain a probability map that characterizes the probability of fault occurrence at each pixel location. S5: Map the probability map to a heat map to obtain the fault diagnosis results.

[0031] Specifically, in S1, low frame rate images and asynchronous event streams of power equipment are acquired, and the asynchronous event streams are preprocessed to obtain the continuous confidence of the asynchronous event streams on the spatiotemporal grid.

[0032] Low-frame-rate images of power equipment are acquired using industrial cameras, while asynchronous event streams of the power equipment are synchronously acquired using event cameras. The low frame rate image and the asynchronous event stream are aligned to a unified time base based on their respective timestamps. The asynchronous event stream... The logarithmic change in light intensity exceeds the threshold Produce born.

[0033] The asynchronous event stream is filtered by introducing a spatiotemporal double-weighted Gaussian kernel and an edge-enhancing Sigmoid function, transforming the asynchronous event stream into continuous confidence scores defined on a spatiotemporal grid. The specific formula is as follows:

[0034]

[0035]

[0036]

[0037] in, The event confidence score after filtering is a dimensionless continuous value in the range [0, 1], representing the event confidence score at a given point in time and space. The credibility of a real brightness change event occurring at that location; These represent the pixel plane x-coordinate, pixel plane y-coordinate, and event timestamp, respectively. Defines the spatial location of the event on the sensor array. It defines the precise moment (typically in microseconds) when the event occurs. This is the basic coordinate framework for the event camera's output data; For the original asynchronous event, in the neighborhood The value can be either 0 or 1. : by The spatiotemporal cube neighborhood centered on the element, for example, 5×5 pixels in space and 10 event stamps in time; For spatial Gaussian weight kernel, The pixel size is determined by the typical fault target size and pixel pitch. The time-exponentially decaying weight kernel, Each event timestamp is determined by the typical vibration frequency of the device and the event time resolution. It is a Laplace edge response for event streams, used to enhance the spatiotemporal continuity of real events.

[0038] This formula achieves noise suppression through neighborhood weighted averaging and by utilizing edge response. Enhance the confidence level of realistic dynamic events. Its output... It is a continuous confidence scalar field, which lays the foundation for the next step of calculating the spatial gradient.

[0039] Specifically, in S2, the output of S1 It is a two-dimensional scalar field defined on a discrete spatiotemporal grid (for each time slice t). The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information. The gradient information is then fused with the low frame rate image to obtain the pixel motion vector field in the high-speed dynamic scene.

[0040] S21: The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information;

[0041]

[0042]

[0043]

[0044] in, and The physical spacing between pixels (usually equal). The calculated gradient vector field. This describes the direction and intensity of the spatial variation in event confidence. The resulting gradient vector field... and time derivative Store in cache.

[0045] S22: Read the low frame rate images from the current and previous moments. and The spatial image gradient components of the low frame rate image are calculated using the Sobel operator. The temporal gradient components of the image are calculated using the frame difference method. ; Convolve the current frame image along the x and y directions respectively:

[0046]

[0047]

[0048]

[0049]

[0050] S23: Read the gradient vector field and time derivative corresponding to the current image block from the cache, and calculate the first gradient variance of the low frame rate image within the image block. And the second gradient variance of the continuous confidence level on the corresponding spatiotemporal grid. ; S24: Calculate the spatiotemporal regularization weights for cross-modal fusion based on the first gradient variance and the second gradient variance;

[0051] in, The acquired image data of the power equipment is low frame rate image; Image scale The image gradient below; For spatiotemporal regularization weights and for multi-scale adaptive fusion weights, in regions with rich texture (large gradients), higher-scale (smoothed) image information is trusted more to avoid noise amplification; in flat regions, lower-scale (more detailed) information is trusted more. Let V be the gradient variance at each scale.

[0052] S26: Based on the spatiotemporal regularization weights, and the spatial and temporal gradient components of the low frame rate image, the following energy function minimization problem is solved using the iterative reweighted least squares method to obtain the final motion vector field V.

[0053]

[0054] in, Let be the pixel motion vector field to be determined. For the image domain, This is the spatial gradient regularization term for the sports field.

[0055] Specifically, in S3, motion compensation is performed on the low frame rate image based on the pixel motion vector field, and the continuous confidence is fused to reconstruct a high frame rate and high-definition image without motion blur.

[0056] S31: Based on low frame rate images and pixel motion vector fields, construct respectively with Image pyramids and sports field pyramids at various scales; S32: At each scale Based on the motion field at this scale, motion compensation is performed on the low frame rate image at the corresponding scale to obtain the motion-compensated image; specifically as follows: Based on the preset event-motion joint modulation function, the scale is calculated. Spatial modulation map on;

[0057] in, This is an event-motion joint modulation function. The core idea of ​​this function is: only when the event confidence is high... And the local movements are consistent Image enhancement is only performed at that time. It is the mean of the local motion vector. Pixels / frame is the standard deviation of the consistency measure. To perform Gaussian smoothing on the amplitude of motion velocity It is used to generate a smooth, enhanced mask.

[0058] S33: The motion-compensated image is weighted and fused according to its corresponding scale fusion weight, and the final high-resolution, high-frame-rate high-definition image is reconstructed scale by scale through upsampling and cross-scale residual connection.

[0059]

[0060] in, For reconstructed high-resolution images; In order to scale The motion compensation operator on the surface utilizes motion vectors. After downsampling to the corresponding scale, the result is obtained The image is inversely distorted to counteract motion blur; For the current reconstruction moment With the most recent image frame moment Time difference, This represents the event enhancement factor.

[0061] The above process achieves pixel-level, motion-aware multimodal fusion. Output Not only does it have high temporal resolution, but it also provides clearer details in key areas such as edges of high-speed motion through the modulation of event information.

[0062] Specifically, in S4, multi-dimensional spatiotemporal features are extracted from the high-definition image and the continuous confidence on the spatiotemporal grid. The extracted multi-dimensional spatiotemporal features are calculated through a probabilistic model to obtain a probability map that characterizes the probability of fault occurrence at each pixel location.

[0063] S41: For each pixel location in the high-resolution image data at time t, calculate a multidimensional feature set, wherein the multidimensional feature set includes at least: The first image spatial feature is calculated directly from the current pixel neighborhood of the high-definition image. Second image spatial features ;

[0064]

[0065] First image spatial features Reconstructing the gradient magnitude of the image to capture changes in edge sharpness; second image spatial features. This represents local variance, reflecting anomalies in texture or hot spots.

[0066] By acquiring high-resolution images of a first sequence centered at time t for a preset duration, and performing frequency domain transformation on the first sequence, a third time-frequency feature is obtained by extracting energy in a specified high-frequency band. ;

[0067] The fourth event cumulative feature is obtained by summing all values ​​of the continuous confidence at the pixel location within a time window of a preset time length ΔT = 50ms backward from time t. ;

[0068] S42: Using the pre-stored continuous confidence scores corresponding to each feature, standardize the first to fourth features respectively to obtain the corresponding standardized features. ;

[0069] For normalized features, Features The mean obtained by statistical analysis on a large amount of historical normal data. Features The standard deviation is obtained by statistical analysis on a large amount of historical normal data.

[0070] S43: Standardization features The pre-trained feature weight coefficients and bias b are input into a preset probability calculation model. The probability calculation model outputs the real-time fault probability of the pixel position at time t. The probabilities of all positions constitute the fault probability map.

[0071] The preset probability calculation model is as follows:

[0072] in, This is a fault probability diagram. , where b is the feature weight; b = -0.5, which is the model bias term; , which is a radial base core.

[0073] Preferably, The method for determining the fault is as follows: collect long-term series data covering various states such as normal, arc discharge, local overheating, and mechanical loosening at multiple power plant sites, and hire domain experts to perform pixel-level or regional-level fault labeling. Calculated from all normal state data slices, used to establish a normal baseline. Features from labeled normal and faulty samples are used. The weights are obtained by training through logistic regression or support vector machine (SVM). and bias The training process aims to maximize the detection rate of faulty samples while keeping the false alarm rate within an acceptable range.

[0074] This formula constructs a lightweight probabilistic classifier based on multi-source feature fusion. Its innovations are: 1) the feature design closely integrates with the physical manifestations of power equipment faults (visual features corresponding to light, heat, vibration, and magnetism); and 2) the introduction of a normalized radial basis kernel. 1) Mapping features to a high-dimensional similarity space enhances the model’s non-linear classification capability; 2) The final output is a continuous probability map, which supports setting different thresholds to achieve multi-level fault responses such as early warning and alarm, making it more flexible than binary decision.

[0075] Specifically, in S5, the probability map is mapped to a heat map and overlaid on the original low frame rate image or the reconstructed high-definition image to obtain the fault diagnosis result. The alarm information (such as the coordinates of the area where the probability exceeds the threshold of 0.8) is pushed to the power plant monitoring system through the network interface.

[0076] Example 2 The system was deployed in the slip ring chamber of a generator at a 660MW coal-fired power plant for a three-month test.

[0077] The method of this invention was compared with methods using only traditional video analysis and commercial infrared thermal imager monitoring systems. The results show that: Arc detection: This invention successfully provided early warning for 5 weak arcs that are difficult to detect with the naked eye, with an average early warning time of 42ms; traditional video methods missed detection 3 times, and commercial infrared systems missed detection 2 times.

[0078] Localized overheating: This invention detects a slowly developing overheating area (temperature rise <5K) at the transformer bushing and issues an early warning one week in advance; other systems do not issue effective alarms.

[0079] Processing latency: The average end-to-end latency from sensor data input to probability map output is 35ms, which meets the real-time requirements.

[0080] False alarm rate: The average number of false alarms per day is 0.7, which is lower than the 3 false alarms per day required by the operations and maintenance department.

[0081] Example 3 A power equipment fault diagnosis system based on dynamic visual perception includes: The data acquisition and preprocessing module is used to acquire low frame rate images and asynchronous event streams of power equipment, and preprocess the asynchronous event streams to obtain the continuous confidence of the asynchronous event streams on the spatiotemporal grid. The motion vector field generation module is used to calculate the spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid using the central difference method, obtain gradient information, and fuse the gradient information with the low frame rate image to estimate the pixel motion vector field in the high-speed dynamic scene. The high-definition image generation module is used to perform motion compensation on low frame rate images based on the pixel motion vector field, and fuse the continuous confidence to reconstruct high frame rate high-definition images without motion blur. The calculation module is used to extract multi-dimensional spatiotemporal features from the high-definition image and the continuous confidence on the spatiotemporal grid, and to calculate the extracted multi-dimensional spatiotemporal features through a probabilistic model to obtain a probability map that characterizes the probability of fault occurrence at each pixel location. The result generation module is used to map the probability map into a heat map to obtain the fault diagnosis result.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for fault diagnosis of power equipment based on dynamic visual perception, characterized in that, The steps include the following: Acquire low frame rate images and asynchronous event streams of power equipment, and preprocess the asynchronous event streams to obtain the continuous confidence of the asynchronous event streams on a spatiotemporal grid; The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information. The gradient information is then fused with the low frame rate image to estimate the pixel motion vector field in the high-speed dynamic scene. Based on the pixel motion vector field, motion compensation is performed on low frame rate images, and the continuous confidence is fused to reconstruct high frame rate and high-definition images without motion blur. Multi-dimensional spatiotemporal features are extracted from the high-definition image and the continuous confidence on the spatiotemporal grid. The extracted multi-dimensional spatiotemporal features are calculated through a probabilistic model to obtain a probability map that represents the probability of fault occurrence at each pixel location. The probability map is mapped to a heat map to obtain the fault diagnosis results.

2. The power equipment fault diagnosis method based on dynamic visual perception according to claim 1, characterized in that, The method for acquiring low frame rate images and asynchronous event streams of power equipment, and preprocessing the asynchronous event streams to obtain the continuous confidence scores of the asynchronous event streams on a spatiotemporal grid is as follows: Acquire low frame rate images and asynchronous event streams of power equipment, and align the low frame rate images and asynchronous event streams to a unified time base based on their respective timestamps; Event filtering is performed on the asynchronous event stream by introducing a spatiotemporal double-weighted Gaussian kernel and an edge-enhanced Sigmoid function, transforming the asynchronous event stream into continuous confidence scores defined on a spatiotemporal grid.

3. The power equipment fault diagnosis method based on dynamic visual perception according to claim 2, characterized in that, The event filtering of the asynchronous event stream, by introducing a spatiotemporal double-weighted Gaussian kernel and an edge-enhancing Sigmoid function, transforms the asynchronous event stream into a continuous confidence score defined on a spatiotemporal grid. The formula is as follows: in, The confidence level of the filtered event; These represent the pixel plane x-coordinate, pixel plane y-coordinate, and event timestamp, respectively. Defines the spatial location of the event on the sensor array. It defines the precise moment when the event occurred; This is a primitive asynchronous event; by The spatiotemporal cube neighborhood centered on the center; For spatial Gaussian weight kernels; It is a time-exponentially decaying weight kernel; The Laplace edge response of the event stream is used to enhance the spatiotemporal continuity of real events.

4. The power equipment fault diagnosis method based on dynamic visual perception according to claim 3, characterized in that, The method for calculating the spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid using the central difference method to obtain gradient information, and then fusing the gradient information with the low frame rate image to estimate the pixel motion vector field in a high-speed dynamic scene is as follows: The spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid is calculated using the central difference method to obtain gradient information, which is then stored in a cache. Read the low frame rate images of the current time and the previous time, and calculate the spatial image gradient components and temporal gradient components of the low frame rate images. Read the gradient vector field corresponding to the current image block from the cache, and calculate the first gradient variance of the low frame rate image within the image block and the second gradient variance of the continuous confidence on the corresponding spatiotemporal grid. The spatiotemporal regularization weights for cross-modal fusion are calculated based on the first gradient variance and the second gradient variance. Based on the spatiotemporal regularization weights, as well as the spatial and temporal gradient components of the low frame rate image, the energy function is constructed by iterative reweighted least squares method to obtain the final motion vector field.

5. The power equipment fault diagnosis method based on dynamic visual perception according to claim 4, characterized in that, In the step of calculating the spatiotemporal regularization weights for cross-modal fusion based on the first gradient variance and the second gradient variance, the calculation formula for the spatiotemporal regularization weights for cross-modal fusion is as follows: in, The acquired image data of the power equipment is low frame rate image; Image scale The image gradient below; For spatiotemporal regularization weights; Let V be the gradient variance at each scale.

6. The power equipment fault diagnosis method based on dynamic visual perception according to claim 5, characterized in that, The method for reconstructing a high-frame-rate, high-resolution image without motion blur by performing motion compensation on a low frame-rate image based on the pixel motion vector field and fusing the continuous confidence score is as follows: Based on low frame rate images and pixel motion vector fields, construct respectively with Image pyramids and sports field pyramids at various scales; At each scale Based on the motion field at this scale, motion compensation is performed on the low frame rate image at the corresponding scale to obtain the motion-compensated image; The motion-compensated images are weighted and fused according to their corresponding scale fusion weights, and then reconstructed scale by scale using upsampling and cross-scale residual connection methods to obtain the final high-resolution, high-frame-rate high-definition images.

7. The power equipment fault diagnosis method based on dynamic visual perception according to claim 6, characterized in that, At each scale Based on the motion field at this scale, motion compensation is performed on the low frame rate image at the corresponding scale to obtain the motion-compensated image. Based on the preset event-motion joint modulation function, the scale is calculated. Spatial modulation map on; in, For event-motion joint modulation function, It is the mean of the local motion vector. Standard deviation is a measure of consistency. To perform Gaussian smoothing on the amplitude of motion velocity.

8. The power equipment fault diagnosis method based on dynamic visual perception according to claim 7, characterized in that, The specific formula for the step of weighted fusion of the motion-compensated image according to its corresponding scale fusion weight, and reconstructing the final high-resolution, high-frame-rate high-definition image scale by scale through upsampling and cross-scale residual connection is as follows: in, For reconstructed high-resolution images; In order to scale Motion compensation operator on; For the current reconstruction moment With the most recent image frame moment The time difference.

9. A method for fault diagnosis of power equipment based on dynamic visual perception according to claim 8, characterized in that, In the step of extracting multi-dimensional spatiotemporal features from the high-definition image and the continuous confidence scores on the spatiotemporal raster, and calculating the extracted multi-dimensional spatiotemporal features using a probabilistic model to obtain a probability map representing the probability of fault occurrence at each pixel location, the probabilistic model is as follows: in, This is a fault probability diagram. As a standardized feature, b is the feature weight, and b is the bias term. It is a radially based core.

10. A power equipment fault diagnosis system based on dynamic visual perception, characterized in that, include: The data acquisition and preprocessing module is used to acquire low frame rate images and asynchronous event streams of power equipment, and preprocess the asynchronous event streams to obtain the continuous confidence of the asynchronous event streams on the spatiotemporal grid. The motion vector field generation module is used to calculate the spatial gradient of the continuous confidence of the asynchronous event stream on the spatiotemporal grid using the central difference method, obtain gradient information, and fuse the gradient information with the low frame rate image to estimate the pixel motion vector field in the high-speed dynamic scene. The high-definition image generation module is used to perform motion compensation on low frame rate images based on the pixel motion vector field, and fuse the continuous confidence to reconstruct high frame rate high-definition images without motion blur. The calculation module is used to extract multi-dimensional spatiotemporal features from the high-definition image and the continuous confidence on the spatiotemporal grid, and to calculate the extracted multi-dimensional spatiotemporal features through a probabilistic model to obtain a probability map that characterizes the probability of fault occurrence at each pixel location. The result generation module is used to map the probability map into a heat map to obtain the fault diagnosis result.

Images