Cross-modal perception collaborative driving method for power system full-scene intelligent inspection

Abstract

The application relates to the technical field of power switch inspection, in particular to a full-scene intelligent inspection method for a power system driven by cross-modal perception cooperation, which comprises the following steps: collecting video images and audio signals of three-phase contacts of a high-voltage isolating switch in a power system; performing space-time alignment on the video images and the audio signals; determining a sound intensity envelope vector, a frequency centroid vector and a vibration amplitude vector; constructing a time lag matrix representing the relative relationship between acoustic excitation and visual response at different time points, adding weights to the time lag matrix by using the frequency centroid vector to obtain a final transmission cost matrix, using the transmission cost matrix to establish a mathematical model of a discrete optimal transmission problem, solving the mathematical model to obtain an energy attenuation delay index of each phase; determining the data validity by using the energy attenuation delay index, further calculating a relative dispersion ratio of each phase, and detecting the contact stiffness of each phase contact. The application improves the accuracy of power system inspection.

Description

Technical Field

[0001] This application relates to the field of power switch inspection technology, specifically to a cross-modal perception-driven intelligent inspection method for power systems across all scenarios. Background Technology

[0002] High-voltage disconnect switches are critical arc-free switching devices in power systems, and the contact performance of their contacts directly affects the safe operation of the power grid. During long-term service, due to wear of the mechanical transmission mechanism, spring fatigue, or metal corrosion, the contact system may experience a hidden danger of "closing in place but insufficient contact pressure." This deterioration of the microscopic contact condition leads to increased contact resistance, potentially causing overheating or even ablation under heavy loads.

[0003] From a mechanical dynamics perspective, the moment a disconnecting switch closes is accompanied by a violent impact of the metal contacts. The stiffness and damping characteristics of the contact system directly determine the dissipation process of the impact energy. Systems with tight contact (high stiffness) have a higher impact sound frequency and rapid mechanical vibration decay; while systems with loose contact (low stiffness) are often accompanied by low-frequency oscillations and long-term energy residue.

[0004] However, existing surveillance cameras in substations typically have low frame rates (e.g., 25fps), making it impossible to directly capture high-frequency mechanical vibration waveforms. While acoustic monitoring alone offers a high sampling rate, it lacks spatial positioning capabilities and is susceptible to environmental noise interference. Currently, there is a lack of an online monitoring method that can effectively quantify and assess the contact status of contacts by fusing existing low-frame-rate video with acoustic signals. Summary of the Invention

[0005] To address the aforementioned technical issues, this application provides a cross-modal perception-driven intelligent inspection method for power systems across all scenarios, thereby resolving existing problems.

[0006] The cross-modal sensing collaborative-driven intelligent inspection method for power systems in this application adopts the following technical solution: One embodiment of this application provides a method for intelligent inspection of power systems across all scenarios using cross-modal sensing and collaborative driving. The method includes the following steps: Acquire video images and audio signals from the three-phase contacts of high-voltage disconnectors in the power system; and perform spatiotemporal alignment of the video images and audio signals; Identify the trigger point when the three-phase contacts of the switch physically collide, and extract the short-time video image and short-time audio signal corresponding to that time point after alignment; Based on the time-domain amplitude of the short-time audio signal, the sound intensity envelope vector is determined; the frequency centroid vector of the short-time audio signal in the frequency domain is extracted; and the vibration amplitude vector representing the relative vibration response intensity of the contact area is extracted based on the displacement change characteristics of pixels in the short-time video image. A time lag matrix representing the relative relationship between acoustic excitation and visual response at different time points is constructed. The frequency centroid vector is used to add weights to the time lag matrix to obtain the final transmission cost matrix, which is used to establish a mathematical model for the discrete optimal transmission problem. The sound intensity envelope vector and vibration amplitude vector are used as constraints to solve the mathematical model and obtain the energy attenuation delay index for each phase. The validity of the data is determined by the energy decay delay index, and the relative dispersion ratio of each phase is further calculated to detect the contact stiffness of each phase contact.

[0007] In one embodiment, the spatiotemporal alignment of the video image and audio signal includes: Based on the distance between the video acquisition device and the audio acquisition device and the disconnect switch, the time delay calibration value is calculated to compensate for the time required for the sound wave to travel from the disconnect switch to the sensor. The video image and audio signal are spatiotemporally aligned based on the time delay calibration value.

[0008] In one embodiment, the process of identifying the trigger time point is as follows: When the short-term energy of the audio signal is detected to exceed the trigger threshold for the first time, that moment is determined to be the trigger point for the physical impact of the corresponding contact.

[0009] In one embodiment, the process of determining the sound intensity envelope vector is as follows: A frequency domain transformation is performed on the short-time audio signal to obtain the frequency domain amplitude envelope. The frequency domain amplitude envelope is sampled, processed, and normalized to form the sound intensity envelope vector.

[0010] In one embodiment, the extraction process of the frequency centroid vector is as follows: In short-time audio signals, a sliding window is set for the audio signal at each time point, the power spectral density of the audio signal spectrum within each sliding window is obtained, and its centroid frequency is calculated. After normalizing the centroid frequencies, a frequency centroid vector is formed.

[0011] In one embodiment, the process of extracting the vibration amplitude vector is as follows: For the locked contact monitoring area in short-time video images, the local phase change of the image sequence in the vertical direction is calculated to determine the displacement signal. The amplitude envelope of the displacement signal is normalized to obtain the vibration amplitude vector.

[0012] In one embodiment, row index i of the time lag matrix corresponds to an acoustic moment. Column index j corresponds to visual time. For elements in the time lag matrix ,like ,Will If it is set to positive infinity, , ,in, This is a preset energy dissipation reference constant.

[0013] In one embodiment, the process of obtaining the transmission cost matrix is ​​as follows: Initialize a weight matrix with the same dimensions as the time lag matrix. At that time, calculate and The time difference between them is used to perform a logarithmic operation on the time difference, and the time lag matrix is ​​obtained. The weights of the elements and the result of the logarithmic operation, and the frequency centroid vector. The corresponding elements at each time point are all positively correlated; The product of the time lag matrix and the weight matrix is ​​used as the transmission cost matrix.

[0014] In one embodiment, solving the mathematical model to obtain the energy decay delay index for each phase includes: Predefine a non-negative energy mapping matrix, where the elements are... The mathematical model states that the global total transmission cost, as the dependent variable, is equal to the sum of the products of all elements in the same position in the energy mapping matrix and the transmission cost matrix. The constraint is: at the same acoustic moment in the energy mapping matrix The sum of elements at all visual moments and the sound intensity envelope vector The corresponding elements at each time point are equal, and the same visual time point in the energy mapping matrix... The sum of the elements at all acoustic moments and the vibration amplitude vector The corresponding elements at any given time are equal; The energy decay delay index is obtained by minimizing the mathematical model using an optimization algorithm.

[0015] In one embodiment, calculating the relative dispersion ratio of each phase and detecting the contact stiffness of each phase contact includes: Obtain the median of the energy attenuation delay index corresponding to the three-phase contacts, calculate the sum of the median and a preset value greater than 0, and the relative dispersion ratio of each phase is the ratio of the energy attenuation delay index corresponding to each phase contact to the sum; if the relative dispersion ratio of any phase contact is greater than the preset alarm threshold, it is determined that the phase contact has a potential problem of insufficient contact stiffness, otherwise, the contact stiffness is determined to be normal.

[0016] This application has at least the following beneficial effects: This application successfully compensates for the low frame rate of visual signals by utilizing the high-frequency characteristics of acoustic signals, significantly improving the monitoring capability of the contact status of high-voltage disconnecting switch contacts. This application simplifies "microscopic stiffness detection" to "acousto-optic correlation analysis of macroscopic loosening envelope duration," enabling effective sensing of insufficient contact pressure without the need for additional investment in expensive high-speed industrial cameras, relying solely on existing security monitoring equipment. This breaks through traditional hardware limitations, reduces inspection costs, and improves the system's economic efficiency by fully utilizing existing resources. Furthermore, this application introduces "causality (response does not precede excitation)" and "stiffness damping characteristics (high stiffness causes rapid attenuation)" as physical constraints, which not only enhances the scientific rigor and rationality of data processing but also effectively eliminates the influence of environmental noise on the results. Causality ensures that the monitoring signal response is a true reflection of the excitation, enabling automatic filtering of non-causal background noise and image jitter unrelated to the equipment in complex environments, significantly reducing the false alarm rate, enhancing the accuracy of inspection results, and ensuring the reliable operation of the power system. This makes power system inspections more efficient and intelligent, enabling real-time monitoring of potential contact problems, allowing for timely maintenance measures and preventing safety hazards caused by equipment failures. In addition, this application improves the system's robustness, maintaining good monitoring performance under different environmental conditions and interference, providing a solid guarantee for the safe and stable operation of the power system. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates the steps of the cross-modal perception-coordinated intelligent inspection method for power systems across all scenarios provided in this application. Detailed Implementation

[0019] To further illustrate the technical means and effects adopted by this application to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the cross-modal perception collaborative driving intelligent inspection method for power systems across all scenarios proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0021] The following description, in conjunction with the accompanying drawings, details the specific scheme of the cross-modal perception collaborative driving method for intelligent inspection of power systems across all scenarios provided in this application.

[0022] This application provides an embodiment of a cross-modal perception-coordinated intelligent inspection method for power systems across all scenarios. Specifically, it provides the following cross-modal perception-coordinated intelligent inspection method for power systems across all scenarios. Please refer to [link to relevant documentation]. Figure 1 The method includes the following steps: Step S001: Acquire video images and audio signals of the three-phase contacts of the high-voltage disconnector in the power system; and perform spatiotemporal alignment on the video images and audio signals.

[0023] First, define a discrete time series in the backend server. This serves as the mathematical framework for the global analysis in this embodiment. The sequence contains a fixed number of... Each sampling time point. In this embodiment, a sampling time point is set. The time interval between adjacent sampling points Set as The sequence Corresponding to physical time length This covers the entire process of contact impact of the disconnector switch and subsequent mechanical vibration attenuation. The time interval between sampling points and the number of sampling points can be set by the implementer according to the actual situation; this embodiment does not impose any restrictions on this.

[0024] Meanwhile, the contact monitoring area is pre-defined in the surveillance camera footage of the substation. This area should cover the physical contact area between the moving and stationary contacts of the disconnector switch, serving as a spatial constraint for subsequent image analysis. Set the time delay calibration value. , representing the acoustic-optical transmission delay parameter, is used to compensate for the time required for sound waves to propagate from the switching device to the sensor. In this embodiment, the straight-line distance between the sensor (the camera and microphone are typically mounted on the same tower) and the target disconnect switch is known. Speed ​​of sound in air Therefore, the calculated time delay calibration value for: Since the propagation time of light is extremely short and negligible, the time delay calibration value represents the physical lag of the audio signal relative to the video signal.

[0025] In addition, during the data acquisition phase, a 2-second FIFO circular buffer queue is created in memory. This buffer holds the real-time audio stream (sampling rate) acquired by the directional microphone. ) and real-time video streams (frame rate) captured by surveillance cameras The data is continuously written to this queue. The purpose of this queue is to ensure that historical data prior to the triggering time can be retrieved.

[0026] It should be understood that the above steps aim to establish a unified physical spatiotemporal reference to capture the transient process of the high-voltage disconnector closing action from the multimodal sensor stream. Given the significant differences in physical transmission speeds between acoustic and optical signals, and the heterogeneous characteristics of sensor sampling rates, rigorous time delay calibration and data interception are performed to provide a causal data foundation for subsequent analysis.

[0027] Step S002: Identify the triggering time point when the three-phase contacts of the high-voltage disconnector experience a physical impact, and extract the short-time video image and short-time audio signal corresponding to that time point after alignment.

[0028] To automatically pinpoint the start time of the closing action, the audio stream is continuously monitored. First, the audio stream undergoes bandpass filtering (passband range...). to This is done to filter out low-frequency wind noise and transformer hum in the environment. The short-time energy of the filtered signal is calculated, and a trigger threshold is set. The threshold is set to 5 times the root mean square (RMS) value of the background noise one second prior at each time step, and a trigger threshold is set when the environment is absolutely silent. This is the base noise floor value. When a short-term energy level is detected to exceed the trigger threshold for the first time... At that time, the moment is determined to be the trigger point for the physical impact of the contact. .

[0029] To eliminate non-causal delays caused by sound wave transmission and ensure that acoustic and visual data are strictly aligned at the moment of physical impact, this embodiment performs a data capture operation with time delay compensation.

[0030] Triggering time point Based on this, audio and video data segments, namely short-time video images and short-time audio signals, are read separately from the circular buffer queue. Specifically: Audio data extraction: The reading time interval is The audio stream. Due to It is the moment when the sensor receives the sound, which actually lags behind the moment the physical impact occurs by approximately [missing information]. .

[0031] Video data capture: To align the video frame (optical signal, no delay) with the audio signal on the timeline, the video data capture window needs to be shifted backward. The reading time interval is... The video stream.

[0032] Through the time-shift operation described above, the captured audio stream and video stream are logically synchronized. Each moment corresponds to the actual instant of physical impact of the disconnector switch contacts. Without calibration, subsequent analysis will reveal non-physical phenomena where the visual response precedes the acoustic excitation, causing causality constraints to fail. The truncated data is defined as short-time audio streams and short-time video streams.

[0033] Step S003: Determine the sound intensity envelope vector based on the time-domain amplitude of the short-time audio signal; extract the frequency centroid vector of the short-time audio signal in the frequency domain; and extract the vibration amplitude vector representing the relative vibration response intensity of the contact area based on the displacement change characteristics of pixels in the short-time video image.

[0034] The time-domain amplitude of a sound signal directly reflects the instantaneous intensity of the mechanical kinetic energy injection system. For short-time audio streams, the sound intensity envelope vector representing the kinetic energy input is first extracted. .

[0035] Specifically, the analytic signal of a short-time audio stream is calculated using the Hilbert Transform, and its modulus is taken to obtain the instantaneous amplitude envelope. Due to the sampling rate of the short-time audio stream (…),… (much higher than discrete time series) Definition of frequency ( Therefore, the amplitude envelope is downsampled. An anti-aliasing filter is used to smooth the envelope signal. Then, a value is extracted every 48 sampling points, making its data length equal to that of the discrete time sequence. length Maintain consistency.

[0036] To meet the "mass conservation" calculation requirements of the subsequent optimal transmission model, the downsampled amplitude envelope sequence is L1 normalized, i.e., the sum of all elements in the sequence is set to 1. The resulting acoustic intensity envelope vector... It is a length of A one-dimensional vector, whose first... element Characterizes at time The relative acoustic energy ratio of the injected system.

[0037] Furthermore, the spectral composition of the sound is positively correlated with the equivalent contact stiffness of the contact surface: the greater the stiffness, the crisper the impact sound, and the higher the proportion of high-frequency components. To capture this physical characteristic, the frequency centroid vector is extracted. .

[0038] Perform a short-time Fourier transform (STFT) on a short-time audio stream, setting the sliding window length to... (Corresponding to 48 audio sampling points), and discrete time series The time intervals are consistent, and the window overlap rate is set to 50% to smooth the features. The power spectral density of the spectrum within each time window is calculated, and its centroid frequency is determined. The formula for calculating the centroid frequency is the integral of the product of the spectral amplitude and the frequency, divided by the integral of the spectral amplitude.

[0039] To ensure that the frequency characteristics have a matching weighted dynamic range in subsequent calculations, the centroid frequency needs to be corrected and normalized. Therefore, an upper limit for the effective physical frequency band is set. This value is set based on the main energy distribution frequency band of the impact sound from the metal contacts of the high-voltage disconnector (typically between 1kHz and 4kHz), which is much lower than the Nyquist frequency of the audio sampling. Specifically, the normalization formula is as follows: In the formula, for The normalized centroid frequency at time intervals also represents the frequency-centroid vector. middle Elements of time for The centroid frequency at time t is min(), which is the function to find the minimum value.

[0040] This process ensures the frequency centroid vector The numerical distribution is in Within the interval and within the effective frequency band, it has high distinguishability. The larger the value, the higher the contact stiffness at that moment.

[0041] Because the frame rate of surveillance cameras is limited by the sampling theorem, they cannot directly record high-frequency mechanical waveforms of several hundred hertz. Therefore, a "macroscopic time-domain envelope" is constructed to characterize the energy dissipation process of mechanical vibration.

[0042] For the contact monitoring area locked in a short video stream The local phase change in the vertical direction of the corresponding image sequence is calculated using the ComplexSteerable Pyramid algorithm. Compared to optical flow, phase analysis is insensitive to illumination changes and can extract sub-pixel-level minute displacement time signals. The sampling rate of the obtained displacement signal is still the same as the video frame rate. This refers to a low-frequency displacement signal.

[0043] Subsequently, the low-frequency displacement signal was upsampled using the cubic spline interpolation algorithm. , and discrete time series Alignment. The absolute value of the interpolated signal is taken to obtain an envelope curve reflecting the change in vibration amplitude. The physical basis for this process is that mechanical loosening caused by insufficient contact stiffness manifests macroscopically as a significant extension of the oscillation decay time. This "long tail" envelope characteristic is entirely within the effective sampling range of low-frequency video.

[0044] Finally, the envelope curve is L1 normalized to obtain the vibration amplitude vector. The first of the vectors element Characterizes at time The relative vibration response intensity exhibited by the contact area.

[0045] Thus far, acoustic excitation ( ) and visual response ( All of these have been mapped to a unified mathematical space. This embodiment aims to convert short-duration high-frequency audio streams and low-frequency video streams into physically meaningful discrete-time sequences. The three strictly aligned eigenvectors represent the system's kinetic energy input intensity, contact stiffness characteristics, and macroscopic persistence of mechanical response, respectively.

[0046] Step S004: Construct a time lag matrix representing the relative relationship between acoustic excitation and visual response at different time points. Add weights to the time lag matrix using the frequency centroid vector to obtain the final transmission cost matrix, which is used to establish a mathematical model for the discrete optimal transmission problem. Solve the mathematical model using the sound intensity envelope vector and vibration amplitude vector as constraints to obtain the energy attenuation delay index for each phase.

[0047] First, a basic time lag matrix is ​​constructed to define the fundamental mathematical cost of energy “transfer” between different points in time. This cost depends only on the time difference and must follow the physical causality law, i.e., the response cannot precede the excitation.

[0048] Initialize a dimension as A two-dimensional matrix, defined as a time lag matrix. Row index of the matrix Corresponding acoustic moment Column index Corresponding visual moment For matrix elements Its value is calculated based on the following logic: like This indicates that the visual response precedes the acoustic excitation, which violates the law of physical causality. Therefore, the corresponding matrix elements... Set to positive infinity ( This forces a ban on energy mapping along this spacetime path.

[0049] like This indicates that energy is dissipated along the positive time axis. The lag cost is calculated based on the square relationship of the time difference, and the specific expression is: ;in, As the energy dissipation reference constant, this embodiment considers the mechanical characteristics of outdoor high-voltage disconnect switches. Values The square operation is used to impose a high nonlinear penalty on long time delays, i.e., long-term vibration tails.

[0050] However, considering only time lag is insufficient to distinguish between normal, highly damped systems and anomalously loose systems. Therefore, this embodiment utilizes acoustic frequency domain characteristics as prior knowledge to construct a frequency-time weighted matrix. Additional stiffness penalty is imposed on the pathological combination of "high-frequency impact sound accompanied by long time lag".

[0051] First, initialize the dimension as follows: weight matrix For those that satisfy the law of causality ( ) elements Utilize time Normalized centroid frequency The weighting coefficients are calculated using the following expression: ;in, The theoretical damping characteristic time is taken as [value] in this embodiment. , representing the rapid decay time under an ideal rigid collision; The stiffness sensitivity gain coefficient is set to a value of [value missing] in this embodiment. ln() is a logarithmic function with the natural constant e as the base.

[0052] It should be understood that when When the frequency is relatively high (implying high stiffness), if a large time difference occurs simultaneously... (Visual long lag) will significantly amplify the weighting coefficient by multiplying the logarithmic term and the stiffness term, which will result in extremely high transmission costs for any feature matching that violates the physical law of "high stiffness should decay quickly".

[0053] By fusing physical constraints, the final transmission cost matrix is ​​synthesized. Its elements The product of the time lag cost at the corresponding location and the weighting coefficient (Hadamard product) is expressed as follows: .

[0054] Subsequently, a mathematical model for the discrete optimal transport problem is established, and a non-negative energy mapping matrix is ​​sought. (Element is) To minimize the total global transmission cost while satisfying the mass conservation constraint. The mathematical model is expressed as follows: The constraints are: Where min represents finding the minimum value. The sound intensity envelope vector represents the sound intensity at acoustic time. The corresponding element, This represents the vibration amplitude vector at the visual moment. The corresponding element.

[0055] To quickly solve this linear programming problem, this embodiment introduces an entropy regularization term and uses the Sinkhorn-Knopp algorithm for iterative solution. Regularization coefficient Set as The minimum objective function value obtained after the algorithm converges is the energy decay delay exponent. A lower energy decay delay index indicates that acoustic kinetic energy can be "mapped" to the visual response at a low cost in accordance with physical laws, meaning that the system has good contact stiffness; conversely, an abnormally high value suggests insufficient contact pressure.

[0056] Because outdoor wind loads and mechanical transmission gaps can cause waveform multi-peaks and noise, simple waveform alignment is insufficient to assess overall energy transfer efficiency. Therefore, this embodiment introduces a mathematical model of the discrete optimal transmission problem. It should be noted that "transmission" in this model refers to a mathematically defined characteristic distribution mapping metric, rather than the actual physical process of sound wave energy being converted into mechanical vibration. This embodiment uses acoustic characteristics... Defined as an "ideal stimulus distribution," visual features Defined as "actual response distribution", the contact state of the system is indirectly quantified by calculating the cost of the difference between the two distribution patterns.

[0057] Step S005: Determine the validity of the data using the energy attenuation delay index, further calculate the relative dispersion ratio of each phase, and detect the contact stiffness of each phase contact.

[0058] For the closing operation of the same high-voltage disconnector in the substation, the audio-visual data of phases A, B, and C, i.e., video images and audio signals, are processed in parallel. For each phase contact, the corresponding energy attenuation delay index is calculated according to the aforementioned steps, denoted as . , and Encapsulate these three scalar values ​​into a three-phase exponential vector for the current operation. .

[0059] Outdoor equipment is significantly affected by environmental factors; strong winds or minor ground vibrations can cause the monitoring screen to shake, resulting in a synchronous increase in the energy attenuation delay index of the three-phase contacts. To avoid false alarms, common-mode features are first used for filtering. The minimum value in the three-phase index vector is then calculated. Set environmental interference thresholds. In this embodiment .

[0060] like This indicates that even the best-performing device exhibits significant non-physical lag, suggesting that the current data is affected by environmental common-mode interference, such as overall shaking caused by gusts of wind. The system records event logs but does not trigger the device defect process. This indicates that the data is within the valid physical range and can proceed to the next stage of analysis.

[0061] After confirming the validity of the data, the specific phases with deteriorated contact performance are identified using differential mode characteristics. Since mechanical fatigue usually occurs randomly in a single phase, using the median as a benchmark can effectively mitigate the impact of single-point extreme values ​​on the overall evaluation.

[0062] First, calculate the median of the three-phase exponential vector. Where median() represents the median function. Subsequently, for each phase... Calculate its relative dispersion ratio .

[0063] Considering that the median is in excellent condition The denominator may approach zero, and direct division could lead to numerical overflow. Therefore, a numerical stability constant is introduced into the denominator. To avoid a denominator of 0, this embodiment... Values The implementer can set the relative dispersion ratio of the k-phase contacts according to the actual situation. The expression is: ;in, This represents the energy decay delay index of the k-phase contact.

[0064] Set alarm thresholds for If the relative dispersion ratio of a certain phase contact... If the hysteresis index of a phase is significantly higher than the baseline level (exceeding 50%), it is determined that the contact stiffness of that phase is potentially insufficient; otherwise, the contact stiffness of that phase is considered normal.

[0065] Based on the above judgment results, a closed-loop linkage strategy is automatically executed to verify potential hazards. When a potential hazard of insufficient contact stiffness is determined in a certain phase, taking phase A as an example, the following operations are performed: Generate a defect work order: Record the defects of this phase. Numerical value, relative dispersion ratio The original video images and audio signals are used to generate work orders to be processed.

[0066] Drive infrared temperature measurement: The system sends control commands to the infrared temperature measurement subsystem at the station, locks the spatial coordinate parameters of the A-phase contact, and adjusts its temperature measurement strategy from the conventional "periodic rotation" (e.g., once a day) to "high-frequency tracking" (e.g., once every 10 minutes).

[0067] Temperature rise verification: The system continuously monitors the temperature rise rate of phase A during subsequent peak load periods. If its temperature rise trend is found to be significantly higher than that of the other two phases (in this embodiment, the temperature difference exceeds 5K), it is determined to be a poor contact fault, and maintenance personnel are immediately notified through the SCADA system for handling.

[0068] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments of this specification have been described above. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0069] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0070] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them; modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions of some of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A cross-modal sensing and collaboratively driven intelligent inspection method for power systems across all scenarios, characterized in that, The method includes the following steps: Acquire video images and audio signals from the three-phase contacts of high-voltage disconnectors in the power system; and perform spatiotemporal alignment of the video images and audio signals; Identify the trigger point when the three-phase contacts of the switch physically collide, and extract the short-time video image and short-time audio signal corresponding to that time point after alignment; Based on the time-domain amplitude of the short-time audio signal, the sound intensity envelope vector is determined; the frequency centroid vector of the short-time audio signal in the frequency domain is extracted; and the vibration amplitude vector representing the relative vibration response intensity of the contact area is extracted based on the displacement change characteristics of pixels in the short-time video image. A time lag matrix representing the relative relationship between acoustic excitation and visual response at different time points is constructed. The frequency centroid vector is used to add weights to the time lag matrix to obtain the final transmission cost matrix, which is used to establish a mathematical model for the discrete optimal transmission problem. The sound intensity envelope vector and vibration amplitude vector are used as constraints to solve the mathematical model and obtain the energy attenuation delay index for each phase. The validity of the data is determined by the energy decay delay index, and the relative dispersion ratio of each phase is further calculated to detect the contact stiffness of each phase contact.

2. The intelligent inspection method for power systems across all scenarios based on cross-modal perception and collaborative driving as described in claim 1, characterized in that, The spatiotemporal alignment of video images and audio signals includes: Based on the distance between the video acquisition device and the audio acquisition device and the disconnect switch, the time delay calibration value is calculated to compensate for the time required for the sound wave to travel from the disconnect switch to the sensor. The video image and audio signal are spatiotemporally aligned based on the time delay calibration value.

3. The intelligent power system inspection method for cross-modal sensing and collaborative driving as described in claim 1, characterized in that, The process for identifying the trigger time point is as follows: When the short-term energy of the audio signal is detected to exceed the trigger threshold for the first time, that moment is determined to be the trigger point for the physical impact of the corresponding contact.

4. The intelligent power system inspection method for cross-modal sensing and collaborative driving as described in claim 1, characterized in that, The process of determining the sound intensity envelope vector is as follows: A frequency domain transformation is performed on the short-time audio signal to obtain the frequency domain amplitude envelope. The frequency domain amplitude envelope is sampled, processed, and normalized to form the sound intensity envelope vector.

5. The intelligent inspection method for power systems across all scenarios based on cross-modal perception and collaborative driving as described in claim 1, characterized in that, The extraction process of the frequency centroid vector is as follows: In short-time audio signals, a sliding window is set for the audio signal at each time point. The power spectral density of the audio signal spectrum within each sliding window is obtained, and its centroid frequency is calculated. After normalizing the centroid frequencies, the frequency-centroid vectors are formed.

6. The intelligent power system inspection method for cross-modal sensing and collaborative driving as described in claim 1, characterized in that, The process of extracting the vibration amplitude vector is as follows: For the locked contact monitoring area in short-time video images, the local phase change of the image sequence in the vertical direction is calculated to determine the displacement signal. The amplitude envelope of the displacement signal is normalized to obtain the vibration amplitude vector.

7. The intelligent inspection method for power systems across all scenarios based on cross-modal perception and collaborative driving as described in claim 1, characterized in that, The row index i of the time lag matrix corresponds to the acoustic moment. Column index j corresponds to visual time. For elements in the time lag matrix ,like ,Will If it is set to positive infinity, , ,in, This is a preset energy dissipation reference constant.

8. The intelligent inspection method for power systems across all scenarios based on cross-modal perception and collaborative driving as described in claim 7, characterized in that, The process of obtaining the transmission cost matrix is ​​as follows: Initialize a weight matrix with the same dimensions as the time lag matrix. At that time, calculate and The time difference between them is used to perform a logarithmic operation on the time difference, and the time lag matrix is ​​obtained. The weights of the elements and the result of the logarithmic operation, and the frequency centroid vector. The corresponding elements at each time point are all positively correlated; The product of the time lag matrix and the weight matrix is ​​used as the transmission cost matrix.

9. The intelligent inspection method for power systems across all scenarios based on cross-modal perception and collaborative driving as described in claim 8, characterized in that, The mathematical model is solved to obtain the energy decay delay index for each phase, including: Predefine a non-negative energy mapping matrix, where the elements are... The mathematical model states that the global total transmission cost, as the dependent variable, is equal to the sum of the products of all elements in the same position in the energy mapping matrix and the transmission cost matrix. The constraint is: at the same acoustic moment in the energy mapping matrix The sum of elements at all visual moments and the sound intensity envelope vector The corresponding elements at each time point are equal, and the same visual time point in the energy mapping matrix... The sum of the elements at all acoustic moments and the vibration amplitude vector The corresponding elements at any given time are equal; The energy decay delay index is obtained by minimizing the mathematical model using an optimization algorithm.

10. The intelligent power system inspection method for cross-modal sensing and collaborative driving as described in claim 1, characterized in that, The calculation of the relative dispersion ratio of each phase and the detection of the contact stiffness of each phase contact include: Obtain the median of the energy attenuation delay index corresponding to the three-phase contacts, calculate the sum of the median and a preset value greater than 0, and the relative dispersion ratio of each phase is the ratio of the energy attenuation delay index corresponding to each phase contact to the sum; if the relative dispersion ratio of any phase contact is greater than the preset alarm threshold, it is determined that the phase contact has a potential problem of insufficient contact stiffness, otherwise, the contact stiffness is determined to be normal.

Images