A multi-modal non-destructive testing system for power equipment based on acousto-optic fusion

Abstract

The application relates to the technical field of power equipment detection, and discloses a power equipment multi-modal nondestructive detection system based on sound-light fusion, which comprises an infrared thermal imaging sensor, an acoustic sensor array, a spatial electric field sensor and an edge computing unit. The spatial electric field sensor is used to acquire an alternating electric field fundamental wave signal, and a hardware phase-locked loop is used to output a power frequency phase angle signal as an acoustic operation synchronous clock. The non-uniformity correction flag of the infrared thermal imaging sensor is monitored to block the transmission of temperature jump signals caused by shutter correction. A dynamic phase window width is generated by using a temperature time derivative and an absolute temperature rise parameter, local discharge energy parameters are extracted, and when the threshold is met, a quadtree local high-resolution calculation is triggered, so that the positioning accuracy of a local discharge integral interval is improved, false triggering and redundant computing power consumption are reduced, and the high-resolution identification capability of a power equipment deterioration discharge area is improved.

Description

Technical Field

[0001] This invention relates to the field of power equipment testing technology, specifically to a multimodal nondestructive testing system for power equipment based on acoustic-optical fusion. Background Technology

[0002] When portable non-destructive testing equipment is used for condition monitoring of power equipment, the system lacks a physical phase reference source. This lack of a physical phase reference source can cause deviations in the partial discharge trigger phase angle positioning, preventing the processor from accurately defining the time integration interval of the partial discharge acoustic pulse.

[0003] In existing multimodal detection equipment, the infrared monitoring process and the acoustic monitoring process operate independently. The infrared thermal imaging sensor experiences shutter closure during non-uniformity correction. This shutter closure causes temperature reference jumps and infrared image freezing. Since the system lacks a connection between underlying hardware flags and acoustic computation logic, infrared image freezing can easily trigger false acoustic computations.

[0004] Furthermore, independently running data processing processes are prone to causing timing misalignment of heterogeneous data and wasting redundant computing power. The device cannot use the temperature-time derivative to modulate the acoustic computation time window span, nor can it use acoustic computation results to back-awaken local infrared high-resolution computation. The system processor continuously consumes a large amount of computing power for global scanning, and cannot trigger high-resolution computation mode on demand, making it impossible to balance reducing computing power consumption with completing high-resolution feature extraction of degraded discharge areas of power equipment.

[0005] Therefore, this invention proposes a multimodal nondestructive testing system for power equipment based on acoustic-optical fusion to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a multimodal nondestructive testing system for power equipment based on acoustic-optical fusion, which solves the problem of positioning deviation in the acoustic pulse time integration interval of partial discharge caused by the lack of a physical phase reference source in portable nondestructive testing equipment.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] This invention provides a multimodal non-destructive testing system for power equipment based on acoustic-optical fusion, comprising an infrared thermal imaging sensor, an acoustic sensor array, a spatial electric field sensor, and an edge computing unit. The edge computing unit internally includes a hardware phase-locked loop circuit and an analog-to-digital converter. The video data output terminal of the infrared thermal imaging sensor is connected to the edge computing unit. The digital audio output pin of the acoustic sensor array is connected to the edge computing unit. The analog signal output pin of the spatial electric field sensor is connected to a signal conditioning circuit. The output terminal of the signal conditioning circuit is connected across the reference frequency input terminal of the hardware phase-locked loop circuit.

[0009] The edge computing unit hierarchically divides the two-dimensional planar pixel array output from the infrared thermal imaging sensor according to a quadtree data structure. The edge computing unit divides the global field of view into multiple macroscopic spatial blocks. The edge computing unit extracts the two-dimensional coordinates of the pixel geometric center of each macroscopic spatial block within the two-dimensional planar pixel array. The edge computing unit calls a spatial projection algorithm to perform coordinate system transformation operations on the two-dimensional coordinates of the pixel geometric center to generate spatial coordinate offset angles. The edge computing unit establishes a spatial mapping hash table in its internal memory to store paired data of the macroscopic spatial block's independent index number and the spatial coordinate offset angle.

[0010] The space electric field sensor detects the fundamental wave signal generated by the alternating electric field in the space surrounding the power equipment. After processing by a signal conditioning circuit and a hardware phase-locked loop circuit, the fundamental wave signal is output as a power frequency phase angle signal. The power frequency phase angle signal serves as the synchronization clock for the edge computing unit to perform calculations on the data collected by the acoustic sensor array.

[0011] The edge computing unit internally divides into infrared processing and acoustic processing processes. The infrared processing process monitors the non-uniformity correction flag pin level at the bottom layer of the infrared thermal imaging sensor. When the non-uniformity correction flag pin is active, the infrared processing process blocks the thread calculating the temperature time derivative for the macroscopic spatial block. When the non-uniformity correction flag pin is silent, the infrared processing process extracts the average temperature inside the macroscopic spatial block, performs a first-order exponential smoothing filter, and calculates the temperature time derivative and absolute temperature rise parameters. The infrared processing process then pushes the corresponding spatial coordinate offset angle, temperature time derivative, and absolute temperature rise parameters into a first-in-first-out task queue.

[0012] The acoustic processing process reads the temperature-time derivative, absolute temperature rise parameter, and spatial coordinate offset angle stored in the first-in-first-out task queue. It then combines the temperature-time derivative and absolute temperature rise parameter to generate a phase window width ratio. Finally, it combines the reference phase window width parameter and the phase window width ratio to output a dynamic phase window width parameter specific to the current temperature derivative state.

[0013] The acoustic processing procedure performs beamforming on the multi-channel digital audio captured by the acoustic sensor array based on the spatial coordinate offset angle, outputting a single-channel acoustic signal. The acoustic processing procedure defines the time integration interval by combining the power frequency phase angle signal and the dynamic phase window width parameter. The acoustic processing procedure performs a time-domain subtraction operation by subtracting the historical single-channel acoustic signal array from the current period's single-channel acoustic signal array to cancel the steady-state mechanical noise floor. The acoustic processing procedure performs a summation operation on the difference signal array output from the time-domain subtraction operation to extract the partial discharge energy parameter.

[0014] The edge computing unit performs interval threshold decision-making on the partial discharge energy parameters to determine the system's execution status logic. When the partial discharge energy parameters fall within a set threshold range, the acoustic processing process sends a high-level toggle signal to the interrupt request register. The interrupt request register captures the high-level toggle signal, triggering the hard interrupt service routine of the edge computing unit's operating system. The hard interrupt service routine forces the infrared processing process to suspend the global image traversal loop and execute quadtree logic to split the corresponding macroscopic spatial block downwards into multiple sub-spatial blocks, transitioning to a local high-resolution computing mode.

[0015] The edge computing unit synchronously extracts the temperature-time derivative and partial discharge energy parameters output for the subspace block, merges them, and substitutes them into the calculation model to solve for the output acoustic-thermal coupling anomaly index. The edge computing unit inputs the acoustic-thermal coupling anomaly index into the system's internal pre-set fault risk level mapping table to determine the corresponding composite defect risk level. The edge computing unit packages the composite defect risk level with associated spatial coordinate information to generate a device alarm data packet and pushes it to the local display terminal.

[0016] This invention provides a multimodal nondestructive testing system for power equipment based on acoustic-optical fusion. It has the following beneficial effects:

[0017] 1. This invention uses a spatial electric field sensor to sense the fundamental wave signal generated by an alternating electric field. A hardware phase-locked loop circuit receives the fundamental wave signal and outputs a power frequency phase angle signal. This power frequency phase angle signal serves as the synchronization clock for the edge computing unit to process the data collected by the acoustic sensor array. The multimodal nondestructive testing system utilizes the spatial electric field coupling path to obtain the power grid reference phase. The edge computing unit delineates the time integration interval of the partial discharge acoustic pulse based on the power frequency phase angle signal, avoiding the operational boundary positioning deviation caused by the lack of a physical phase reference source in portable nondestructive testing systems.

[0018] 2. This invention monitors the non-uniformity correction flag pin level at the bottom layer of the infrared thermal imaging sensor through the infrared processing process. When the non-uniformity correction flag pin is active, the infrared processing process blocks the temperature-time derivative calculation thread for the macroscopic spatial block and outputs the temperature-time derivative calculation value stored in system memory. The edge computing unit uses the bottom-layer hardware flag to block the transmission of the temperature reference jump signal caused by the shutter closing action of the infrared thermal imaging sensor to the acoustic processing process, avoiding false triggering of acoustic calculations caused by the freezing of the infrared image.

[0019] 3. This invention generates a dynamic phase window width parameter by calculating the temperature-time derivative and absolute temperature rise parameter in the infrared processing process. The acoustic processing process combines the dynamic phase window width parameter to define a time integration interval and extract partial discharge energy parameters. When the partial discharge energy parameters fall within a set threshold range, a hard interrupt service routine is triggered. The infrared processing process responds to the hard interrupt service routine by executing quadtree logic to split the corresponding macroscopic spatial block downwards into multiple sub-spatial blocks and enter a local high-resolution calculation mode. The edge computing unit modulates the acoustic operation time window span using the temperature-time derivative and uses the acoustic calculation results to back-awaken the local infrared high-resolution operation. The multimodal non-destructive testing system completes high-resolution feature extraction and fusion calculation of the deteriorated discharge area of ​​power equipment while reducing the redundant computing power consumption of heterogeneous processors. Attached Figure Description

[0020] Figure 1 This is a system framework diagram of the present invention.

[0021] Figure 2 This is a flowchart of the method of the present invention.

[0022] Figure 3 This is a comparison chart of partial discharge energy parameter capture according to the present invention. Detailed Implementation

[0023] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] See attached document Figure 1 The present invention provides a multimodal non-destructive testing system for power equipment based on acoustic-optical fusion, including an infrared thermal imaging sensor, an acoustic sensor array, a spatial electric field sensor, and an edge computing unit.

[0025] The edge computing unit integrates a central processing unit, a graphics processing unit, and a neural network processor. The edge computing unit has a hardware phase-locked loop circuit and an analog-to-digital converter. The infrared thermal imaging sensor has a resolution of 640×512. The video data output terminal of the infrared thermal imaging sensor is connected to the parallel video interface of the edge computing unit. The acoustic sensor array consists of 128 channels of microelectromechanical system microphones. The digital audio output pins of the acoustic sensor array are multiplexed through a field-programmable gate array (FPGA) and then connected to the edge computing unit via the high-speed peripheral component interconnect standard (PCIe) bus. The infrared thermal imaging sensor acquires the temperature distribution image matrix on the surface of the power equipment, and the acoustic sensor array captures the sound pressure waveform emitted by the power equipment during operation.

[0026] The space electric field sensor adopts a capacitive sensing plate structure. The space electric field sensor is set at the front end of the housing of the multimodal non-destructive testing system. The analog signal output pin of the space electric field sensor is connected to the signal conditioning circuit. The output of the signal conditioning circuit is connected to the sampling channel of the analog-to-digital converter. The output of the signal conditioning circuit is connected across the reference frequency input of the hardware phase-locked loop circuit. The space electric field sensor senses the fundamental wave signal generated by the alternating electric field in the space around the power equipment. After the fundamental wave signal is processed by the signal conditioning circuit and the hardware phase-locked loop circuit, a 50Hz or 60Hz power frequency phase angle signal is output. The power frequency phase angle signal serves as the synchronization clock for the edge computing unit to perform calculations on the data collected by the acoustic sensor array.

[0027] The edge computing unit internally divides into infrared processing and acoustic processing. The infrared processing is mounted on the central processing unit and the graphics processing unit, while the acoustic processing is mounted on the central processing unit and the neural network processor. The memory space of the edge computing unit is allocated with an interrupt request register and a first-in-first-out (FIFO) task queue. The non-uniformity correction flag pin at the bottom layer of the infrared thermal imaging sensor is connected to the general input / output interface of the edge computing unit. The non-uniformity correction flag pin triggers the interrupt request register. The infrared processing process calculates the temperature-time derivative based on the state change of the non-uniformity correction flag pin. When the temperature-time derivative reaches a set value, the infrared processing process writes the calculation result and the corresponding spatial coordinate offset angle into the FIFO task queue. The acoustic processing process polls and reads the data in the FIFO task queue to start acoustic differential operation. The acoustic processing process generates a memory split interrupt instruction based on the calculation result to control the operation resolution of the infrared processing process.

[0028] See attached document Figure 1The edge computing unit's hardware baseboard integrates a system bus and a double data rate synchronous dynamic random access memory. The central processing unit (CPU) is divided into a multi-threaded scheduling core cluster, the graphics processing unit (GPU) is allocated a tensor operation unit array, and the neural network processor (NNZ) provides a dedicated multiply-accumulate operation array. The infrared processing process occupies the high-priority scheduling thread of the CPU and the tensor operation unit array of the GPU, while the acoustic processing process is bound to the low-priority scheduling thread of the CPU and the multiply-accumulate operation array of the NNZ. The infrared and acoustic processing processes open up a direct memory access shared area within the double data rate synchronous dynamic random access memory. The direct memory access shared area stores the spatial mapping hash table and matrix data generated by the multimodal nondestructive testing system during operation.

[0029] During the global timing monitoring phase, the infrared processing process writes the generated spatial offset angle data packets into the first-in-first-out (FIFO) task queue instantiated within the direct memory access shared region. The acoustic processing process periodically polls the FIFO task queue via the system bus to obtain trigger instructions and wake up the beamforming operation task. When the acoustic processing process determines that the partial discharge energy has reached a set threshold range, it sends a high-level toggle signal to the interrupt request register. The interrupt request register captures the high-level toggle signal and triggers the hard interrupt service routine of the edge computing unit operating system. The hard interrupt service routine forces the infrared processing process to suspend the global image traversal loop and switch to the quadtree high-resolution operation mode of the corresponding spatial region. The infrared processing process and the acoustic processing process rely on the FIFO task queue and the interrupt request register to build a bidirectional asynchronous communication feedback mechanism.

[0030] See attached document Figure 2 Based on the aforementioned multimodal nondestructive testing system for power equipment based on acoustic-optical fusion, this invention further provides a multimodal nondestructive testing method for power equipment based on acoustic-optical fusion, which includes the following steps:

[0031] S100: When the edge computing unit is powered on, it reads the calibration matrix output by the infrared thermal imaging sensor and the acoustic sensor array to establish a spatial mapping hash table based on a quadtree structure. The edge computing unit divides the infrared field of view into multiple macroscopic spatial blocks, extracts the geometric center projection of the macroscopic spatial blocks, generates the corresponding spatial coordinate offset angle, and writes it into the system memory. The edge computing unit simultaneously acquires the 50Hz or 60Hz power frequency phase angle signal extracted and extracted by the spatial electric field sensor to lock the system clock reference.

[0032] S200: The infrared processing process monitors the non-uniformity correction flag pin level to determine the system operating environment. When the non-uniformity correction flag pin is active, the infrared processing process suspends the temperature-time derivative calculation process and maintains the original numerical output. When the non-uniformity correction flag pin is silent, the infrared processing process extracts the average temperature inside the macroscopic spatial block and calculates the temperature-time derivative after passing through a first-order exponential smoothing filter. When the temperature-time derivative exceeds the environmental reference drift threshold, the infrared processing process pushes the corresponding spatial coordinate offset angle and the temperature-time derivative into the first-in-first-out task queue.

[0033] S300: The acoustic processing process reads the temperature-time derivative, absolute temperature rise parameter, and spatial coordinate offset angle stored in the first-in-first-out task queue. The acoustic processing process uses the acquired temperature-time derivative to reconstruct the underlying constants of the acoustic calculation and output the dynamic phase window width parameter.

[0034] S400: The acoustic processing process performs beamforming operations on the multi-channel digital audio captured by the acoustic sensor array based on the spatial coordinate offset angle, and outputs a single-channel acoustic signal. The acoustic processing process combines the power frequency phase angle signal and the dynamic phase window width parameter to define the time integration interval, and performs time-domain differential integration operations on the single-channel acoustic signal to extract and output partial discharge energy parameters.

[0035] S500, the edge computing unit performs interval threshold decision-making system execution state logic for partial discharge energy parameters. Under the condition that the partial discharge energy parameters fall into the set threshold interval, the acoustic processing process triggers the interrupt request register to send a high-level toggle signal to the infrared processing process. The infrared processing process responds to the interrupt request register by calling the hardware interrupt service subroutine to execute the quadtree logic to split the corresponding macroscopic space block downward into multiple subspace blocks and enter the local high-resolution computing mode.

[0036] S600: The edge computing unit synchronously extracts the temperature time derivative and the corresponding partial discharge energy parameters output for the subspace block, merges them into the calculation model to solve for the output acoustic-thermal coupling anomaly index. The edge computing unit inputs the acoustic-thermal coupling anomaly index into the system's internal preset fault risk level mapping table to output the corresponding composite defect risk level. The edge computing unit packages the composite defect risk level with the three-dimensional spatial coordinate information recorded in the system memory to generate a device alarm data packet and pushes it to the local display terminal.

[0037] The following section will provide a detailed explanation of each execution step in conjunction with the internal structure of the multimodal nondestructive testing system.

[0038] The S100 step includes the following sub-steps:

[0039] S101, the edge computing unit reads the calibration matrix burned into the infrared thermal imaging sensor and the acoustic sensor array during the factory stage. The calibration matrix is ​​composed of the camera intrinsic parameter matrix and the external translation and rotation matrix between heterogeneous sensors. The output image format of the infrared thermal imaging sensor is limited to a two-dimensional planar pixel array, and the output direction data of the acoustic sensor array is limited to three-dimensional spherical polar coordinate system parameters. The edge computing unit determines the origin of the multimodal global coordinate system based on the calibration matrix.

[0040] For the process of obtaining the calibration matrices of the intrinsic and extrinsic parameters of infrared thermal imaging sensors and acoustic sensor arrays, those skilled in the art use the Zhang Zhengyou checkerboard calibration method combined with the sound source emission reference array to perform calculations. The steps for obtaining the calibration matrices of the intrinsic and extrinsic parameters are well-known technologies in this field and will not be described in detail here.

[0041] S102, the edge computing unit divides the 640×512 resolution image pixel array output by the infrared thermal imaging sensor into hierarchical levels according to the quadtree data structure. At the root node level of the quadtree, the edge computing unit divides the global field of view into a constant number of macroscopic spatial blocks. The edge computing unit extracts the two-dimensional coordinates of the pixel geometric center of each macroscopic spatial block in the two-dimensional planar pixel array.

[0042] S103, the edge computing unit calls the spatial projection algorithm to perform coordinate system transformation operations on the extracted two-dimensional coordinates of the pixel geometric center. The edge computing unit uses the calibration matrix to project the two-dimensional coordinates of the pixel geometric center inside the two-dimensional planar pixel array to the three-dimensional spherical polar coordinate system corresponding to the acoustic sensor array. The edge computing unit calculates and generates the spatial coordinate offset angle pointing to the target spatial region. The algebraic processing logic of the edge computing unit performing the coordinate system projection operation refers to the following equation:

[0043] ;

[0044] In the formula, Representing the Spatial coordinate offset angle generated by the projection of a macroscopic spatial block; Representing the The pixel index value of the two-dimensional coordinates of the geometric center of a macroscopic spatial block pixel in the horizontal direction; Representing the The pixel index value of the two-dimensional coordinates of the geometric center of a macroscopic spatial block pixel in the vertical direction; This represents the joint calibration matrix of the infrared thermal imaging sensor and the acoustic sensor array; This represents the projection function of the multimodal coordinate system.

[0045] S104, the edge computing unit allocates an independent addressing space in the double data rate synchronous dynamic random access memory within the system. The edge computing unit sets the independent index number of the macroscopic spatial block as the key value and sets the spatial coordinate offset angle generated by the projection calculation as the data value. The edge computing unit pairs the key value and the data value and writes them into the addressing space to construct a spatial mapping hash table based on a quadtree structure. In subsequent operating cycles, the edge computing unit reads the corresponding spatial coordinate offset angle through the spatial mapping hash table based on the independent index number of the macroscopic spatial block. The edge computing unit replaces the pixel-by-pixel floating-point coordinate conversion operation generated during operation with a memory lookup operation with fixed time complexity by relying on the spatial mapping hash table.

[0046] Furthermore, the edge computing unit allocates and initializes a system-level mutex in a double data rate synchronous dynamic random access memory for the space-mapped hash table to establish a read-write protection mechanism to prevent dirty reads and race condition conflicts when multiple heterogeneous processors access the memory concurrently.

[0047] S105, the space electric field sensor utilizes a capacitive sensing plate structure to form a spatial coupling capacitor with the external charged body of the power equipment. The space electric field sensor senses the stray alternating electric field diffused in the space around the power equipment and converts the changing state of the alternating electric field into a weak induced current signal output.

[0048] S106 is a signal conditioning circuit arranged inside the multi-mode non-destructive testing system for inputting induced current signals. The signal conditioning circuit uses an integrated transimpedance amplifier module to convert the induced current signal into an AC voltage signal. The signal conditioning circuit uses a low-pass filter network to remove high-frequency electromagnetic interference components mixed in the AC voltage signal and outputs an analog fundamental signal.

[0049] S107, the analog fundamental signal is connected across the reference frequency input terminal of the hardware phase-locked loop circuit set inside the input edge computing unit. The hardware phase-locked loop circuit captures the zero-crossing characteristics of the analog fundamental signal and outputs a 50Hz or 60Hz power frequency phase angle signal that tracks the frequency of voltage change of the equipment.

[0050] For frequency tracking and phase locking within the hardware phase-locked loop circuit, those skilled in the art can consult integrated circuit manuals to obtain the voltage-controlled oscillator connection relationships. Its frequency tracking and phase locking are well-known technologies in the field and will not be elaborated here.

[0051] S108, the hardware phase-locked loop circuit outputs a 50Hz or 60Hz square wave synchronization pulse signal synchronized with the power grid frequency. This square wave synchronization pulse signal is directly connected to the hardware capture channel of the advanced timer inside the edge computing unit. The edge computing unit triggers a hardware capture interrupt on the transition edge of the square wave synchronization pulse signal, directly recording the absolute timestamp of the current system hardware clock as the starting reference for the power frequency cycle and allocating it to the acoustic processing process. The multimodal non-destructive testing system overcomes the engineering obstacle of the lack of a power grid phase reference source in portable devices through the electric field coupling path and the hardware timer capture mechanism.

[0052] The S200 step includes the following sub-steps:

[0053] S201, the non-uniformity correction flag pin at the bottom layer of the infrared thermal imaging sensor is electrically connected to the general input / output interface of the edge computing unit. The infrared processing process polls and reads the level signal of the general input / output interface in each monitoring cycle. The level signal corresponds to the active or silent state of the non-uniformity correction shutter inside the infrared thermal imaging sensor.

[0054] For the analysis of the pin level definition of the non-uniformity correction flag, those skilled in the art can consult the infrared detector datasheet to obtain the interface timing logic. The pin level definition of the non-uniformity correction flag is a well-known technology in this field and will not be elaborated here.

[0055] S202, when the non-uniformity correction flag pin is active as read by the general input / output interface, the infrared thermal imaging sensor shutter is in the closed stage, blocking the infrared radiation input. The infrared processing process responds to the status flag bit of the general input / output interface. The infrared processing process blocks the temperature time derivative calculation thread for the macroscopic spatial block in the central processing unit and the graphics processing unit. The infrared processing process directly calls the temperature time derivative calculation value of the previous cycle stored in the system memory for output. The infrared processing process blocks the transmission of the temperature reference jump signal caused by the shutter closure to the acoustic processing process.

[0056] S203: When the non-uniformity correction flag pin is in a silent state as read from the general-purpose input / output interface, the infrared thermal imaging sensor shutter opens and enters the normal spatial radiation infrared energy receiving state. The infrared processing process allows the blocked computing threads to proceed, and executes the subsequent task of extracting the average temperature inside the macroscopic spatial block. The multimodal non-destructive testing system utilizes low-level hardware flags to prevent false triggering of acoustic calculations caused by infrared image freezing.

[0057] Furthermore, in the first monitoring cycle when the non-uniformity correction flag pin has just switched from the active state to the silent state, in order to avoid the step response of the smoothing filter caused by the reconstruction of the sensor target surface thermal balance and the excessive time difference span, the infrared processing process executes the filter reset logic, forces the smoothed temperature value of the previous monitoring cycle to be equal to the average temperature of the current frame, and forces the temperature time derivative output of this cycle to be zero, providing a one-cycle operation silence period.

[0058] S204, the infrared processing process reads the two-dimensional planar pixel array output by the infrared thermal imaging sensor. The infrared processing process extracts the first pixel from the spatial mapping hash table based on a quadtree structure. The infrared processing steps calculate the temperature values ​​of all pixels within a macroscopic spatial block. The extracted temperature values ​​are then summed and averaged to obtain the... The current frame average temperature of a macroscopic spatial block.

[0059] S205, the infrared processing process calls a first-order exponential smoothing filter algorithm to perform temporal noise reduction on the average temperature of the current frame. The infrared processing process calculates the smoothed current mean value by combining the smoothed temperature data from the previous monitoring period. The infrared processing process then uses the time difference between the smoothed current mean value and the smoothed temperature value from the previous monitoring period to solve for the temperature time derivative. Simultaneously, the infrared processing process reads the system's preset ambient reference temperature and calculates the absolute temperature rise parameter by subtracting the ambient reference temperature from the smoothed current mean value. The algebraic processing logic for the temperature time derivative calculation by the infrared processing process is shown in the following equation:

[0060] ;

[0061] In the above formula, Representing the Temperature time derivative of a macroscopic spatial block; Represents the smoothing coefficient of a first-order exponential smoothing filter; Representing the The current frame average temperature of a macroscopic spatial block; Representing the The smoothed temperature values ​​of a macroscopic spatial block in the previous monitoring period; The time interval represents the monitoring period. The algebraic processing logic for calculating the absolute temperature rise parameter in the infrared processing procedure is shown in the following equation:

[0062] ;

[0063] In the above formula, Representing the The absolute temperature rise parameter of a macroscopic spatial block; Representing the The current frame average temperature of a macroscopic spatial block; This represents the system's preset ambient reference temperature.

[0064] S206, the infrared processing process reads the pre-set environmental reference drift threshold from the internal register of the edge computing unit. The environmental reference drift threshold characterizes the upper limit of the rate of natural temperature fluctuation around the multimodal nondestructive testing system. The infrared processing process performs a numerical comparison operation between the calculated temperature-time derivative and the environmental reference drift threshold.

[0065] For the calibration of the environmental reference drift threshold, those skilled in the art can use long-term recording of environmental temperature change curves to solve for the maximum slope and set it. The calibration of the environmental reference drift threshold is a well-known technology in this field and will not be described in detail here.

[0066] S207: When the temperature-time derivative exceeds the environmental reference drift threshold, or the absolute temperature rise parameter exceeds the system's preset static temperature rise alarm threshold, the infrared processing process determines that the corresponding macroscopic spatial block exhibits an abnormal temperature rise trend or is in a state of severe thermal degradation. The infrared processing process retrieves the... The spatial coordinate offset angles of each macroscopic spatial block are mapped in a spatial hash table. The infrared processing process encapsulates the spatial coordinate offset angles, the temperature-time derivative, and the absolute temperature rise parameters into a trigger command data packet.

[0067] The S300 procedure includes the following sub-steps:

[0068] S301: Abnormal heat is released when the insulation medium inside the power equipment deteriorates or when the contact resistance at conductive connections increases. This heat is conducted to the surface of the power equipment's casing, causing an increase in the temperature time derivative calculated by infrared processing. The insulation medium absorbs heat, causing volume expansion and deformation of the air gaps within it. Simultaneously, the relative permittivity of the dielectric material changes due to temperature.

[0069] S302, the partial discharge initiation voltage at the defect location of the insulating medium, shows a decreasing trend with the increase of the temperature-time derivative and absolute temperature rise due to the intensified internal electric field distortion and changes in gas ionization characteristics. Extensive field measurement data and statistical models from high-voltage equipment indicate that thermal degradation of the insulating medium leads to a significant broadening effect in the distribution range of its discharge trigger pulse within the power frequency phase angle.

[0070] S303, partial discharge occurs when the instantaneous value of the power frequency AC voltage applied across the electrical equipment exceeds the phase angle range of the partial discharge initiation voltage. A decrease in the partial discharge initiation voltage causes the phase angle node where the sinusoidal waveform of the power frequency AC voltage crosses the discharge trigger threshold to occur earlier.

[0071] S304, the premature discharge trigger phase angle node causes a widening of the phase angle range of the partial discharge acoustic pulse distribution within the 50Hz or 60Hz power frequency cycle. The acoustic processing process sets the temperature-time derivative from the infrared processing process as a control variable to adjust the underlying operational constants. The multimodal nondestructive testing system utilizes the temperature-time derivative of the infrared mode to construct an adaptive modulation theoretical benchmark for the acoustic mode operation time window.

[0072] S305, the acoustic processing process reads the reference phase window width parameter stored in the edge computing unit's memory. The reference phase window width parameter defines the original span occupied by the partial discharge acoustic pulse in the power frequency phase angle under conditions where no abnormal temperature rise has occurred. For the test records of the reference phase window width parameter, those skilled in the art can use a high-frequency current transformer to perform pulse phase statistics in a standard laboratory environment. The test records of the reference phase window width parameter are well-known technology in the field and will not be elaborated further here.

[0073] S306, the acoustic processing procedure establishes an adaptive modulation function for the phase window based on the exponential acceleration characteristic of thermal degradation of the insulating medium. To prevent abnormal compression or even negative time windows from occurring during the normal cooling period when the power equipment load decreases, the acoustic processing procedure first performs a unidirectional non-negative truncation operation on the input temperature-time derivative and absolute temperature rise parameters, forcibly assigning a value of zero when the relevant parameter values ​​are less than zero. Subsequently, the acoustic processing procedure inputs the truncated parameters into a nonlinear exponential mapping operation module with maximum boundary constraints to calculate and output the corresponding broadening ratio.

[0074] S307, the acoustic processing process combines the reference phase window width parameter and the expansion ratio to output a dynamic phase window width parameter for the current temperature derivative and absolute temperature rise state. The algebraic processing logic for the acoustic processing process to calculate the dynamic phase window width parameter refers to the following equation:

[0075] ;

[0076] In the formula, Represents the dynamic phase window width parameter; Represents the reference phase window width parameter; This represents the system's preset maximum window expansion ratio constant; Represents the natural constant exponentiation module; This represents a non-negative truncation function operation; The degradation rate weighting constant represents the degradation rate of the time derivative at the corresponding temperature. The thermal stress broadening weighting constant represents the corresponding absolute temperature rise parameter; The time derivative of the temperature input during the infrared processing; This represents the absolute temperature rise parameter transmitted during the infrared processing.

[0077] The multimodal nondestructive testing system utilizes this joint index mapping module to ensure that it can output the correct phase window expansion parameters during periods of rapid temperature rise or during periods of stable high temperature.

[0078] In S308, the acoustic processing process stores the calculated dynamic phase window width parameter in the register array of the central processing unit. The dynamic phase window width parameter provides reference data for subsequent acoustic processing processes to define the acoustic feature integration time range within the power frequency cycle. The edge computing unit relies on the dynamic phase window width parameter to complete the cross-modal data transfer from the infrared temperature status to the acoustic computation time boundary.

[0079] The S400 procedure includes the following sub-steps:

[0080] In step S401, before attempting to access the spatially mapped hash table, the acoustic processing process requests a system-level mutex lock from the system kernel. After successfully acquiring the lock, it parses the spatial coordinate offset angle obtained from the unpacking operation and immediately releases the system-level mutex lock after reading. The acoustic processing process decomposes the spatial coordinate offset angle into elevation and azimuth parameters in a three-dimensional spherical polar coordinate system. These elevation and azimuth parameters together serve as spatial control parameters guiding the acoustic main lobe direction of the multimodal nondestructive testing system.

[0081] The S402 acoustic sensor array, driven by the edge computing unit, synchronously samples sound pressure fluctuations in the power equipment operating environment. The acoustic sensor array outputs 128 independently sampled multi-channel digital audio, which is then converged and downsampled via a field-programmable gate array (FPGA) before being transmitted to the edge computing unit via Direct Memory Access (DMA) on a high-speed peripheral interconnect standard bus. The edge computing unit pre-allocates a multi-channel ring buffer within a double data rate synchronous dynamic random access memory (DRAM), configured with an address space covering at least 10 power frequency cycles. The DMA mechanism uses a pointer-based circular overwrite mode to continuously write the multi-channel digital audio along with an absolute timestamp based on the system clock into the multi-channel ring buffer.

[0082] S403, the acoustic processing process reads the spatial coordinate topology of each channel's microelectromechanical system (MEMS) microphone within the acoustic sensor array. Based on elevation and azimuth parameters, the acoustic processing process calculates the time-of-flight difference between the target sound source and each MEMS microphone. The acoustic processing process then generates time delay compensation parameters corresponding to the multi-channel digital audio. For the measurement record of the acoustic sensor array's spatial coordinate topology, those skilled in the art can use laser interferometry to calibrate the microphone's relative position coordinates. The measurement record of the acoustic sensor array's spatial coordinate topology is well-known in the field and will not be elaborated upon here.

[0083] S404, the acoustic processing process schedules the multiply-accumulate array of the neural network processor to apply time delay compensation parameters to the multi-channel digital audio. The acoustic processing process performs a weighted summation operation on the time-aligned multi-channel digital audio. The acoustic processing process outputs a single-channel acoustic signal that converges the sound energy of the target macroscopic spatial block region. The algebraic processing logic for beamforming operations performed by the acoustic processing process is referenced in the following equation:

[0084] ;

[0085] In the formula, The single-channel acoustic signal representing the computational output; The channel index number representing the acoustic sensor array; Representative bestowed upon the first The element amplitude weighting coefficients for each channel; Representing the The original multi-channel digital audio was captured in one channel; The representation is generated based on the spatial coordinate offset angle for the first... Delay compensation parameters for each channel; Represents the independent variable of time.

[0086] The S405 edge computing unit utilizes beamforming operations to create a spatial filtering effect. The multimodal nondestructive testing system relies on the spatial coordinate offset angle output by the infrared thermal imaging sensor to filter out background environmental noise that deviates from the abnormal temperature rise area and purify high-frequency pulse waveforms diverging in a specific direction.

[0087] S406, the acoustic processing process reads the absolute timestamp recorded by the advanced timer inside the edge computing unit when a hardware interrupt is captured. The acoustic processing process directly sets this absolute timestamp as the time start reference for an independent power frequency cycle. The system skips the software zero-crossing point search for the low-frequency analog phase and replaces the software calculation delay based on ADC sampling with nanosecond-level hardware trigger recording.

[0088] S407, the acoustic processing procedure defines the operation boundary based on the time start reference and the phase interval where partial discharge of power equipment is prone to occur. Partial discharge pulses are densely distributed in the voltage rising edge stages of the positive and negative half cycles of the power frequency AC sinusoidal waveform. Since the external space electric field is generated by the superposition of multi-phase voltage vectors, its zero-crossing point has an unknown offset from the partial discharge initiation phase angle of a single physical phase.

[0089] Because there is an unknown offset between the zero-crossing point extracted from the external space electric field and the phase angle at the onset of partial discharge in a single physical phase, the system introduces a phase offset locking mechanism. In the first feature extraction cycle for a specific macroscopic spatial block, the acoustic processing process performs global sliding integration within a complete power frequency cycle of 50Hz or 60Hz to extract the maximum energy peak value, and records and locks the real time offset between the time point corresponding to the peak value and the zero-crossing time starting reference. In subsequent continuous monitoring cycles, the acoustic processing process directly uses the zero-crossing time starting reference superimposed with the locked real time offset as the phase window starting time point, and extends it backward to the time span covered by the aforementioned dynamic phase window width parameter, defining a single precise closed time integration interval, thereby significantly reducing the redundant computing power consumption of the multi-core neural network processor and avoiding phase offset interference.

[0090] S408, the single-channel acoustic signal is mixed with steady-state mechanical noise generated by the magnetostrictive vibration of the transformer core or the operation of the cooling fan. The steady-state mechanical noise maintains a strictly repeating waveform within adjacent power frequency cycles. Partial discharge acoustic pulses exhibit discrete and randomly occurring transient time characteristics. The acoustic processing process extracts the current-cycle single-channel acoustic signal array falling within the time integration interval. The acoustic processing process reads the historical single-channel acoustic signal array from the previous power frequency cycle that is in the same phase interval. The acoustic processing process performs a time-domain subtraction operation by subtracting the historical single-channel acoustic signal array from the current-cycle single-channel acoustic signal array to cancel out the steady-state mechanical noise floor.

[0091] S409, the acoustic processing process performs a square summation operation on the difference signal array output from the time-domain subtraction operation to obtain the total transient pulse energy. The acoustic processing process calculates and outputs partial discharge energy parameters. The algebraic processing logic for the time-domain difference integration operation of the acoustic processing process is referenced in the following equation:

[0092] ;

[0093] In the formula, This represents the partial discharge energy parameters extracted from the calculation; The discrete index number representing the time-dependent independent variable; This represents the starting time point of the phase window generated by superimposing the zero-crossing time reference. The dynamic window width parameter represents the number of discrete sampling points. This represents a single-channel acoustic signal that falls within the current time integration interval; This represents the offset of the number of discrete sampling points corresponding to a 50Hz or 60Hz power frequency cycle (its value is equal to the sampling frequency of the analog-to-digital converter divided by the power frequency). This represents the historical single-channel acoustic signal extracted after the offset delay of the above sampling points.

[0094] The S500 procedure includes the following sub-steps:

[0095] S501, the acoustic processing process extracts the partial discharge energy parameters output by the time-domain differential integration operation. The acoustic processing process accesses the system's internal double data rate synchronous dynamic random access memory to read the pre-programmed set threshold range. The set threshold range is defined by a combination of the lower limit of the ambient background noise and the upper limit of the pulse amplitude. The specific numerical calibration of the upper and lower limits of the set threshold range can be determined by those skilled in the art based on the statistical model of partial discharge acoustic pulses in typical high-voltage equipment combined with field test experience data. The specific numerical calibration of the upper and lower limits of the set threshold range is well-known technology in the field and will not be elaborated here.

[0096] In S502, the edge computing unit calls the arithmetic logic unit inside the central processing unit to perform interval threshold decision operations on the partial discharge energy parameters. The arithmetic logic unit performs bidirectional numerical comparison operations on the partial discharge energy parameters with the lower limit of the ambient background noise and the upper limit of the pulse amplitude within the set threshold interval.

[0097] S503: When the partial discharge energy parameter is below the lower limit of the ambient background noise or above the upper limit of the pulse amplitude, the edge computing unit determines that the extracted result belongs to residual mechanical vibration interference or external space lightning electromagnetic pulse-related interference. The edge computing unit discards the partial discharge energy parameter and performs a dequeue operation on the current task node that triggered the current event in the first-in-first-out task queue, while retaining other task nodes in the queue. The infrared processing process maintains its current state of temperature-time derivative calculation for the macroscopic spatial block dimension.

[0098] S504, under the condition that the partial discharge energy parameter falls within the set threshold range, the edge computing unit determines that there are indeed acoustic characteristics of insulation degradation discharge in the associated area inside the target power equipment. The acoustic processing process locks the independent index number of the macroscopic spatial block corresponding to the source of the partial discharge energy parameter.

[0099] In S505, the acoustic processing process takes over the operation of the internal system control bus of the edge computing unit. The acoustic processing process writes a forced trigger control word to a specific register address mapped to the interrupt controller (GIC) integrated on-chip within the edge computing unit. This write operation generates a cross-core software interrupt (SGI) signal within the edge computing unit, which is routed via the on-chip system bus to the central processing unit core cluster hosting the infrared processing process. The multimodal nondestructive testing system utilizes the cross-core interrupt routing mechanism of the internal bus to establish a high-speed cross-modal communication link that uses acoustic calculation results to trigger high-precision infrared calculations.

[0100] In S506, when the infrared processing process captures a cross-core software interrupt signal routed by the interrupt controller, it first polls the non-uniformity correction flag pin level of the general-purpose input / output interface. If the flag pin is active (i.e., shutter closed), the CPU sets the cross-core software interrupt signal to a pending state and delays the response; if the flag pin is silent (i.e., shutter open), the infrared processing process suspends the current global macroscopic spatial block polling scan task. The CPU inside the edge computing unit saves the current context state of the arithmetic registers and program counter, and then jumps to execute a pre-programmed interrupt service routine in memory to establish a cross-modal response mechanism.

[0101] S507, the hardware interrupt service routine reads the independent index number of the target macroscopic spatial block passed by the acoustic processing process. The infrared processing process executes quadtree logic based on this independent index number. The infrared processing process performs bisection operations on the two-dimensional planar pixel boundaries of the target macroscopic spatial block along the horizontal and vertical directions. The infrared processing process then splits the corresponding macroscopic spatial block downwards into four sub-space blocks with equal pixel areas.

[0102] Before executing the quadtree logic, the infrared processing process compares the current pixel resolution of the target macroscopic spatial block with the system's preset minimum pixel splitting threshold. If the current pixel resolution of the target macroscopic spatial block has reached the minimum pixel splitting threshold, the infrared processing process stops splitting downwards and directly retains the current resolution to enter high-resolution calculation mode, preventing excessive consumption of system memory and recursive deadlock.

[0103] For the memory node generation and boundary partitioning algorithms of quadtree data structures, those skilled in the art can consult basic computer data structure materials to obtain the code implementation logic. The memory node generation and boundary partitioning algorithms are well-known technologies in this field and will not be described in detail here.

[0104] S508, the infrared processing process re-extracts the two-dimensional coordinates of the pixel geometric centers corresponding to the multiple subspace blocks. The infrared processing process calls a spatial projection algorithm containing an intrinsic and extrinsic parameter calibration matrix to generate subspace coordinate offset angles that match the multiple subspace blocks. Before performing hash table updates, the infrared processing process requests to acquire the aforementioned system-level mutex lock. If the mutex lock is already held by the acoustic processing process, it suspends and waits. After acquiring the system-level mutex lock, the infrared processing process writes the independent index number of the subspace block and the subspace coordinate offset angle into a double data rate synchronous dynamic random access memory, dynamically updates the spatial mapping hash table based on a quadtree structure, and releases the mutex lock upon completion of the write operation.

[0105] Meanwhile, to prevent the smoothing filter algorithm from crashing or generating division-by-zero anomalies due to a lack of historical data in the initial stage of the local high-resolution computing mode, the infrared processing process, in the first monitoring cycle after switching to high-resolution computing mode, retrieves the original image of the previous frame from the system memory based on the system's absolute timestamp, and uses the historical average temperature of the pixels in the region where the four subspace blocks are located as the cold start historical state approximation value when performing the first first-order exponential smoothing filter calculation for multiple subspace blocks.

[0106] The S509 multimodal nondestructive testing system switches to a local high-resolution computing mode. The edge computing unit performs local computing power reallocation on the internal processing chip. The edge computing unit terminates the computation threads in non-abnormal areas and centrally maps the parallel computing resources of the graphics processor to the pixel regions covered by multiple sub-space blocks. The infrared processing process extracts the pixel temperature values ​​within each sub-space block and calculates and outputs temperature-time derivatives with high spatial resolution. The algebraic processing logic for calculating the temperature-time derivatives of the sub-space blocks in the infrared processing process is referenced in the following equation:

[0107] ;

[0108] In the formula, Representing the The first macroscopic spatial block is generated by downward splitting and decomposition. Temperature time derivative of each subspace block; The branch index number representing the subspace block, with values ​​limited to integers from 1 to 4; Represents the smoothing coefficient of a first-order exponential smoothing filter; Representing the The average temperature of the current frame for all pixels contained in each subspace block; Representing the The smoothed temperature value of each subspace block in the previous monitoring period; The time interval representing the monitoring cycle.

[0109] Similarly, the infrared processing extracts pixel temperature values ​​from multiple sub-blocks to calculate an absolute temperature rise parameter with high spatial resolution, denoted as... .

[0110] S510, the acoustic processing process responds to the local high-resolution computing mode, retrieving the subspace coordinate offset angles corresponding to multiple subspace blocks from the double data rate synchronous dynamic random access memory. Simultaneously, the acoustic processing process extracts the absolute timestamp corresponding to the moment the hardware interrupt was triggered. Using this absolute timestamp as a retrieval reference, the acoustic processing process backtracks to access the multi-channel circular buffer, retrieving the unoverwritten multi-channel digital audio historical data segments. The acoustic processing process schedules the tensor operation unit array to re-execute beamforming and time-domain differential integration operations on this historical data segment, pointing to the aforementioned subspace coordinate offset angles. The acoustic processing process then calculates and outputs the partial discharge energy parameters corresponding one-to-one with each of the multiple subspace blocks.

[0111] The S600 procedure includes the following sub-steps:

[0112] S601, the edge computing unit locks the first step of the downward split. Each subspace block is processed by an edge computing unit. The edge computing unit retrieves the temperature-time derivative of the infrared processing output for that subspace block from the double data rate synchronous dynamic random access memory. Simultaneously, the edge computing unit extracts the partial discharge energy parameters calculated by the acoustic processing unit at the spatial coordinate offset angle corresponding to that subspace block.

[0113] S602, the edge computing unit performs timing alignment operations on the extracted temperature time derivative and partial discharge energy parameters based on the absolute timestamp of the system clock.

[0114] For timestamp alignment and synchronization buffer queue allocation of heterogeneous sensors before data fusion, those skilled in the art can use hardware nanosecond-level timers combined with time series matching algorithms to perform the operation. The timestamp alignment and synchronization buffer queue allocation are well-known technologies in the field and will not be described in detail here.

[0115] S603, the edge computing unit reads the pre-configured energy gain constant from the internal non-volatile memory. The energy gain constant establishes the dimensional balance between infrared thermal characteristic data and acoustic characteristic data during numerical computation.

[0116] In S604, the edge computing unit calls the floating-point unit within the central processing unit to perform algebraic fusion calculations. The floating-point unit multiplies the time-aligned temperature derivative and partial discharge energy parameters to establish cross-coupling relationships. The floating-point unit then inputs the product result, along with the energy gain constant, into a logarithmic function model to solve for the quantization index. The algebraic processing logic for the edge computing unit to solve the acoustic-thermal coupling anomaly index is based on the following equation:

[0117] ;

[0118] In the formula, This represents the calculated acoustic-thermal coupling anomaly index; This represents the logarithmic operation module with the natural constant as the base; This represents the system's preset energy gain constant; and These represent the dynamic fusion weights assigned by the system to the time derivative of temperature and the absolute temperature rise parameter, respectively; Representative regarding the first Temperature time derivative of each subspace block output; Representative regarding the first The absolute temperature rise parameters output by each subspace block; This represents the partial discharge energy parameter that matches the spatial orientation of the subspace block; the number 1 is used to prevent the logarithmic function from generating meaningless negative infinity values ​​when there is no abnormal input.

[0119] The S605 edge computing unit utilizes the nonlinear compression characteristics of the logarithmic function model to map large-span dynamic fluctuations of the product variable into a smooth and convergent numerical sequence. The edge computing unit relies on a mathematical fusion model to convert isolated single-modal parameters into multi-modal comprehensive evaluation indicators.

[0120] S606, the edge computing unit accesses its internal non-volatile memory to read a preset fault determination threshold array. The fault determination threshold array is divided into multiple discrete numerical intervals, each corresponding to a progressively increasing fault level from normal operation and minor degradation of the equipment's insulation to potential severe breakdown. The edge computing unit invokes the arithmetic logic unit to compare the acoustic-thermal coupling anomaly index with the boundary values ​​in the fault determination threshold array level by level.

[0121] S607, when the acoustic-thermal coupling anomaly index crosses the threshold boundary representing the normal state at the bottom layer, the edge computing unit triggers the alarm generation mechanism. The edge computing unit locks the first [element / unit] that generated the acoustic-thermal coupling anomaly index. Each subspace block is processed. The edge computing unit extracts the pixel coordinate parameters of the subspace block in the two-dimensional planar pixel array. The edge computing unit retrieves the subspace coordinate offset angle that matches the subspace block from a spatial mapping hash table based on a quadtree structure.

[0122] The S608 edge computing unit drives its internal direct memory access controller to move heterogeneous parameters to the communication transmission buffer. The edge computing unit concatenates the device physical identifier, system absolute timestamp, acoustic-thermal coupling anomaly index, fault level, pixel coordinate parameters, and subspace coordinate offset angle. The edge computing unit sequentially combines data fragments to generate a standardized spatial coordinate alarm packet. The algebraic processing logic for data encapsulation performed by the edge computing unit is shown in the following equation:

[0123] ;

[0124] In the formula, Represents the spatial coordinate alarm packet generated by splicing edge computing units; The physical identifier code representing the multimodal nondestructive testing system; This represents the system's absolute timestamp when the alarm was triggered; The acoustic-thermal coupling anomaly index represents the output of the fusion calculation; This represents the fault level derived from the acoustic-thermal coupling anomaly index mapping. Representing the Pixel coordinate parameters of each subspace block; Represents the value retrieved from the space-mapped hash table and the first... The subspace coordinate offset angle corresponding to each subspace block.

[0125] S609, the edge computing unit converts the generated spatial coordinate alarm packet into a serial data stream. The edge computing unit controls the physical layer communication interface to send the serial data stream to the remote monitoring backend. The monitoring backend anchors the spatial location of the device's deterioration discharge in the physical environment model based on the pixel coordinate parameters and subspace coordinate offset angle contained in the spatial coordinate alarm packet.

[0126] For the encapsulation of communication protocols and error verification at the network transmission layer, those skilled in the art can use the Transmission Control Protocol combined with the Cyclic Redundancy Check algorithm to perform the calculation. The encapsulation of communication protocols and error verification at the network transmission layer are well-known technologies in the field and will not be elaborated here.

[0127] In the S610 multimodal nondestructive testing system, after issuing a spatial coordinate alarm packet, the edge computing unit maintains the high-resolution tracking operation mode of the target subspace block and dequeues the currently completed task nodes from the first-in-first-out task queue, while retaining other pending spatial mapping trigger instructions in the queue. The infrared processing process inside the edge computing unit resets the pin level of the interrupt request register and writes the independent index number of the target subspace block into the interrupt mask table allocated in memory.

[0128] During the preset cooling period, the acoustic processing process ceases sending cross-core interrupt triggers to the infrared processing process for the target subspace block to prevent the system bus from experiencing an interrupt storm due to continuous equipment degradation and discharge. Simultaneously, the edge computing unit continuously calculates and pushes high-resolution multimodal feature value updates for the region to the local display terminal, enabling continuous and uninterrupted audio-visual tracking of high-risk defects until the defect is downgraded or reset. The remaining concurrent threads in the multimodal nondestructive testing system synchronously enter the next round of non-contact data acquisition and feature extraction.

[0129] Specific application examples:

[0130] An internal insulation of a 110kV transformer bushing deteriorated due to moisture during operation. The edge computing unit monitors the insulation according to a 1-second cycle. ) to perform polling.

[0131] Set the system's preset ambient reference temperature .

[0132] exist At that moment, for the first time the casing is located A macroscopic spatial block, the smoothed temperature value of the previous monitoring period. .

[0133] The average temperature of the current frame acquired by the current infrared thermal imaging sensor .

[0134] The smoothing coefficient of the first-order exponential smoothing filter is set to .

[0135] The infrared processing steps are calculated according to the formulas in the instruction manual:

[0136] First, calculate the smoothed current mean:

[0137] ;

[0138] Next, the temperature-time derivative is calculated. and absolute temperature rise parameters :

[0139] ;

[0140] ;

[0141] because Greater than the set environmental reference drift threshold (set to) The infrared processing process pushes data into the first-in-first-out task queue.

[0142] Heating of the bushing alters the ionization characteristics of the internal insulating medium, causing the partial discharge pulse to be triggered earlier, and the originally fixed phase window can no longer fully capture the pulse.

[0143] The acoustic processing process reads data and reconstructs underlying constants:

[0144] The system-defined reference phase window width parameter (Corresponding to a portion of the phase angle region at a 50Hz power frequency).

[0145] Maximum window width ratio constant .

[0146] Degradation rate weighting constant Thermal stress broadening weighting constant .

[0147] Will and Substitute the dynamic window width parameter into the calculation formula:

[0148] Index section:

[0149] ;

[0150] Natural constant exponentiation: ;

[0151] ;

[0152] Based on this, the edge computing unit dynamically widened the time integration interval from the original 2.0ms to 4.439ms, successfully covering the discharge pulse group that broadened due to heat, and calculated and extracted the true partial discharge energy parameters. (Relative energy unit).

[0153] because If the value falls within a set threshold range, a cross-core software interrupt signal is triggered, and the quadtree logic is executed to split the region downwards into four subspace blocks.

[0154] Assuming the most severe anomaly is locked... Subspace blocks:

[0155] High-resolution temperature-time derivative after focusing in this region .

[0156] High-resolution absolute temperature rise parameters .

[0157] High-fidelity partial discharge energy parameters after re-beamforming .

[0158] Substitute the acoustic-thermal coupling anomaly index Fusion formula (assuming energy gain constant) Dynamic fusion weights , ):

[0159] ;

[0160] ;

[0161] Edge computing unit determination If the fault judgment threshold (e.g., 5.0) corresponding to a serious breakdown risk is exceeded, a spatial coordinate alarm package is finally generated and pushed.

[0162] The conclusions are as follows:

[0163] Reference Appendix Figure 3 In the first 15 monitoring cycles (i.e., the period when the equipment temperature is stable), the capture results of the traditional method and the system of the present invention are basically consistent.

[0164] However, as operating time progresses, the bushing insulation begins to heat up abnormally, causing a significant broadening of the discharge phase distribution and an earlier onset phase angle. Traditional systems, using a fixed time integration interval, experience a large number of high-frequency pulses slipping out of the computational boundary, resulting in a "distorted cliff-like drop" in the calculated partial discharge energy parameters, which can easily lead to the system missing serious defects.

[0165] The edge computing unit of this invention relies on the real-time temperature-time derivative and absolute temperature rise parameters calculated by the infrared processing process to synchronously update the dynamic phase window width parameters in the background. This allows the integration interval of the acoustic processing process to adaptively stretch in accordance with the physical phenomena, successfully extracting the continuously rising malignant partial discharge energy parameters completely. This demonstrates that, in a COTS heterogeneous hardware environment, this invention overcomes the physical bottleneck of traditional single-dimensional detection by utilizing cross-modal parameter correlation analysis, improving the diagnostic robustness for thermal degradation-related discharge defects.

Claims

1. A multimodal nondestructive testing system for power equipment based on acoustic-optical fusion, characterized in that, Includes infrared thermal imaging sensors, acoustic sensor arrays, spatial electric field sensors, and edge computing units; The infrared thermal imaging sensor acquires a matrix of temperature distribution images on the surface of the power equipment. The acoustic sensor array captures the sound pressure waveform emitted by the power equipment during operation; The spatial electric field sensor senses the fundamental wave signal generated by the alternating electric field in the space surrounding the power equipment and outputs the power frequency phase angle signal for acoustic data processing; The edge computing unit extracts the spatial coordinate offset angle, temperature time derivative, and absolute temperature rise parameter based on the temperature distribution image matrix. When the temperature time derivative or absolute temperature rise parameter meets the set conditions, it triggers an acoustic detection task pointing to the corresponding spatial region. The edge computing unit combines the power frequency phase angle signal and the spatial coordinate offset angle to perform acoustic calculations on the sound pressure waveform to obtain partial discharge energy parameters. The edge computing unit generates a memory splitting interrupt instruction based on the partial discharge energy parameter, and in turn controls the edge computing unit to improve the infrared image processing resolution corresponding to the corresponding spatial region.

2. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 1, characterized in that, The edge computing unit integrates a central processing unit, a graphics processing unit, and a neural network processor, and is equipped with a hardware phase-locked loop circuit and an analog-to-digital converter. The video data output terminal of the infrared thermal imaging sensor is connected to the parallel video interface of the edge computing unit; The acoustic sensor array consists of 128 channels of microelectromechanical system microphones. The digital audio output pins of the acoustic sensor array are multiplexed by a field-programmable gate array and then connected to the edge computing unit via a high-speed peripheral component interconnect standard bus. The space electric field sensor adopts a capacitive sensing plate structure. The analog signal output pin of the space electric field sensor is connected to the sampling channel of the analog-to-digital converter via a signal conditioning circuit, and the output terminal of the signal conditioning circuit is connected across the reference frequency input terminal of the hardware phase-locked loop circuit.

3. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 2, characterized in that, The hardware phase-locked loop circuit outputs a square wave synchronization pulse signal that tracks the power frequency, and the square wave synchronization pulse signal is directly connected to the hardware capture channel of the advanced timer inside the edge computing unit. The edge computing unit triggers a hardware capture interrupt on the transition edge of the square wave synchronization pulse signal and directly records the absolute timestamp of the current system hardware clock as the starting reference of the power frequency cycle, which is then allocated to the acoustic processing process.

4. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 1, characterized in that, The edge computing unit divides the infrared field of view into multiple macroscopic spatial blocks and reads the calibration matrix output by the infrared thermal imaging sensor and the acoustic sensor array to establish a spatial mapping hash table based on a quadtree structure. The edge computing unit extracts the two-dimensional coordinates of the pixel geometric center of the macroscopic spatial block and calls the spatial projection algorithm to perform coordinate system transformation operations to generate spatial coordinate offset angles; The edge computing unit allocates an independent addressing space in its internal double data rate synchronous dynamic random access memory, sets the independent index number of the macroscopic spatial block as the key value, sets the spatial coordinate offset angle as the data value, and writes them in pairs to construct the spatial mapping hash table.

5. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 4, characterized in that, The edge computing unit internally divides the process into infrared processing and acoustic processing, and creates an interrupt request register and a first-in-first-out task queue in the memory space. The non-uniformity correction flag pin of the infrared thermal imaging sensor is electrically connected to the general input / output interface of the edge computing unit and triggers the interrupt request register. The infrared processing process extracts the average temperature inside the macroscopic spatial block when the non-uniformity correction flag pin is in a silent state, and calculates the temperature time derivative and absolute temperature rise parameters after first-order exponential smoothing filtering. When the temperature-time derivative exceeds the environmental reference drift threshold, the infrared processing process encapsulates the corresponding spatial coordinate offset angle, the temperature-time derivative, and the absolute temperature rise parameter into a trigger instruction data packet and writes it into the first-in-first-out task queue.

6. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 5, characterized in that, The acoustic processing process polls and reads the trigger instruction data packets stored in the first-in-first-out task queue; After performing a unidirectional non-negative truncation operation on the acquired temperature time derivative and absolute temperature rise parameter, the acoustic processing process inputs a nonlinear exponential mapping operation module with maximum boundary constraints to calculate and output the broadening ratio. The acoustic processing process combines the reference phase window width parameter stored in the edge computing unit memory with the widening ratio to reconstruct the underlying constants of acoustic calculation and output the dynamic phase window width parameter for the current temperature derivative and absolute temperature rise state.

7. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 6, characterized in that, The acoustic sensor array outputs multi-channel digital audio, and the acoustic processing process decomposes the acquired spatial coordinate offset angle into elevation and azimuth parameters in a three-dimensional spherical polar coordinate system. The acoustic processing process calculates the time difference of flight of the target sound source to each microelectromechanical system microphone based on the elevation angle parameter and the azimuth angle parameter, and calculates and generates the time delay compensation parameter corresponding to the multi-channel digital audio. The acoustic processing process schedules the multiply-accumulate array of the neural network processor to perform a weighted summation operation on the multi-channel digital audio using the time delay compensation parameters, and outputs a single-channel acoustic signal that converges the sound energy of the target macroscopic spatial block region.

8. The multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 7, characterized in that, The acoustic processing process combines the power frequency phase angle signal with the dynamic phase window width parameter to define the time integration interval; The acoustic processing process extracts the current period single-channel acoustic signal array that falls within the time integration interval, and performs a time-domain subtraction operation by subtracting the historical single-channel acoustic signal array of the previous power frequency cycle. The acoustic processing process performs a square summation operation on the difference signal array output by the time-domain subtraction operation to obtain the total energy of the transient pulse, and calculates and outputs the partial discharge energy parameters.

9. A multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 8, characterized in that, The edge computing unit performs interval threshold decision on the partial discharge energy parameters; When the partial discharge energy parameter falls within a set threshold range, the acoustic processing process sends a high-level toggle signal to the interrupt request register to trigger the memory split interrupt instruction. The edge computing unit captures the high-level toggle signal and triggers a hardware interrupt service routine, forcing the infrared processing process to suspend the global image traversal loop. The infrared processing process executes quadtree logic to split the corresponding macroscopic spatial block into multiple sub-space blocks and enters a local high-resolution calculation mode. It then re-extracts the two-dimensional coordinates of the corresponding pixel geometric center and calculates and updates the sub-space coordinate offset angle that matches the multiple sub-space blocks.

10. A multimodal nondestructive testing system for power equipment based on acoustic-optical fusion according to claim 9, characterized in that, The edge computing unit synchronously extracts the temperature-time derivative and the corresponding partial discharge energy parameters output for the subspace block, and performs timing alignment operation based on the absolute timestamp of the system clock. The edge computing unit performs a cross-coupling operation by multiplying the time derivative of the temperature after time alignment with the partial discharge energy parameter, and combines the energy gain constant with the logarithmic function model to solve for the quantitative index and generate the acoustic-thermal coupling anomaly index. The edge computing unit inputs the acoustic-thermal coupling anomaly index into the fault risk level mapping table, outputs the corresponding composite defect risk level, and packages it with pixel coordinate parameters and subspace coordinate offset angle to generate a device alarm data packet and pushes it to the local display terminal.

Images