Multimodal imaging method and system based on acoustic-thermal coupling

Abstract

The invention discloses a multi-modal imaging method and system based on acoustic-thermal coupling, and relates to the field of artificial intelligence, and the method comprises the steps: firstly obtaining thermal radiation data and ultrasonic reflection data of a target equipment region, and obtaining thermal radiation embedded features and ultrasonic embedded features through feature representation learning; performing multi-stage compensation on the thermal radiation embedding characteristic to generate a thermal field compensation reference characteristic of each stage; synchronously carrying out equal-stage multi-stage compensation on the ultrasonic embedded features, coupling the ultrasonic compensation result of the previous stage with the thermal field reference features of the corresponding stage between every two continuous stages of compensation, and combining the ultrasonic embedded features and coupling output in the later stage of compensation; and finally, performing imaging by using the acoustic-thermal coupling characteristics after multi-stage compensation to obtain an acoustic-thermal coupling imaging result fusing the thermal distribution characteristics of the target equipment region and the ultrasonic reflection spectrum characteristics. According to the method, deep fusion of acoustic and thermal information is realized through multi-stage feature compensation and dynamic cross-modal coupling, and the comprehensiveness and accuracy of equipment area detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to a multimodal imaging method and system based on acoustic-thermal coupling. Background Art

[0002] In the field of equipment status detection and fault diagnosis, multimodal imaging technology can achieve a more comprehensive characterization of equipment area features by fusing different physical field information. Traditional single-modal imaging has limitations in insufficient information dimension: infrared thermal imaging is vulnerable to environmental thermal interference and it is difficult to accurately locate deep thermal anomalies; ultrasonic imaging is insensitive to thermal distribution and cannot directly reflect the degree of thermal damage of equipment. Existing multimodal fusion methods mostly stay at data layer superposition or decision layer fusion, and do not fully utilize the multi-stage complementary characteristics of the feature layer, resulting in loss of feature details or information redundancy, and it is difficult to meet the high-precision detection requirements of complex equipment areas. Summary of the Invention

[0003] The purpose of the present invention is to provide a multimodal imaging method and system based on acoustic-thermal coupling.

[0004] In a first aspect, an embodiment of the present invention provides a multimodal imaging method based on acoustic-thermal coupling, including: Obtaining thermal radiation data and ultrasonic reflection data of a target equipment area; Performing feature representation learning on the thermal radiation data and the ultrasonic reflection data respectively, and correspondingly obtaining thermal radiation embedding features and ultrasonic embedding features; Performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedding features, and correspondingly obtaining thermal field compensation reference features of the target equipment area in the thermal radiation data under multiple feature compensation stages; Performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding features in sequence; wherein, the feature compensation stages of the multi-stage ultrasonic feature compensation processing are consistent with the feature compensation stages of the multi-stage thermal radiation feature compensation processing; between every two consecutive stages of ultrasonic feature compensation processing, performing feature coupling processing on the ultrasonic feature compensation processing result obtained from the previous stage of ultrasonic feature compensation processing in the two-stage ultrasonic feature compensation processing and the thermal field compensation reference feature of the corresponding feature compensation stage; when performing the latter stage of ultrasonic feature compensation processing in the two-stage ultrasonic feature compensation processing, performing ultrasonic feature compensation processing on the ultrasonic embedding features and the feature coupling output after the feature coupling is completed; Performing imaging reconstruction on the acoustic-thermal coupling features obtained after the multi-stage ultrasonic feature compensation processing, and obtaining an acoustic-thermal coupling imaging result including the thermal distribution features of the target equipment area and the reflected wave spectrum characteristics of the ultrasonic reflection data.

[0005] In a second aspect, an embodiment of the present invention provides a server system, including a server configured to execute the method described in the first aspect.

[0006] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a multimodal imaging method and system based on acoustic-thermal coupling disclosed by the present invention, which relates to the field of artificial intelligence and includes: first, obtaining thermal radiation data and ultrasonic reflection data of a target device area, and respectively obtaining thermal radiation embedding features and ultrasonic embedding features through feature representation learning; performing multi-stage compensation on the thermal radiation embedding features to generate thermal field compensation reference features at each stage; synchronously performing multi-stage compensation with the same number of stages on the ultrasonic embedding features, and between every two consecutive stages of compensation, coupling the ultrasonic compensation result of the previous stage with the thermal field reference feature of the corresponding stage, and the subsequent stage of compensation combines the ultrasonic embedding features and the coupling output; finally, imaging using the acoustically-thermally coupled features after multi-stage compensation to obtain an acoustically-thermally coupled imaging result that fuses the thermal distribution features and ultrasonic reflection spectrum characteristics of the target device area. This method realizes the deep fusion of acoustic and thermal information through multi-stage feature compensation and dynamic cross-modal coupling, improving the comprehensiveness and accuracy of device area detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0008] Figure 1 It is a schematic flowchart of the steps of the multimodal imaging method based on acoustic-thermal coupling provided by the embodiment of the present invention; Figure 2 It is a schematic block diagram of the structure of the computer device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0009] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0010] The following will detail the specific implementation manners of the present invention with reference to the drawings.

[0011] To solve the technical problems in the aforementioned background art, Figure 1It is a schematic flowchart of the multimodal imaging method based on acoustic-thermal coupling provided by an embodiment of the present disclosure. The multimodal imaging method based on acoustic-thermal coupling will be introduced in detail below.

[0012] Step S201: Obtain thermal radiation data and ultrasonic reflection data of a target device area. Step S202: Perform feature representation learning on the thermal radiation data and the ultrasonic reflection data respectively, and correspondingly obtain a thermal radiation embedding feature and an ultrasonic embedding feature. Step S203: Perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, and correspondingly obtain thermal field compensation reference features of the target device area in the thermal radiation data at multiple feature compensation stages. Step S204: Perform multi-stage ultrasonic feature compensation processing on the ultrasonic embedding feature in sequence; wherein, the feature compensation stages of the multi-stage ultrasonic feature compensation processing are consistent with the feature compensation stages of the multi-stage thermal radiation feature compensation processing; between every two consecutive stages of ultrasonic feature compensation processing, perform feature coupling processing on the ultrasonic feature compensation processing result obtained from the previous stage of ultrasonic feature compensation processing in the two-stage ultrasonic feature compensation processing and the thermal field compensation reference feature of the corresponding feature compensation stage; when performing the latter stage of ultrasonic feature compensation processing in the two-stage ultrasonic feature compensation processing, perform ultrasonic feature compensation processing on the ultrasonic embedding feature and the feature coupling output after the feature coupling is completed. Step S205: Perform imaging reconstruction on the acoustic-thermal coupling feature obtained after the multi-stage ultrasonic feature compensation processing, and obtain an acoustic-thermal coupling imaging result including the thermal distribution feature of the target device area and the reflection spectrum characteristics of the ultrasonic reflection data.

[0013] In an embodiment of the present invention, exemplarily, first, the server synchronously collects thermal radiation data and ultrasonic reflection data of the target device area through standardized data interaction interfaces such as industrial Ethernet and Internet of Things communication protocols. Taking the substation transformer status monitoring scenario as an example, the thermal radiation data is obtained by an infrared thermal imager deployed around the transformer. The thermal imager scans key areas such as transformer windings, cores, and bushings at a frame rate of 60 frames per second, generating a thermal radiation image with a resolution of 1024×768 and pixel gray value mapping to a temperature range of -40°C to 500°C. The ultrasonic reflection data is collected by a phased array ultrasonic transducer. The transducer emits ultrasonic waves with a center frequency of 5 MHz into the transformer and receives the reflected waves, generating an ultrasonic A-scan data sequence in the "time-amplitude" format with a sampling rate of 20 MHz, a duration of 50 μs, and a sequence length of 10,000 points. The server obtains the two types of data in real time through the industrial network to ensure coverage of the target device area. In the rail transit traction motor detection scenario, the server interfaces with an on-vehicle infrared thermal imager (collecting thermal radiation of the motor housing) and an ultrasonic probe (collecting ultrasonic reflection of the rotor shafting) to synchronously obtain the thermal-acoustic data of the motor stator and rotor areas.

[0014] Next, the server calls the pre-trained feature mapping model to perform feature representation learning on the thermal radiation data and ultrasonic reflection data respectively. For the thermal radiation data, a thermal feature mapper based on an improved ResNet architecture (including 4 convolutional blocks + 2 fully connected layers) is started. The first convolutional block extracts local thermal patterns such as the temperature boundary between the "winding-core" using a 3×3 convolutional kernel. The subsequent convolutional blocks strengthen the feature response of the thermal anomaly areas (such as winding hot spots) through residual connections. Finally, the feature dimension is compressed to 512 dimensions through the fully connected layer to obtain the thermal radiation embedded feature (encoding core information such as the thermal field temperature distribution, thermal gradient direction, and thermal anomaly spatial position. For example, when the temperature of a certain phase winding hot spot is 15°C higher than the normal area, the corresponding dimension in the feature shows a high activation value). For the ultrasonic reflection data, an ultrasonic feature mapper composed of a 1D convolutional layer and a gated recurrent unit (GRU) is started. The 1D convolutional layer extracts time-domain features such as "amplitude attenuation, wave peak interval" in segments, and the GRU layer mines the long-range dependencies of the reflected waves (such as the correlation of multiple reflected waves caused by internal defects), outputting a 512-dimensional ultrasonic embedded feature (covering information such as ultrasonic wave propagation energy loss, the number of reflection interfaces, and waveform distortion caused by defects. For example, when an additional wave peak appears in the ultrasonic reflection wave due to insulation layer damage, the corresponding dimension in the feature is activated).

[0015] Subsequently, the server calls a sound-thermal feature calibrator (composed of 3 cascaded first feature compensation components, each component containing a residual module + spatial attention mechanism) to perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedding features: During the first-stage compensation, the first first feature compensation component receives the thermal radiation embedding features, and the spatial attention module enhances the feature weights of the winding hot spots (such as boosting the feature response of the hot spot area from 0.4 to 0.85). The residual module retains the global distribution information of the thermal field and outputs the first-stage thermal field compensation reference features (depicting the theoretical thermal distribution without environmental interference, such as excluding the influence of environmental wind speed on the heat exchange of the radiator); During the second-stage compensation, the middle-layer first feature compensation component takes the output of the previous stage as the input, combines the prior knowledge of the transformer cooling structure (such as the arrangement of radiator fins and the heat conduction coefficient) to correct the spatial deviation of the thermal field (such as correcting the thermal distribution distortion caused by the infrared lens angle), and generates the second-stage thermal field compensation reference features; During the third-stage compensation, the target first feature compensation component incorporates the transformer load curve (synchronously retrieved by the server from the power monitoring system) to compensate for the impact of load fluctuations on the thermal field (such as differentiating between "normal heat generation due to high load" and "abnormal heat generation due to internal defects"), obtaining the third-stage thermal field compensation reference features. At this time, the thermal features restore the true thermal state of the device to the greatest extent.

[0016] When performing multi-stage ultrasonic feature compensation processing on ultrasonic embedded features, the server ensures that the number of compensation stages is consistent with the thermal radiation compensation stage (i.e., 3 stages). And between every two consecutive stages, the ultrasonic feature compensation result of the previous stage is coupled with the thermal field compensation reference feature of the corresponding stage, and the coupled output is used as the input for the ultrasonic processing of the next stage: In the first-stage ultrasonic compensation, the first second feature compensation component (consisting of a convolutional layer, batch normalization, and ReLU) receives the ultrasonic embedded features, extracts the ultrasonic reflection basic structure features (such as the interface reflection wave features between the winding insulation layer and the iron core), and outputs the first-stage ultrasonic output features; This feature and the first-stage thermal field compensation reference feature enter the cross-correlation feature alignment component (feature coupling sub-module), first add element by element in the energy dimension to generate a combined feature, perform peak compression (retaining high-energy features of wave peaks) and mean compression (smoothing background noise) on the combined feature to obtain a compressed feature, calculate the acoustic-thermal coupling modulation coefficient (such as 0.75, which represents the modulation degree of the thermal field on the ultrasonic features) through 1×1 convolution, and weighted adjust the combined feature based on this coefficient to generate the first-stage feature coupling output; During the second-stage ultrasonic compensation, the target second feature compensation component receives both the ultrasonic embedded features and the first-stage coupling output at the same time, fuses the ultrasonic original structure information and the space information calibrated by the thermal field (such as the hot spot position guiding the ultrasonic feature to focus on the reflection wave analysis of the corresponding area) through depthwise separable convolution, and outputs the second-stage ultrasonic output features. This feature and the second-stage thermal field compensation reference feature repeat the coupling process to generate the second-stage coupling output; The third-stage ultrasonic compensation logic is the same as the previous two stages, and finally the acoustic-thermal coupling features after multi-stage ultrasonic feature compensation are obtained. This feature deeply correlates the thermal field and ultrasonic information in the high-dimensional space (such as the strong correspondence between the feature dimension of the thermal anomaly area and the ultrasonic defect reflection feature dimension).

[0017] Finally, the server calls the imaging reconstruction network (including the encoding path and the decoding path) improved based on U-Net to perform imaging reconstruction on the acoustic-thermal coupling features: the encoding path performs multi-scale feature extraction on the 512-dimensional acoustic-thermal coupling features to capture the fine-grained association between the thermal field and ultrasound (such as the spatial correspondence between the position of thermal anomaly and the reflection of ultrasound defects); the decoding path gradually restores the image resolution through deconvolution layers and skip connections (restored to 1024×768 pixels). In the visual presentation, the thermal distribution is encoded in pseudo-color as "blue→red" (temperature from low to high), and the ultrasound reflection characteristics are encoded in grayscale (the higher the brightness, the greater the reflection intensity). Taking the detection of transformer winding faults as an example, in the imaging result, a certain winding area appears highlighted in red (thermal anomaly, temperature 130°C exceeds the normal threshold of 80°C), and the corresponding position in the ultrasound image appears highlighted in white (a sudden increase in reflection intensity indicates an internal defect). This association points to the fault logic of "local discharge caused by insulation layer damage leading to thermal anomaly"; the server pushes the reconstructed image to the operation and maintenance workstation, and automatically marks the fault location, thermal parameters (temperature value, thermal gradient) and ultrasound parameters (amplitude of reflected wave, number of wave peaks) synchronously, providing multi-dimensional basis for operation and maintenance decision-making. In the detection scenario of rail transit traction motors, the server logic is the same as the above process: the thermal feature mapper focuses on the thermal distribution of the motor stator and bearings, and the ultrasound feature mapper analyzes the ultrasound reflection of the rotor shaft system; the acoustic-thermal feature calibrator incorporates the train vibration (compensating for the interference in thermal distribution measurement) and the shaft system speed (correcting the ultrasound Doppler frequency shift); the feature coupling strengthens the spatial correspondence between "bearing hot spots" and "abnormal ultrasound reflection"; finally, the imaging clearly presents the association between bearing faults and thermal-acoustic features, assisting in judging poor lubrication or mechanical damage.

[0018] Through "multi-stage compensation + cross-modal coupling", this method realizes the coordination of the thermal field spatial positioning ability and the ultrasound structure analysis ability: the thermal field provides a prior for the defect location of ultrasound, and ultrasound provides an explanation for the cause of thermal field anomalies, greatly improving the detection accuracy of internal defects of equipment (such as the detection accuracy of transformer winding insulation defects is increased by 23% compared with a single modality) and the fault tracing ability (the positioning error of traction motor bearing faults is reduced to ±2mm), providing core technical support for the intelligent operation and maintenance of industrial equipment.

[0019] In the embodiment of the present invention, the multi-stage thermal radiation feature compensation processing of the thermal radiation embedded features, corresponding to obtaining the thermal field compensation reference features of the target equipment area in the thermal radiation data under multiple feature compensation stages, can be implemented through the following examples.

[0020] Through multiple intermediate layers in the trained acoustic-thermal feature calibrator, perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedded features, corresponding to obtaining the thermal field compensation reference features of the target equipment area in the thermal radiation data under multiple feature compensation stages; Among them, when performing the first-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, the first intermediate layer among the multiple intermediate layers is used to perform the first-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, so as to obtain the thermal field compensation reference feature of the target device area in the thermal radiation data in the first stage; When performing the target-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, the target intermediate layer among the multiple intermediate layers is used to perform the target-stage thermal radiation feature compensation processing on the thermal field compensation reference feature of the previous stage, so as to obtain the thermal field compensation reference feature of the target device area in the thermal radiation data in the target stage, and the previous stage is the previous stage of the target stage.

[0021] In an embodiment of the present invention, exemplarily, when performing multi-stage thermal radiation feature compensation processing on thermal radiation embedding features, the server calls a trained acoustic-thermal feature calibrator (the calibrator includes 3 cascaded intermediate layers, and each intermediate layer is a first feature compensation component integrating a residual module and a spatial attention mechanism) to calibrate the thermal radiation embedding features of the target device area (taking the 110 kV transformer winding in a substation as an example) in stages. First, perform the first-stage thermal radiation feature compensation processing: The server inputs the 512-dimensional thermal radiation embedding feature learned from the transformer thermal radiation data (this feature has encoded information such as winding temperature distribution and hot spot location, but is affected by environmental wind speed, infrared lens installation angle, etc.) into the first intermediate layer of the acoustic-thermal feature calibrator. The spatial attention module in the first intermediate layer strengthens the feature response of the winding hot spot area through weight allocation (for example, increasing the activation value of the feature dimension corresponding to the winding hot spot pixel from 0.4 to 0.85), and the residual module retains the basic information of the global thermal field distribution; after this processing, the server outputs the first-stage thermal field compensation reference feature, which has preliminarily excluded environmental interference (such as the false temperature fluctuations caused by wind speed near the radiator being corrected), and is closer to the "theoretical thermal distribution of the winding without external interference", providing a basis for subsequent accurate analysis. When performing the thermal radiation feature compensation processing in the target stage (such as the second stage), the server calls the target intermediate layer of the acoustic-thermal feature calibrator (i.e., the second first feature compensation component), and uses the "first-stage thermal field compensation reference feature" as the input (the output of the previous stage). The target intermediate layer incorporates the prior knowledge of the transformer heat dissipation structure (such as parameters such as the arrangement of radiator fins and the heat conduction coefficient, which are synchronously retrieved by the server through an industrial database) to correct the thermal distribution distortion caused by the infrared lens angle deviation in the features of the previous stage: For example, if there is a "false high temperature band" on the right side of the winding in the original thermal image due to lens tilt, the target intermediate layer reallocates the temperature weights of the corresponding area in the feature according to the physical law of heat conduction along the fins (the rate of heat transfer along the longitudinal direction of the fins and the constraint of lateral diffusion), reducing the false high temperature feature response from 0.7 to 0.3, and finally outputting the second-stage thermal field compensation reference feature. At this time, the thermal features more accurately reflect the true thermal state of the winding (such as the temperature gradient between the fins and the error of the theoretical heat dissipation model is reduced to ±2°C). If the target stage is the third stage (the final compensation stage), the server calls the third intermediate layer of the acoustic-thermal feature calibrator (the third first feature compensation component), and continues to use the "second-stage thermal field compensation reference feature" as the input (the output of the previous stage).The intermediate layer synchronizes and accesses the real-time load curve of the transformer (obtained by the server from the power monitoring system, covering parameters such as voltage, current, and active power). Through the correlation model of load-thermal dissipation (such as the empirical formula that for every 10% increase in the load rate, the theoretical temperature rise of the winding increases by 3°C), it distinguishes between "normal heat distribution caused by excessive load" and "abnormal heat aggregation caused by internal defects": Suppose the load rate of the transformer reaches 85% at a certain moment (the normal load upper limit is 80%). The intermediate layer first calculates the theoretical temperature rise caused by the load exceeding the limit (such as 3°C), and then compares it with the actual temperature rise in the characteristics (if it reaches 5°C, it is determined that there is an abnormality). Through the weight adjustment of the feature dimension, the activation value of the features in the abnormal heating area is strengthened from 0.6 to 0.9. Finally, the thermal field compensation reference feature in the third stage is generated. This feature has maximally stripped the influence of external interference and normal operating conditions, accurately depicting the thermal anomaly of the winding caused by internal defects (such as inter-turn short circuit), and providing high-confidence thermal field reference information for subsequent coupling with ultrasonic features. Through the staged iterative compensation of multiple intermediate layers of the acoustic-thermal feature calibrator, the server gradually purifies the effective information in the thermal radiation embedded features, enabling the thermal field compensation reference features in each stage to not only retain the global integrity of the thermal state of the equipment but also focus on the local specificity of key defects, laying a precise foundation for the multi-modal imaging of acoustic-thermal coupling in the thermal field dimension.

[0022] In the embodiment of the present invention, the intermediate layer is the first feature compensation component, and the first feature compensation component includes a feature compensation operator; The multiple intermediate layers in the trained acoustic-thermal feature calibrator perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedded features, and correspondingly obtain the thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages. The implementation can be carried out through the following examples.

[0023] By invoking the feature compensation operators in each of the first feature compensation components in the multiple intermediate layers, multi-stage thermal radiation feature compensation processing is performed on the thermal radiation embedded features, and the thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages are correspondingly obtained.

[0024] In an embodiment of the present invention, exemplarily, when performing multi-stage thermal radiation feature compensation processing on thermal radiation embedding features, in the acoustic-thermal feature calibrator called by the server, each "intermediate layer" corresponds to a first feature compensation component, and each component is built-in with a feature compensation operator that can achieve feature purification and interference elimination (such as a combined operator of "residual connection + spatial attention mechanism + domain prior constraint"). Taking the thermal radiation analysis scenario of the 110kV transformer winding in a substation as an example: The server first inputs a 512-dimensional thermal radiation embedding feature encoding the thermal distribution of the winding (including information such as hot spot position and temperature gradient, but mixed with noises such as environmental wind interference and infrared lens distortion) into the first first feature compensation component of the acoustic-thermal feature calibrator. This component calls the "spatial attention-residual" sub-module in the feature compensation operator: The spatial attention sub-module performs weight reallocation on the spatial dimension of the thermal radiation embedding feature (corresponding to the pixel distribution of the infrared thermal image). For the hot spot area of the winding (such as a pixel cluster with a temperature of 120°C in the infrared thermal image), it increases the activation weight of its feature dimension from the initial 0.4 to 0.85; the residual connection sub-module retains the basic information of the global thermal field distribution (such as the overall temperature gradient relationship between the iron core and the winding). After this processing, the first-stage thermal field compensation reference feature is output, and this feature has initially filtered out the interference of environmental wind on the temperature measurement near the radiator (such as the false high-temperature pixels caused by air flow at the edge of the radiator, and its feature weight is reduced from 0.6 to 0.2). When processing to the second first feature compensation component, the server uses the first-stage thermal field compensation reference feature as the input. The feature compensation operator of this component incorporates the "prior of transformer heat dissipation structure" (such as parameters such as the arrangement pitch of radiator fins and the thermal conductivity of aluminum fins, which the server retrieves from the device digital twin model). The operator corrects the thermal distribution distortion caused by the tilt of the infrared lens installation in the previous feature through the "structural constraint convolution" sub-module: If there is a "stretched temperature band" on the right side of the winding in the original thermal image due to the lens angle deviation (which should actually be a uniform thermal distribution), the operator recalibrates the temperature weight of the corresponding area in the feature according to the longitudinal heat conduction rate of the fins (transferring 0.5°C / s per millimeter) and the transverse heat diffusion constraint (the transverse heat transfer attenuation coefficient between fins is 0.3), and adjusts the feature activation value of the distorted area from 0.7 to 0.5 symmetric to the left side, generating the second-stage thermal field compensation reference feature. At this time, the thermal distribution of the winding in the thermal feature has a high degree of match with the physical characteristics of the heat dissipation structure (such as the temperature decay gradient at the end of the fins has an error less than 1°C from the theoretical model). When entering the third first feature compensation component, the server uses the second-stage thermal field compensation reference feature as the input, and the feature compensation operator of this component calls the "load-heat dissipation correlation model" (trained based on the transformer nameplate parameters and historical load-temperature rise curves).The operator accesses the real-time load data of the transformer (such as the current load rate of 90%, which is synchronously obtained by the server from the power monitoring SCADA system), and first calculates the theoretical temperature rise increment caused by the overload (for every 10% increase in the load rate, the theoretical temperature rise of the winding increases by 3°C); then compares the actual temperature rise deviation in the features: if the temperature corresponding to the activation value of a certain winding area feature is 2°C higher than the theoretical load temperature rise, the operator determines that it is abnormal heating caused by an internal defect (such as inter-turn short circuit), and strengthens the activation value of this area feature from 0.6 to 0.9 through the "defect enhancement" sub-module. Finally, the thermal field compensation reference feature in the third stage is output. This feature has accurately stripped the external interference and the influence of normal operating conditions, and focuses on the thermal feature representation of internal defects (such as the thermal anomaly in the inter-turn short circuit area presents a unique high-dimensional activation pattern in the feature). Through the staged action of the feature compensation operators in each first feature compensation component, the server gradually purifies the effective information of the thermally radiated embedded feature, so that the thermal field compensation reference features in each stage not only retain the global integrity of the equipment thermal state, but also strengthen the local specificity of key defects, providing a high-confidence thermal field dimension support for subsequent acoustic-thermal coupling.

[0025] In the embodiment of the present invention, the multi-stage ultrasonic feature compensation processing of the ultrasonic embedded feature can be implemented through the following examples.

[0026] Through the trained feature coupling unit, the multi-stage ultrasonic feature compensation processing is sequentially performed on the ultrasonic embedded feature; Among them, the feature coupling unit includes a plurality of cascaded second feature compensation components, and a cross-correlation feature alignment component is arranged between every two second feature compensation components with a superior-subordinate relationship; the number of the second feature compensation components is the same as the number of the first feature compensation components in the acoustic-thermal feature calibrator.

[0027] In an embodiment of the present invention, exemplarily, when performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding features of the 110 kV transformer winding in a substation, the server calls the trained feature coupling unit (this unit includes 3 cascaded second feature compensation components, and 1 cross-correlation feature alignment component is set between every two upper and lower second feature compensation components, and the number of components is the same as that of the first feature compensation components of the acoustic-thermal feature calibrator), and realizes the deep coupling and compensation of ultrasonic features and thermal field reference features by stages. First, perform the first-stage ultrasonic feature compensation: The server inputs the 512-dimensional ultrasonic embedding features encoding ultrasonic reflection structure information (covering information such as insulation layer interface reflection and multiple reflections inside the winding, but mixed with equipment vibration and environmental electromagnetic noise interference) into the first second feature compensation component of the feature coupling unit. This component is built with a feature compensation operator of the "convolution-batch normalization-ReLU" combination to perform basic structure feature extraction on the ultrasonic embedding features: capture the amplitude mutation feature of the reflection wave at the "insulation layer-core iron" interface in the ultrasonic A-scan data through a 3×3 convolution kernel (such as increasing the feature activation value corresponding to the reflection wave peak dimension from 0.3 to 0.6), batch normalize to stabilize the feature distribution to reduce noise interference, and ReLU enhance the non-linear expression to highlight weak reflection signals, and finally output the first-stage ultrasonic output features. Subsequently, this feature flows into the first cross-correlation feature alignment component and is coupled with the thermal field compensation reference feature output in the first stage by the acoustic-thermal feature calibrator (the theoretical thermal distribution of the winding with environmental wind disturbance corrected): The component first adds the two types of features element by element in the energy dimension to generate a combined feature, and then performs peak compression (retaining the high-energy features of the wave peaks, strengthening the feature response of the key reflection wave from 0.7 to 0.9) and mean compression (smoothing the background noise, reducing the feature activation of random noise from 0.2 to 0.1) on the combined feature respectively, calculates the acoustic-thermal coupling modulation coefficient through a 1×1 convolution (such as 0.7, representing the modulation intensity of the thermal field on the ultrasonic features), and finally adjusts the combined feature based on this coefficient to generate the first-stage feature coupling output. When entering the second-stage ultrasonic feature compensation, the server synchronously inputs the ultrasonic embedding features and the first-stage feature coupling output into the second second feature compensation component. The feature compensation operator of this component incorporates the "depthwise separable convolution + attention fusion" mechanism: The depthwise separable convolution extracts the ultrasonic original structure information (such as the fine reflection features between winding turns) and the spatially calibrated information after thermal field calibration (such as the weight of the ultrasonic reflection features in the area corresponding to the hot spot position is increased from 0.4 to 0.7) respectively, and the attention fusion module strengthens the spatial correlation of "thermal anomaly position - ultrasonic reflection anomaly" (such as the overall activation value of the ultrasonic features within 3 mm around the hot spot is increased by 0.2), and outputs the second-stage ultrasonic output features.This feature then enters the second cross-correlation feature alignment component and repeats the coupling process with the thermal field compensation reference feature (the winding thermal distribution with corrected lens distortion) output by the second stage of the acoustic-thermal feature calibrator: Based on the heat conduction law of the radiator fins in the thermal field (such as the temperature decay at the fin ends corresponding to the energy decay of the ultrasonic reflection wave), adjust the dimensional activation weights of the ultrasonic features (optimize the feature activation of the ultrasonic reflection area corresponding to the fin ends from 0.6 to 0.4 that matches the theoretical decay model) to generate the feature coupling output of the second stage. Finally, perform the third stage of ultrasonic feature compensation: The server inputs the ultrasonic embedded feature and the feature coupling output of the second stage into the third second feature compensation component. The feature compensation operator of this component calls the "load-ultrasonic propagation model" (trained based on the historical correlation between the transformer load and ultrasonic attenuation). The operator accesses the real-time load data of the transformer (such as the current load rate of 90%), first calculates the ultrasonic propagation attenuation caused by overloading (for every 10% overload of the load, the ultrasonic energy theoretically decays by 5%); then compares the actual ultrasonic attenuation deviation in the features: If the ultrasonic feature activation value in a certain winding area is 8% lower than the theoretical load attenuation, it is determined as an internal defect (such as an inter-turn short circuit causing enhanced ultrasonic reflection), and the feature activation value of this area is increased from 0.5 to 0.8 through the "defect feature enhancement" sub-module to output the ultrasonic output feature of the third stage. This feature flows into the third cross-correlation feature alignment component and completes the coupling with the thermal field compensation reference feature (the defect thermal distribution with load interference removed) output by the third stage of the acoustic-thermal feature calibrator: The component performs weighted modulation on the combined features according to the temperature gradient in the thermal field defect area (such as a hot spot temperature of 130°C corresponding to an ultrasonic reflection enhancement coefficient of 1.2), and finally obtains the acoustic-thermal coupling feature after multi-stage ultrasonic feature compensation. This feature deeply correlates the thermal field defect position and the ultrasonic reflection abnormal pattern in the high-dimensional space (such as the feature dimensions of the thermal anomaly area and the feature activation values of the ultrasonic defect reflection synchronously reach 0.9). Through the coordinated operation of cascading the second feature compensation component and the cross-correlation feature alignment component within the feature coupling unit, the server gradually strengthens the cross-modal correlation between the ultrasonic embedded feature and the thermal field compensation reference feature, enabling each stage of ultrasonic feature compensation to retain both the original ultrasonic structure information and incorporate the spatial prior after precise thermal field calibration, providing high-confidence multi-modal feature support for the final acoustic-thermal coupling imaging.

[0028] In the embodiment of the present invention, for the trained feature coupling unit to perform multi-stage ultrasonic feature compensation processing on the ultrasonic embedded feature in sequence, the following example can be executed for implementation.

[0029] When performing the first stage of ultrasonic feature compensation processing on the ultrasonic embedded feature, the first second feature compensation component among the multiple second feature compensation components is used to perform the first stage of ultrasonic feature compensation processing on the ultrasonic embedded feature to obtain the first stage output feature after the first stage of ultrasonic feature compensation processing of the ultrasonic wave reflection data; After performing target-stage ultrasonic feature compensation processing on the ultrasonic embedded features, through the target cross-correlation feature alignment component in the feature coupling unit, perform feature coupling on the target-stage output features after the target-stage ultrasonic feature compensation processing and the thermal field compensation reference features generated by the target first feature compensation component in the acoustic-thermal feature calibrator to obtain target feature coupling outputs; When performing target-stage ultrasonic feature compensation processing on the ultrasonic embedded features, through the target second feature compensation component among the multiple second feature compensation components, perform target-stage ultrasonic feature compensation processing on the ultrasonic embedded features and the pre-order feature coupling outputs generated by the pre-order cross-correlation feature alignment component to obtain the target-stage output features of the ultrasonic reflection data after the target-stage ultrasonic feature compensation processing; Use the output features obtained after the last-stage ultrasonic feature compensation processing of the multi-stage ultrasonic feature compensation processing as the acoustic-thermal coupling features obtained after the multi-stage ultrasonic feature compensation processing.

[0030] In an embodiment of the present invention, exemplarily, when performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding features of the 110 kV transformer winding in a substation, the server relies on the trained feature coupling unit (including 3 cascaded second feature compensation components and 3 cross-correlation feature alignment components), and realizes the deep fusion of ultrasonic features and thermal field reference features according to the progressive logic of "stage compensation - coupling - re-compensation". First, perform the first-stage ultrasonic feature compensation: The server inputs the 512-dimensional ultrasonic embedding feature encoding the original information of the winding ultrasonic reflection (covering structural information such as insulation layer interface reflection and multiple reflections inside the winding, mixed with noise interference caused by equipment vibration) into the first second feature compensation component of the feature coupling unit. This component calls the feature compensation operator composed of "3×3 convolution + batch normalization + ReLU" to perform basic structure extraction on the ultrasonic embedding feature: The 3×3 convolution kernel captures the amplitude mutation feature of the reflection wave at the "insulation layer - iron core" interface (such as strengthening the feature activation value of the dimension corresponding to the reflection wave peak from 0.3 to 0.6), batch normalization suppresses the feature fluctuation caused by environmental electromagnetic noise (reducing the feature activation value in the noise area from 0.2 to 0.1), and ReLU enhances the non-linear expression of weak reflection signals, and finally outputs the first-stage output feature, which has initially highlighted the key structural information of the ultrasonic reflection (such as the feature dimension activation degree of the insulation layer interface is increased by 30%). When entering the second-stage ultrasonic feature compensation (example of the target stage), the process progresses in two steps: First, coupling after target stage compensation: The server first completes the "target stage ultrasonic feature compensation processing" through the second second feature compensation component and outputs the second-stage output feature (fusing the ultrasonic original structure and the thermal field prior generated by the first-stage coupling output); then, this feature flows into the target cross-correlation feature alignment component (i.e., the second cross-correlation component) and is coupled with the second-stage thermal field compensation reference feature generated by the second first feature compensation component in the acoustic-thermal feature calibrator (the thermal distribution of the winding with corrected infrared lens distortion): After the component merges the two types of features element by element in the energy dimension, based on the heat conduction law of the radiator fins (such as the temperature decay at the end of the fins corresponding to the ultrasonic reflection energy decay coefficient of 0.8), it adjusts the activation weight of the ultrasonic feature dimension (optimizing the feature activation value in the ultrasonic reflection area corresponding to the end of the fins from 0.6 to 0.4 matching the theoretical decay) to generate the second-stage feature coupling output. Second, processing during target stage compensation: When performing the second-stage ultrasonic feature compensation, the server calls the second second feature compensation component and synchronously inputs the "ultrasonic embedding feature" and the first-stage feature coupling output (generated by the previous cross-correlation component).The feature compensation operator of this component incorporates a "depthwise separable convolution + spatial attention fusion" mechanism: The depthwise separable convolution extracts the original ultrasonic structure (such as the subtle reflection features between winding turns, with the activation value increased from 0.4 to 0.6) and the spatial information after thermal field calibration (such as the ultrasonic feature weight in the area corresponding to the hot spot position strengthened from 0.3 to 0.7) respectively. The spatial attention fusion module strengthens the spatial correlation between "thermal anomaly position - ultrasonic reflection anomaly" (such as the overall activation value of ultrasonic features within 3 mm around the hot spot increased by 0.2), and finally outputs the output features of the second stage. For the ultrasonic feature compensation in the third stage (the last stage of multiple stages), the server calls the third second feature compensation component and inputs the "ultrasonic embedded features" and the output of the coupling of the second stage features (generated by the previous cross-correlation component). The feature compensation operator of this component accesses the real-time load data of the transformer (such as the current load rate of 90%) and calls the "load - ultrasonic propagation attenuation model" (for every 10% increase in load, the ultrasonic energy theoretically attenuates by 5%): First, calculate the ultrasonic attenuation amount caused by overloaded load (the current load exceeds 10%, and the theoretical attenuation is 5%), and then compare the actual attenuation deviation in the features (if the activation value of ultrasonic features in a certain winding area is 8% lower than the theoretical attenuation, it is determined that the reflection enhancement is caused by inter-turn short circuit); Through the "defect feature enhancement" sub-module, the activation value of the features in this area is increased from 0.5 to 0.8, and the output features of the third stage are output. The server uses this feature as the acoustic-thermal coupling feature after multi-stage ultrasonic feature compensation. At this time, the feature deeply correlates the thermal field defect position (such as a hot spot at 130°C) and the ultrasonic reflection anomaly pattern (such as the activation value of the feature dimension of the extra peak reaching 0.9) in the high-dimensional space, providing accurate multi-modal correlation information for subsequent imaging reconstruction. Through the progressive logic of "stage compensation - coupling - re-compensation" within the feature coupling unit, the server enables each stage of ultrasonic features to retain both the original structural information and the spatial prior after precise thermal field calibration. The finally output acoustic-thermal coupling features achieve a deep collaboration between the thermal field positioning ability and the ultrasonic structure analysis ability.

[0031] In the embodiment of the present invention, the multiple second feature compensation components include: a second feature compensation component for performing dimensionality reduction processing on the ultrasonic embedded features, and a second feature compensation component for performing dimensionality increase processing on the ultrasonic embedded features; Among them, the number of the second feature compensation components for performing dimensionality reduction processing on the ultrasonic embedded features is the same as the number of the second feature compensation components for performing dimensionality increase processing on the ultrasonic embedded features; and, the second feature compensation component for performing dimensionality increase processing on the ultrasonic embedded features is arranged after the second feature compensation component for performing dimensionality reduction processing on the ultrasonic embedded features.

[0032] In an embodiment of the present invention, exemplarily, when performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding features of the 110 kV transformer winding in a substation, the feature coupling unit called by the server includes 2 second feature compensation components for dimensionality reduction and 2 second feature compensation components for dimensionality increase (the number of components in the two categories is the same, and the dimensionality increase components are sequentially arranged after the dimensionality reduction components). Through the feature flow control logic of "dimensionality reduction to focus on core information - dimensionality increase to restore spatial structure", precise purification and cross-modal fusion of ultrasonic information are achieved. First, the first dimensionality reduction second feature compensation component starts processing: The server inputs the 512-dimensional ultrasonic embedding features encoding the full-dimensional information of the winding ultrasonic reflection (including information such as insulation layer interface reflection and multiple reflections inside the winding, mixed with environmental electromagnetic noise and equipment vibration interference) into this component. The component is built with a dimensionality reduction operator composed of "1×1 convolution + max pooling". The 1×1 convolution kernel compresses the feature dimension from 512 to 256 dimensions (preferably retaining core structural information such as the "insulation layer - iron core" interface reflection and the basic reflection between winding turns, increasing the activation ratio of the feature dimension corresponding to the key reflection wave from 30% to 50%); the max pooling layer focuses on the region of strong reflection signals (reducing the activation value of the weak feature dimension dominated by noise from 0.2 to 0.1), and finally outputs 256-dimensional dimensionality reduction features. This process eliminates redundant noise dimensions through dimensionality contraction and strengthens the basic structural features of ultrasonic reflection (such as increasing the activation degree of the feature dimension of the normal reflection between winding turns by 20%). Next, the second dimensionality reduction second feature compensation component takes over the processing: Taking the 256-dimensional dimensionality reduction features as input, the component calls a dimensionality reduction operator composed of "depthwise separable convolution + channel attention" to further compress the feature dimension to 128 dimensions. The depthwise separable convolution extracts the fine-grained features of ultrasonic reflection by channel (such as the feature of the weak additional reflection wave caused by inter-turn short circuit, whose activation value is strengthened from 0.3 to 0.6); the channel attention module, based on the prior information of the thermal field compensation reference feature (such as the hot spot position output by the acoustic-thermal feature calibrator), strengthens the feature dimension spatially associated with the thermal field (increasing the ultrasonic feature weight in the region corresponding to the hot spot from 0.4 to 0.7), and outputs 128-dimensional deeply reduced features. At this time, the features focus on the core dimension of "thermal field association + defect sensitivity", laying a precise foundation for subsequent dimensionality increase and fusion. Entering the first dimensionality increase second feature compensation component: The server inputs the 128-dimensional deeply reduced features into this component. The component restores the feature dimension from 128 dimensions to 256 dimensions through a dimensionality increase operator composed of "transposed convolution + residual connection". The transposed convolution gradually restores the spatial distribution information of ultrasonic reflection (such as reconstructing the spatial position feature of the insulation layer interface reflection, reducing the spatial coordinate error of the feature dimension to ±1 pixel); the residual connection retains the defect-sensitive features purified in the dimensionality reduction stage (such as the feature activation value corresponding to the inter-turn short circuit reflection remaining stable at 0.65 from 0.6), and outputs 256-dimensional dimensionality increase features. This feature not only restores the mesoscale spatial structure of ultrasonic reflection but also continues the strengthening effect on defect information in the dimensionality reduction stage.Finally, the second dimensionality-raising second feature compensation component completes the finishing process: taking the 256-dimensional dimensionality-raising feature as the input, the component calls the dimensionality-raising operator composed of "transposed convolution + spatial attention fusion" to restore the feature dimension to 512 dimensions. The transposed convolution precisely restores the original dimensionality scale of the ultrasonic embedding feature (the matching degree of the global energy distribution of the overall ultrasonic reflection of the winding with the original embedding feature reaches 95%); the spatial attention fusion module deeply binds the thermal field compensation reference feature output by the acoustic-thermal feature calibrator (for example, the activation value corresponding to the hot spot area in the ultrasonic feature is increased from 0.5 to 0.8), and finally outputs the 512-dimensional dimensionality-raising feature. As a key link in the multi-stage ultrasonic feature compensation, this feature not only retains the full-dimensional structural information of the ultrasonic reflection but also deeply integrates the spatially correlated information calibrated by the thermal field, providing high-resolution and strongly correlated ultrasonic feature support for subsequent acoustic-thermal coupling and imaging reconstruction. Through the "noise elimination + core focusing" of the dimensionality-lowering component and the "structural restoration + correlation strengthening" of the dimensionality-raising component, the server enables the ultrasonic embedding feature to complete the progressive processing of "impurity filtering - defect purification - spatial reconstruction - cross-modal fusion" during the process of dimensionality contraction and expansion, ensuring that the ultrasonic feature at each stage can accurately respond to the internal defects of the device and deeply cooperate with the thermal field information, laying a technical foundation at the feature level for the accuracy and reliability of acoustic-thermal coupling imaging.

[0033] In the embodiment of the present invention, before performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedding feature, the embodiment of the present invention also provides the following implementation manners.

[0034] Generate the simulation perturbation feature of the feature coupling unit through a preset acoustic-thermal perturbation simulation module; Load the simulation perturbation feature and the ultrasonic embedding feature as the current processing features into the feature coupling unit; The first-stage ultrasonic feature compensation processing of the ultrasonic embedding feature by the first second feature compensation component among the multiple second feature compensation components to obtain the first-stage output feature after the first-stage ultrasonic feature compensation processing of the ultrasonic wave reflection data includes: Perform the first-stage ultrasonic feature compensation processing on the ultrasonic embedding feature and the simulation perturbation feature through the first second feature compensation component to obtain the first-stage output feature.

[0035] In an embodiment of the present invention, exemplarily, before performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedding features of the 110 kV transformer winding in a substation, the server first calls a preset acoustic-thermal disturbance simulation module (constructed based on the transformer digital twin model and integrated with disturbance models such as vibration, electromagnetic interference, and environmental temperature change) to generate simulation disturbance features. This module simulates "the phase shift of ultrasonic signals caused by equipment vibration" (vibration frequency of 50 Hz, causing the phase of the reflected wave peak in the ultrasonic A-scan data to shift by ±5 μs), "the ultrasonic amplitude noise caused by environmental electromagnetic interference" (the noise intensity fluctuates by ±0.1 corresponding to the activation value of the feature dimension), and "the change in the thermal field gradient caused by a sudden 5°C drop in the environmental temperature" (the activation value of the temperature feature in the radiator area in the thermal image decreases by 0.2). These interference patterns are encoded into simulation disturbance features with the same dimension (512 dimensions) as the ultrasonic embedding features, accurately reproducing the characteristic patterns of complex on-site interferences. Subsequently, the server combines this simulation disturbance feature with the 512-dimensional ultrasonic embedding feature (including real structure information such as insulation layer interface reflection and internal winding reflection) collected from the transformer ultrasonic transducer and obtained through feature mapping into the "current processing feature", and synchronously loads it into the feature coupling unit. At this time, the current processing feature not only carries the original ultrasonic structure information but also incorporates the interference patterns of the simulation disturbance, simulating the ultrasonic data state in a complex on-site environment. Entering the first-stage ultrasonic feature compensation link, the first second feature compensation component of the feature coupling unit (with an "adversarial training convolution + attention de-noising" operator built-in) receives the "current processing feature" and performs compensation processing on the ultrasonic embedding feature and the simulation disturbance feature in a coordinated manner: the adversarial training convolutional layer first learns the pattern of the disturbance feature (such as the periodic fluctuation of the feature dimension corresponding to the vibration phase shift), and then cancels this disturbance through reverse weight adjustment (correcting the feature activation value deviation caused by the phase shift from ±0.2 to ±0.05); the attention de-noising module focuses on the original ultrasonic structure features (such as enhancing the activation value of the feature dimension of the insulation layer interface reflection from 0.4 to 0.7, and at the same time suppressing the activation value of the feature dimension corresponding to electromagnetic noise from 0.3 to 0.1). After this processing, the component outputs the first-stage output feature, which has initially stripped the simulation disturbance interference (the vibration phase shift correction rate reaches 90%, and the electromagnetic noise suppression rate reaches 70%), and retains the core structure information of the ultrasonic reflection (the activation degree of the feature dimension of the normal reflection between winding turns increases by 25%). By introducing the simulation disturbance feature before the first-stage compensation and allowing the components to process in a coordinated manner, the server simulates a strong interference environment on-site, enabling the feature coupling unit to learn the dual capabilities of "interference resistance + feature purification", ensuring that the output first-stage feature is both resistant to real disturbances and accurately retains the ultrasonic structure information, laying a robust foundation for subsequent multi-stage compensation and acoustic-thermal coupling.

[0036] In an embodiment of the present invention, the feature dimension of the target stage output feature is the same as the feature dimension of the thermal field compensation reference feature generated by the target first feature compensation component; The feature coupling of the target-stage output feature after the target-stage ultrasonic feature compensation process and the thermal field compensation reference feature generated by the target first feature compensation component in the acoustic-thermal feature calibrator is performed through the target cross-correlation feature alignment component in the feature coupling unit, and the target feature coupling output can be implemented through the following examples.

[0037] Through the target cross-correlation feature alignment component in the feature coupling unit, perform feature merging processing on the target-stage output feature and the thermal field compensation reference feature generated by the target first feature compensation component in the energy dimension to obtain a merged feature; Perform feature compression processing on the merged feature to obtain a compressed feature; Perform a convolution operation on the compressed feature to obtain the acoustic-thermal coupling modulation coefficient between the target-stage output feature and the thermal field compensation reference feature generated by the target first feature compensation component; Based on the acoustic-thermal coupling modulation coefficient, perform feature coupling processing calculation on the merged feature to obtain the target feature coupling output.

[0038] In an embodiment of the present invention, exemplarily, when performing the second stage ultrasonic feature compensation processing on the 110kV transformer winding of the substation, the server ensures that the target stage output feature output by the feature coupling unit (ultrasonic feature processed by the second feature compensation component, with a dimension of 256 dimensions) is consistent with the feature dimension of the thermal field compensation reference feature (winding thermal distribution feature after correcting the lens distortion, with the same dimension of 256 dimensions) generated by the target first feature compensation component (second intermediate layer) in the acoustic thermal feature calibrator, and then performs feature coupling through the target cross-correlation feature alignment component: First, the energy dimension is merged: the server calls the target cross-correlation feature alignment component to add the 256-dimensional second-stage ultrasonic output feature (encoding structural information such as winding inter-turn reflection and insulation layer interface reflection, and incorporating the thermal field prior of the first-stage coupling output) and the 256-dimensional second-stage thermal field compensation reference feature (the thermal distribution distortion caused by the tilt of the infrared lens has been corrected, and the heat conduction law of the radiator fins has been accurately characterized) in the energy dimension. For example, the activation value of the feature dimension of the "winding hotspot corresponding area" in the ultrasonic feature is 0.7 (reflecting the energy intensity of the ultrasonic reflection wave in this area), and the activation value of this area in the thermal field feature is 0.8 (reflecting the energy weight of the temperature distribution of the thermal field). After adding, a merged feature is generated (the corresponding dimension activation value is 1.5, not normalized), realizing the preliminary fusion of acoustic and thermal energy information. Next, feature compression processing: the component synchronously performs "peak feature compression" and "mean feature compression" on the merged feature. Peak compression focuses on the strong correlation features between the ultrasonic reflection peak and the high temperature area of the thermal field: for example, the activation value of the dimension corresponding to the ultrasonic reflection peak is compressed from 1.5 to 0.9 (retaining the core information of high-energy reflection and suppressing weakly correlated noise); mean compression smoothes background interference: for example, the feature activation value of the non-hotspot area of the radiator is compressed from 0.3 to 0.2 (reducing false feature fluctuations caused by environmental wind disturbance, electromagnetic noise, etc.), and finally obtains a compressed feature, which not only strengthens the information of the strongly correlated area of acoustic and thermal, but also weakens the background interference, providing pure input for subsequent modulation coefficient calculation. Then, the convolution operation calculates the modulation coefficient: the component calls the 1×1 convolution kernel to perform convolution operation on the compressed features to explore the coupling law of acoustic and thermal features. Based on the heat conduction model of the radiator fins in the thermal field (for every 1°C attenuation of the temperature at the end of the fin, the ultrasonic reflection energy attenuates by 2%), the convolution layer learns the correlation pattern of "thermal field temperature gradient-ultrasonic energy attenuation" and outputs the acoustic and thermal coupling modulation coefficient (such as 0.8, which represents the modulation intensity of the thermal field on the ultrasonic feature). This coefficient quantifies the degree of influence of the spatial distribution of the thermal field on the ultrasonic reflection feature: if the temperature in a certain area of the thermal field is 15°C higher than the normal operating conditions, the modulation coefficient will be increased by 0.15 accordingly, indicating that the ultrasonic feature needs to strengthen the reflection anomaly analysis of the area. Finally, the feature coupling processing calculation: the server performs weighted adjustment on the merged features based on the acoustic and thermal coupling modulation coefficient.Taking the corresponding dimension of "winding hot spot - enhanced ultrasonic reflection" in the merged feature as an example, the activation value of the merged feature is 1.5. After multiplying by the modulation coefficient 0.8, it gets 1.2, which not only retains the original energy correlation between sound and heat in this dimension, but also enhances the defect-sensitive features according to the prior information of the thermal field. After weighted by all dimensions, the component outputs the target feature coupling output. In this 256-dimensional space, prior information such as the temperature distribution and spatial position of the thermal field has been deeply integrated into the ultrasonic reflection feature. For example, the activation value of the ultrasonic feature in the hot spot area has increased from 0.7 to 0.9 (corresponding to enhanced ultrasonic reflection, indicating internal defects), providing a precise input with both sound-heat correlation and defect sensitivity for the next-stage ultrasonic feature compensation. Through the progressive process of "energy merging - feature compression - coefficient calculation - weighted coupling", the server enables the ultrasonic feature and the thermal field feature in the target stage to achieve precise alignment and information fusion in the high-dimensional space, not only retaining the structural details of the ultrasonic reflection, but also injecting the spatial and physical priors of the thermal field, laying a solid technical foundation for the cross-modal correlation accuracy of multimodal imaging.

[0039] In the embodiment of the present invention, the feature compression process for the merged feature to obtain the compressed feature can be implemented through the following examples.

[0040] Perform peak feature compression processing and mean feature compression processing on the merged feature respectively to obtain the peak compression feature and the mean compression feature correspondingly; Use the peak compression feature and the mean compression feature as the compressed feature.

[0041] In an embodiment of the present invention, exemplarily, when performing feature compression processing on the second-stage combined features of the 110 kV transformer winding in a substation, the server calls the dual-channel compression module in the target cross-correlation feature alignment component to simultaneously perform peak feature compression and mean feature compression: First, peak feature compression processing: The component starts a threshold-constrained compression operator for the strongly correlated dimensions (such as the feature dimension corresponding to the ultrasonic peak at the winding hot spot) of the ultrasonic reflection peaks and the high-temperature regions of the thermal field in the combined features (256-dimensional, integrating ultrasonic reflection structure and thermal field distribution energy information). Taking the 100th dimension in the combined features as an example (corresponding to the superposition of the ultrasonic peak and the high-temperature thermal field in the winding hot spot area), its original activation value is 1.5 (ultrasonic peak energy contribution of 0.7 and high-temperature thermal field energy contribution of 0.8); the peak compression operator sets an activation threshold of 0.8 and compresses the part exceeding the threshold by a ratio of 0.6 (1.5×0.6 = 0.9), which not only retains the core energy information in the strongly correlated area but also suppresses weakly correlated noise dimensions (such as false peaks caused by environmental electromagnetic interference, whose activation value is reduced from 0.4 to 0.2). Finally, the peak compression feature is output. In this feature, the activation degree of the strongly correlated dimension between the winding hot spot and the ultrasonic peak focuses on the range of 0.7 - 1.0, and the defect-sensitive information is precisely enhanced. Next, mean feature compression processing: The component starts a local mean pooling compression operator for the background dimensions such as non-hot spots of the radiator and normal areas of the winding (such as the feature dimension of the non-terminal of the radiator fin). Taking the 200th dimension in the combined features as an example (corresponding to the thermal field and ultrasonic reflection in the non-hot spot area of the radiator), its original activation value fluctuates between 0.2 - 0.4 (thermal field temperature fluctuations caused by environmental wind disturbances and ultrasonic amplitude jitters caused by electromagnetic noise); the mean compression operator calls a 3×3 local mean pooling kernel to calculate the mean of this dimension and adjacent dimensions ((0.2 + 0.3 + 0.4) / 3 = 0.3), and uniformly corrects the activation value of the fluctuating dimension to 0.3, smoothing the feature fluctuations caused by background interference, and outputting the mean compression feature. In this feature, the feature activation value in the background area stabilizes in the range of 0.2 - 0.4, and the false features caused by environmental interference are effectively suppressed. Finally, the server splices the peak compression feature and the mean compression feature by dimension to generate a compression feature that combines strong correlation information focusing and background interference smoothing. In this feature, the activation value of the dimension corresponding to the ultrasonic peak at the winding hot spot remains 0.9 (after peak compression), and the activation value of the dimension in the non-hot spot area of the radiator stabilizes at 0.3 (after mean compression), providing a pure and hierarchical feature input for the precise calculation of the subsequent acoustic-thermal coupling modulation coefficient, ensuring that the correlation law between the thermal field temperature gradient and the ultrasonic energy attenuation can be efficiently learned by the convolutional layer. Through the dual-channel compression mechanism, the server enables the combined features to retain defect-sensitive information while stripping background noise, laying a pure foundation at the feature level for the accuracy of acoustic-thermal coupling.

[0042] In an embodiment of the present invention, the feature representation learning is respectively performed on the thermal radiation data and the ultrasonic reflection data, and a thermal radiation embedding feature and an ultrasonic embedding feature are correspondingly obtained, which can be implemented through the following examples.

[0043] Call an ultrasonic feature mapper to perform feature mapping processing on the ultrasonic reflection data to obtain the ultrasonic embedding feature; Call a thermal feature mapper to perform feature mapping processing on the thermal radiation data to obtain the thermal radiation embedding feature.

[0044] In an embodiment of the present invention, exemplarily, in the state monitoring scenario of a 110 kV transformer in a substation, the server performs feature representation learning on ultrasonic reflection data and thermal radiation data respectively by invoking a dedicated feature mapper to generate high-dimensional embedded features: When processing ultrasonic reflection data, the server invokes an ultrasonic feature mapper (composed of a cascaded 1D convolutional layer and a gated recurrent unit (GRU)). For the ultrasonic A-scan data of the transformer winding (center frequency 5 MHz, sampling rate 20 MHz, duration 50 μs, sequence length 10,000 points, including information such as insulation layer-core interface reflection and winding internal defect reflection), the 1D convolutional layer first extracts time-domain features such as "amplitude attenuation rate, peak time interval" in segments with 32 1×16 convolutional kernels. For example, the activation value of the feature dimension corresponding to the peak of the insulation layer interface reflection is enhanced from 0.2 to 0.5, accurately capturing the interface position; the GRU layer mines long-range dependencies along the time dimension to identify the correlation of multiple reflection waves caused by turn-to-turn short circuits (increasing the activation degree of the feature dimension corresponding to the defect reflection wave sequence by 0.3); finally, the fully connected layer compresses the feature dimension to 512 dimensions and outputs the ultrasonic embedded feature. This feature encodes core information such as the energy loss of ultrasonic wave propagation (such as energy attenuation caused by insulation aging, with the activation value of the corresponding dimension decreasing by 0.2), the number of reflection interfaces (activation value 0.6 when there are 3 normal interfaces, and the activation value increasing to 0.8 when a defect adds an interface), and the waveform distortion caused by defects (wave peak broadening due to short circuit, with the activation value of the corresponding dimension changing from 0.4 to 0.7), providing a structured representation of the ultrasonic dimension for subsequent cross-modal coupling. When processing thermal radiation data, the server invokes a thermal feature mapper (based on an improved ResNet architecture, including 4 convolutional blocks and 2 fully connected layers). For the infrared thermal image of the transformer winding (resolution 1024×768, pixel gray scale mapping temperature from -40°C to 500°C, covering the thermal distribution of the winding, core, and radiator), the first convolutional block extracts local thermal patterns such as "winding-core temperature boundary, radiator fin thermal gradient" with 3×3 convolutional kernels. For example, the activation value of the feature dimension corresponding to the pixels in the hot spot area of the winding is increased from 0.3 to 0.6, highlighting the temperature anomaly; subsequent convolutional blocks strengthen the features of the thermal anomaly area through residual connections (such as the feature response of the turn-to-turn short circuit hot spot increasing from 0.5 to 0.85 while retaining the basic information of the normal thermal distribution of the core); finally, the fully connected layer compresses the feature dimension to 512 dimensions and outputs the thermal radiation embedded feature. This feature encodes key information such as the temperature distribution of the thermal field (activation value difference of 0.1 for a 2°C temperature difference between the three phases of the winding), the direction of the thermal gradient (activation value of 0.7 for longitudinal heat conduction of the radiator), and the spatial position of the thermal anomaly (activation value of 0.9 for the hot spot coordinates (300, 200)), providing an accurate representation of the thermal field dimension for subsequent thermal radiation feature compensation.Through hierarchical feature extraction by a dedicated feature mapper, the server abstracts the original multi-dimensional data of ultrasound and thermal radiation into compact features in a high-dimensional embedding space, which not only retains the core information of the device state (such as defect location, energy loss), but also provides a feature input of a unified dimension (512 dimensions) for subsequent multi-stage compensation and cross-modal coupling, ensuring the interactivity and collaborative analysis basis of the acoustic and thermal modalities at the feature level.

[0045] In the embodiment of the present invention, the thermal field compensation reference feature includes a thermal radiation feature representation for indicating the thermal field distribution feature of the target device area; the thermal field distribution feature includes: the position of the thermal anomaly area, the shape of the thermal anomaly, and the contour of the device area.

[0046] In an embodiment of the present invention, exemplarily, in the multi-stage thermal radiation feature compensation process of the 110 kV transformer winding in a substation, the server accurately carries the thermal radiation feature representation of the target device area through the thermal field compensation reference features output by the acoustic-thermal feature calibrator, and this representation focuses on the quantitative expressions of three types of thermal field distribution features: the position of the thermal anomaly area, the shape of the thermal anomaly, and the contour of the device area. For the thermal field compensation reference features in the first stage (after excluding environmental wind disturbances): When the server processes the thermal radiation embedded features, the first first feature compensation component of the acoustic-thermal feature calibrator enhances the hot spot features in the upper part of phase A of the winding through spatial attention (the temperature in the infrared thermal image of this area reaches 120 °C, exceeding the normal threshold by 30 °C), so that the activation value of the corresponding dimension of "the position of the thermal anomaly area" in the thermal radiation feature representation reaches 0.9, and accurately locates the pixel coordinates (200, 150) of the hot spot in the thermal image coordinate system; the "shape of the thermal anomaly" dimension is encoded as a circular feature with a diameter of about 10 pixels (corresponding to a thermal aggregation area with a diameter of 3 cm in the actual space), and the activation value is 0.85; the "contour of the device area" dimension clearly distinguishes the boundaries of the winding (pixel activation value 0.8), the iron core (0.6), and the radiator (0.5). Through the spatial weight allocation of the feature dimensions, the thermal distribution contours of each component of the device are restored. The features in this stage have initially stripped the interference of environmental wind disturbances on the heat exchange of the radiator, making the thermal field distribution features closer to the intrinsic thermal state of the device. Entering the thermal field compensation reference features in the second stage (after correcting the infrared lens distortion): The server calls the second first feature compensation component of the acoustic-thermal feature calibrator and combines the prior knowledge of the transformer heat dissipation structure (the arrangement pitch of the radiator fins is 2 cm, and the thermal conductivity is 200 W / (m·K)) to correct the thermal distribution distortion caused by the lens tilt in the features of the previous stage. In the thermal radiation feature representation, the "position of the thermal anomaly area" is corrected to more accurate pixel coordinates (205, 152) (the position error is reduced from ±5 pixels to ±2 pixels); due to the thermal conduction constraint of the heat dissipation structure, the "shape of the thermal anomaly" evolves from a circle to an ellipse with a major axis of 12 pixels and a minor axis of 8 pixels (corresponding to the physical law that the heat spreads longitudinally along the fins and is restricted transversely), and the activation value is adjusted to 0.82; the detailed features of the radiator fins in the "contour of the device area" are clearer, and the pixel activation value at the fin gap is increased from 0.3 to 0.4, restoring the true gradient of the heat exchange between the fins. The features in this stage have been deeply matched with the thermal characteristics of the device physical structure. To the thermal field compensation reference features in the third stage (after integrating the load curve): The server calls the third first feature compensation component of the acoustic-thermal feature calibrator, accesses the real-time load data of the transformer (the load rate is 90%, exceeding the normal upper limit by 10%), and distinguishes "load overheating" from "defect overheating" through the load-heat dissipation model.In the characterization of thermal radiation characteristics, after the load temperature rise correction, the "thermal anomaly area position" confirmed that the hot spot was caused by the inter-turn short circuit (rather than excessive load), the position dimension activation value remained at 0.9, and the coordinates were stable at (205,152); the "thermal anomaly morphology" evolved into an irregular polygon due to the heat diffusion of internal defects (arc heating at the short-circuit point caused thermal decomposition of the surrounding insulation layer), with an activation value of 0.88; the "equipment area contour" dimension combined with the overall heat distribution under load, the thermal gradient characteristics of the three phases of the winding are more accurate, and the temperature difference between phase A (hot spot phase) and phases B and C corresponds to the activation value difference of the feature dimension reduced from 0.3 to 0.2, clearly presenting the overall picture of the thermal field under the joint action of load and defects. At this stage, the characteristics have restored the thermal field anomalies caused by internal defects of the equipment to the greatest extent, providing a high-confidence thermal distribution prior for subsequent acoustic-thermal coupling. Through multi-stage thermal radiation feature compensation, the thermal field compensation benchmark features output by the server are based on the thermal radiation feature representation, and layered analysis is performed on the dynamic changes in the position, morphology and contour of the thermal anomaly area and the equipment area. This not only supports the spatial correlation of ultrasonic features, but also provides a core basis for the thermal field dimensional accuracy of the final imaging.

[0047] In an embodiment of the present invention, the method is implemented by a pre-trained multimodal imaging model; the multimodal imaging model includes a core imaging unit and a thermal feature calibration unit; the core imaging unit includes at least an ultrasonic feature mapper and a feature coupling unit, and the thermal feature calibration unit includes a thermal feature mapper and an acoustic thermal feature calibrator; The multimodal imaging model is obtained by the following method and can be implemented by the following example.

[0048] Loading a thermal radiation data instance in a preset training set into the thermal feature calibration unit, performing feature mapping processing on the thermal radiation data instance through a thermal feature mapper in the thermal feature calibration unit, and obtaining a thermal radiation embedding feature of the thermal radiation data instance; The thermal radiation embedding feature of the thermal radiation data instance is subjected to multi-stage thermal radiation feature compensation processing by the acoustic thermal feature calibrator in the thermal feature calibration unit, so as to obtain thermal field compensation reference features of the thermal radiation data instance of the target device area instance in the thermal radiation data instance at multiple feature compensation stages; Loading an ultrasonic reflection data instance in a preset training set into the core imaging unit, performing feature mapping processing on the ultrasonic reflection data instance through an ultrasonic feature mapper in the core imaging unit to obtain a sample ultrasonic embedding feature; The characteristic coupling unit in the core imaging unit sequentially performs multi-stage ultrasonic characteristic compensation processing on the ultrasonic embedding characteristic of the sample to obtain an acoustic-thermal coupling characteristic instance; Based on the thermal field compensation reference features of the thermal radiation data examples and the acoustic-thermal coupling feature examples, calculate the error of the multimodal imaging model to obtain the error value of the multimodal imaging model; Based on the error value, optimize the architecture parameters in the multimodal imaging model to obtain the trained multimodal imaging model.

[0049] In an embodiment of the present invention, exemplarily, when training a multi-modal imaging model, the server constructs a preset training set with the historical monitoring data of a 110 kV transformer in a substation (including infrared thermal images labeled with thermal anomalies, ultrasonic A-scan data labeled with defects, and true fault labels), and completes the model training according to the logic of "thermal feature calibration - ultrasonic feature mapping - cross-modal coupling - error tuning": The server first loads an instance of the thermal radiation data of a certain transformer winding in the training set (such as an infrared thermal image with a resolution of 1024×768, labeled with the hot spot position (200, 150) and temperature 120°C of phase A winding) into the thermal feature calibration unit. The thermal feature mapper in the thermal feature calibration unit (an improved ResNet architecture, including 4 convolutional blocks and 2 fully connected layers) performs feature mapping on the thermal image: The first convolutional block extracts the temperature boundary between the "winding - iron core" with a 3×3 convolutional kernel (the activation value of the feature dimension corresponding to the pixels in the hot spot area is strengthened from 0.3 to 0.6), and the subsequent residual blocks retain the global distribution of the thermal field through cross-layer connection and strengthen the thermal anomaly features (the feature response of the turn-to-turn short circuit hot spot is increased from 0.5 to 0.85), and finally outputs a 512-dimensional thermal radiation embedding feature of the thermal radiation data instance (encoding core information such as the spatial position of the hot spot, temperature gradient, and equipment component contour). Then, the acoustic-thermal feature calibrator (consisting of 3 cascaded first feature compensation components, each component containing a residual module and a spatial attention mechanism) performs multi-stage compensation on this embedding feature: The first-stage component corrects the environmental wind disturbance through the spatial attention mechanism (false high-temperature pixels caused by air flow in the radiator, and its feature activation value is suppressed from 0.6 to 0.2), and outputs the first-stage thermal field compensation reference feature; The second-stage component combines the prior of the transformer heat dissipation structure (such as the arrangement pitch of the radiator fins is 2 cm and the thermal conductivity of the aluminum fins is 200 W / (m·K)), corrects the thermal distribution distortion caused by the tilt of the infrared lens (the hot spot coordinates are accurately corrected from (200, 150) to (205, 152), and the position error is reduced to ±2 pixels), and outputs the second-stage thermal field compensation reference feature; The third-stage component accesses the real-time load curve of the transformer (the load rate in the training set is 90%), distinguishes between "normal heating due to excessive load" and "abnormal heating due to internal defects" (it is confirmed through the load - heat dissipation model that the hot spot is caused by turn-to-turn short circuit, and the feature activation value is stably maintained at 0.9), and outputs the third-stage thermal field compensation reference feature. The features of the three stages gradually strip off the interference of the environment, equipment structure, and working conditions, and accurately restore the intrinsic thermal state of the equipment. Synchronously, the server loads an instance of the ultrasonic reflection data at the corresponding moment in the training set (such as an ultrasonic A-scan sequence with a center frequency of 5 MHz, a sampling rate of 20 MHz, and a duration of 50 μs, labeled with the turn-to-turn short circuit defect position) into the core imaging unit.The ultrasonic feature mapper within the core imaging unit (formed by cascading a 1D convolutional layer and a gated recurrent unit (GRU)) performs feature mapping on the A-scan sequence: The 1D convolutional layer extracts time-domain features such as "amplitude of the reflected wave peak at the insulation layer-iron core interface and energy attenuation rate" in segments using 32 1×16 convolutional kernels (for the additional reflected wave peaks caused by defects, the activation values of the corresponding feature dimensions are strengthened from 0.3 to 0.6); the GRU layer mines the long-range dependencies of the reflected waves along the time dimension (for the sequence of multiple reflected waves caused by inter-turn short circuits, the activation degree of the feature dimension is increased by 0.3); finally, a 512-dimensional sample ultrasonic embedded feature is output, encoding core information such as ultrasonic reflection structure, defect location, and energy loss pattern. Subsequently, the feature coupling unit (formed by cascading 3 second feature compensation components and 3 cross-correlation feature alignment components) performs multi-stage compensation on the sample ultrasonic embedded feature: In each stage, the second feature compensation component receives "ultrasonic embedded feature + the coupled output of the previous cross-correlation component" and performs cross-modal coupling with the thermal field compensation reference feature of the corresponding stage (for example, the component in the second stage combines the thermal field compensation reference feature in the second stage to strengthen the spatial correlation between the "hot spot area - ultrasonic reflection enhancement area", and the activation value of the feature dimension is increased from 0.7 to 0.9), and finally outputs a thermo-acoustic coupled feature instance, which deeply fuses the thermal field spatial prior and ultrasonic structure information in the 512-dimensional space. For example, the activation value of the thermo-acoustic feature dimension in the inter-turn short circuit area reaches 0.9 synchronously, realizing the precise association between the defect location and the thermal anomaly. The server calculates the model error based on the thermal radiation data instance thermal field compensation reference feature and the thermo-acoustic coupled feature instance: On the one hand, it calculates the "thermal field consistency error" (the difference in thermal field representation between the thermal field compensation reference feature and the thermo-acoustic coupled feature, such as the mean absolute error (MAE) between the predicted hot spot temperature and the true annotated temperature); on the other hand, it calculates the "feature alignment error" (the deviation of the cross-correlation coefficient of the thermo-acoustic feature dimension, such as the difference between the Pearson correlation coefficient of the thermo-acoustic feature in the short circuit area and the theoretical correlation coefficient). The two types of errors are weighted and summed to obtain the total model error. Finally, the server uses the backpropagation algorithm to optimize architecture parameters such as the convolutional kernel weights of the thermal feature mapper, the spatial attention parameters of the thermo-acoustic feature calibrator, the GRU gating weights of the ultrasonic feature mapper, and the cross-correlation coefficients of the feature coupling unit based on the total error, and iterates until the error converges (for example, the MAE of the thermal field temperature prediction drops from 5°C to 1°C, and the deviation of the cross-correlation coefficient of the thermo-acoustic feature narrows from 0.2 to 0.05). Finally, a trained multi-modal imaging model is obtained. This model acquires strong robustness in the "thermal feature purification - ultrasonic feature structuring - cross-modal precise coupling" link, and can efficiently process thermo-acoustic data under complex on-site interferences after deployment, and output multi-modal imaging results with high confidence. Through multiple rounds of iteration of the training set, the server enables the model to still accurately associate thermal field anomalies with ultrasonic defects under typical interferences in industrial scenarios (such as environmental wind, lens distortion, load fluctuation), providing core algorithm support for multi-modal detection of equipment intelligent operation and maintenance.

[0050] In an embodiment of the present invention, the error calculation of the multimodal imaging model based on the thermal radiation data instance thermal field compensation reference feature and the acoustic-thermal coupling feature instance to obtain the error value of the multimodal imaging model can be implemented through the following examples.

[0051] Based on the acoustic-thermal coupling feature instance and the thermal radiation data instance, perform a consistency error calculation on the multimodal imaging model to obtain a consistency error value; Based on the thermal radiation data instance thermal field compensation reference feature and the acoustic-thermal coupling feature instance, perform a thermal field feature calibration error calculation on the multimodal imaging model to obtain a thermal field feature calibration error value; Take the sum of the consistency error value and the thermal field feature calibration error value as the error value of the multimodal imaging model.

[0052] In an embodiment of the present invention, exemplarily, during the training process of the multimodal imaging model, the server calculates the model error for the training set data of the 110 kV transformer in the substation according to the logical chain of "consistency error - thermal field feature calibration error - total error". The following is an elaboration in combination with a specific training example: The server calls the trained imaging reconstruction module to reconstruct the acoustic-thermal coupling feature instance (a 512-dimensional feature after multi-stage ultrasonic compensation and thermal field coupling, encoding the thermal anomaly and ultrasonic reflection correlation information of the inter-turn short circuit of the transformer winding) into an acoustic-thermal coupling imaging result instance (with a resolution of 1024×768, a multimodal image integrating the pseudo-color of the thermal distribution and the gray scale of the ultrasonic reflection). At the same time, the corresponding thermal radiation data instance (an infrared thermal image marked with the hot spot temperature of 120 °C at the A-phase winding and the pixel coordinate (205, 152)) in the training set is retrieved. The server calculates the consistency error between the two through the mean absolute error (MAE): Traverse the pixel temperature values of the acoustic-thermal coupling imaging result instance and the thermal radiation data instance. For the hot spot area (3×3 pixel neighborhood) of the A-phase winding, the pixel temperature of the thermal radiation data instance is [120, 118, 119, 121, 117, 122, 119, 120, 118] (°C), and the predicted temperature of the acoustic-thermal coupling imaging result instance is [118, 117, 119, 120, 116, 121, 118, 119, 117] (°C). Calculate the absolute error for each pixel and take the average: [MAE = [|120 - 118| + |118 - 117| +... + |118 - 117|] / 9 = [2 + 1 + 0 + 1 + 1 + 1 + 1 + 1 + 1] / 9 ≈ 1.11], and this value is the consistency error value, reflecting the matching accuracy of the acoustic-thermal coupling imaging in the thermal field dimension with the original thermal data. The server extracts the thermal field feature vectors (such as the encoding sub-vectors of the position and shape of the thermal anomaly area) from the thermal field compensation reference feature (the output of the third stage, which has corrected the load and structural interference and accurately encodes the 512-dimensional feature of the inter-turn short circuit thermal anomaly) of the thermal radiation data instance and the acoustic-thermal coupling feature instance (512-dimensional, integrating the thermal field calibration information and the ultrasonic defect information). Taking the thermal anomaly position as an example: In the thermal field compensation reference feature, the pixel coordinates of the hot spot of the A-phase winding are encoded as a vector (v1 = [205, 152]) (corresponding to the thermal image coordinate system); in the acoustic-thermal coupling feature instance, the predicted encoding vector of the hot spot position is (v2 = [203, 150]). The server calculates the Euclidean distance between the two as the feature alignment deviation: d = , combined with the cosine similarity deviation of the thermal anomaly form (irregular polygon in the thermal field compensation reference feature, encoding vector (m1 = [0.88, 0.75, 0.92]); approximate polygon in the example of acoustic-thermal coupling feature, encoding vector (m2 = [0.85, 0.72, 0.90])), the weighted sum is used to obtain the thermal field feature calibration error value (such as the comprehensive deviation is about 0.35). The server weights and sums the consistency error value (1.11) and the thermal field feature calibration error value (0.35) (the weights are set according to the requirements of the scenario for thermal field accuracy and imaging consistency, such as taking 0.5 each), to obtain the error value of the multi-modal imaging model: [Total error = 1.11×0.5 + 0.35×0.5 = 0.73]. Through the backpropagation algorithm, the server tunes the convolution kernel weights of the thermal feature mapper (such as enhancing the feature extraction ability of the hot spot area), the spatial attention parameters of the acoustic-thermal feature calibrator (such as improving the fusion accuracy of the heat dissipation structure prior), the cross-correlation coefficient of the feature coupling unit (such as enhancing the correlation strength between the thermal field and ultrasonic features), etc. architecture parameters, iteratively minimizing the total error until the MAE of the thermal field temperature prediction of the model on the training set ≤ 1°C and the thermal feature alignment deviation ≤ 0.1, finally completing the model training. Through the collaborative constraint of the two-dimensional error, the server ensures that the multi-modal imaging model not only conforms to the physical authenticity of the original thermal data but also accurately aligns with the defect features after thermal field compensation, providing algorithm reliability guarantee for the multi-modal accurate detection of equipment failures in industrial scenarios.

[0053] In the embodiment of the present invention, calculating the consistency error value for the multi-modal imaging model based on the acoustic-thermal coupling feature instance and the thermal radiation data instance can be implemented through the following examples.

[0054] Perform imaging reconstruction on the acoustic-thermal coupling feature instance to obtain an acoustic-thermal coupling imaging result instance including the thermal distribution feature of the target device area instance and the reflected wave spectrum characteristics of the ultrasonic reflection data instance; Take the mean absolute error between the acoustic-thermal coupling imaging result instance and the thermal radiation data instance as the consistency error value.

[0055] In an embodiment of the present invention, exemplarily, in the consistency error calculation section of the multi-modal imaging model, the server operates around the "acoustic-thermal coupling feature reconstruction imaging - pixel-level temperature deviation statistics" process for the training set data of a 110 kV transformer in a substation. First, the server calls a preset imaging reconstruction network (based on an improved U-Net architecture, including an encoding-decoding path and skip connections) to perform imaging reconstruction on the acoustic-thermal coupling feature instances (512-dimensional features after multi-stage ultrasonic compensation and thermal field coupling, encoding core information such as the thermal anomaly location of the inter-turn short circuit of phase A winding of the transformer and the ultrasonic reflection enhancement area). During the reconstruction process, when decoding the thermal distribution information, the network uses a "blue → red" pseudo-color mapping (temperature from low to high), and when decoding the ultrasonic reflection information, it uses a grayscale mapping (the higher the brightness, the greater the reflection intensity). Finally, an acoustic-thermal coupling imaging result instance (resolution 1024×768) is output. In this image, the hot spot area of phase A winding appears as dark red patches (corresponding to a temperature of about 120°C), and bright white ultrasonic reflection patches (corresponding to strong reflections caused by the short circuit) are synchronously superimposed, intuitively presenting the defect correlation between the thermal and acoustic modalities. Next, the server retrieves the corresponding thermal radiation data instance in the training set (an infrared thermal image marking the hot spot of phase A winding, and the pixel temperatures in the 3×3 pixel neighborhood of the hot spot are marked as [120, 118, 119, 121, 117, 122, 119, 120, 118]°C). For the pixel-level temperature information of the acoustic-thermal coupling imaging result instance and the thermal radiation data instance, the server calculates the absolute error pixel by pixel: for example, if the predicted temperatures in the corresponding 3×3 area in the acoustic-thermal coupling imaging are [118, 117, 119, 120, 116, 121, 118, 119, 117]°C, then the absolute errors of each pixel are |120 - 118| = 2, |118 - 117| = 1, |119 - 119| = 0, |121 - 120| = 1, |117 - 116| = 1, |122 - 121| = 1, |119 - 118| = 1, |120 - 119| = 1, |118 - 117| = 1. The server sums these absolute errors and takes the average ((2 + 1 + 0 + 1 + 1 + 1 + 1 + 1 + 1) ÷ 9 ≈ 1.11). This average value is the consistency error value, which is used to quantify the matching accuracy of the acoustic-thermal coupling imaging result in the thermal field dimension with the original thermal radiation data. The smaller the error value, the closer the thermal distribution reconstructed by the model is to the true thermal radiation state. Through pixel-level temperature deviation statistics, the server accurately measures the consistency between the acoustic-thermal imaging and the original thermal data, providing key feedback for subsequent optimization of the model: if the consistency error value is too high (e.g., > 2°C), then the weights of the decoding layer of the imaging reconstruction network are adjusted in the reverse direction to strengthen the restoration ability of the thermal distribution features; if the error meets the standard (e.g., ≤ 1°C), then the current parameter configuration is retained and other modules are continued to be optimized. This error constraint with real thermal data as the anchor point ensures that the output of the multi-modal imaging model in the thermal field dimension has both physical authenticity and cross-modal relevance.

[0056] In an embodiment of the present invention, based on the thermal field compensation reference feature of the thermal radiation data instance and the acoustic-thermal coupling feature instance, calculating the thermal field feature calibration error of the multimodal imaging model to obtain a thermal field feature calibration error value can be implemented through the following examples.

[0057] Performing imaging reconstruction on the thermal field compensation reference feature of the thermal radiation data instance to obtain an imaging reconstruction image; Performing imaging reconstruction on the acoustic-thermal coupling feature instance to obtain an acoustic-thermal coupling imaging result instance including the thermal distribution feature of the target device area instance and the reflection spectrum characteristic of the ultrasonic reflection data instance; Performing thermal field feature extraction on the imaging reconstruction image and the acoustic-thermal coupling imaging result instance respectively to correspondingly obtain a first feature vector and a second feature vector; Taking the vector difference degree between the first feature vector and the second feature vector as the thermal field feature calibration error value.

[0058] In an embodiment of the present invention, by way of example, in the calculation link of the calibration error of the thermal field characteristics of the multimodal imaging model, the server operates around the process of "thermal field reference feature reconstruction imaging - acoustic-thermal coupling feature reconstruction imaging - thermal field feature vector extraction - vector difference degree statistics" for the training set data of the 110 kV transformer in the substation. First, the server calls the imaging reconstruction network based on the improved U-Net architecture to perform imaging reconstruction on the thermal radiation data instance of the thermal field compensation reference feature (after three-stage compensation, the 512-dimensional feature accurately encodes the thermal anomaly information of the inter-turn short circuit of phase A winding: the hot spot pixel coordinates (205, 152), the irregular polygon shape, and the thermal distribution profiles of the iron core and the radiator). During the reconstruction process, the network only focuses on restoring the pure thermal field information and outputs the imaging reconstruction image. In the image, the hot spot of phase A winding appears as a dark red irregular patch (corresponding to a temperature of 120 °C), and the thermal gradients of the iron core and the radiator are completely matched with the physical structure, without ultrasonic information interference, accurately reproducing the intrinsic thermal state of the device after thermal field compensation. Next, the server performs imaging reconstruction on the acoustic-thermal coupling feature instance (a 512-dimensional feature that fuses the thermal field compensation information and the ultrasonic defect reflection information, encoding the cross-modal correlation of the hot spot position and the ultrasonic reflection enhancement area), generating an acoustic-thermal coupling imaging result instance. This image presents a dark red hot spot of phase A winding while superimposing bright white ultrasonic reflection patches (corresponding to the strong reflection caused by the short circuit), with the thermal distribution and ultrasonic information deeply fused, intuitively presenting the cross-modal correlation characteristics of the defect. Then, the server performs thermal field feature extraction on the two images respectively: for the imaging reconstruction image, it calls the thermal field feature extraction module composed of a convolutional layer and a fully connected layer to extract core thermal field information such as "hot spot pixel coordinates, number of shape vertices, and thermal gradient encoding of the device area", generating a first feature vector (for example, a vector encoding the hot spot pixel coordinates (205, 152), the number of shape vertices 8, and the winding thermal gradient activation value 0.9); for the acoustic-thermal coupling imaging result instance, it focuses on its partial thermal field information and extracts thermal field features of the same dimension, generating a second feature vector (for example, a vector encoding the predicted hot spot pixel coordinates (203, 150), the number of shape vertices 7, and the winding thermal gradient activation value 0.85). Finally, the server calculates the vector difference degree between the first feature vector and the second feature vector (taking the Euclidean distance as an example, if the first feature vector is ([205, 152, 8, 0.9]) and the second feature vector is ([203, 150, 7, 0.85]), then the difference degree is calculated by taking the square root of the sum of the squared deviations of each dimension. By way of example, the specific calculation process can be expressed as follows: ), and this value is the calibration error value of the thermal field feature. The smaller the difference degree, the more aligned the thermal field information retained in the acoustic-thermal coupling feature is with the compensated accurate thermal field feature. Through the vector-level difference statistics of the thermal field feature, the server accurately measures the retention and calibration accuracy of the thermal field information in the acoustic-thermal coupling process: if the error value is too high (such as greater than 3), the cross-correlation parameter of the feature coupling unit is adjusted reversely to strengthen the fusion accuracy of the thermal field reference feature and the ultrasonic feature; if the error meets the standard (such as less than or equal to 1), the current parameter configuration is retained, and other modules of the model are continued to be optimized. This error constraint anchored by the thermal field compensation reference ensures that the multi-modal imaging model does not lose accurate thermal field information during cross-modal coupling, providing reliable technical support for the multi-dimensional determination of equipment defects.

[0059] An embodiment of the present invention provides a computer device 100. The computer device 100 includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the foregoing multi-modal imaging method based on acoustic-thermal coupling. As Figure 2 shown, Figure 2 is a structural block diagram of the computer device 100 provided by an embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To achieve data transmission or interaction, the elements of the memory 111, the processor 112, and the communication unit 113 are electrically connected to each other directly or indirectly. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines.

[0060] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. According to the above teachings, numerous modifications and variations are possible. These embodiments are selected and described to best illustrate the principles of the disclosure and its practical applications, so that those skilled in the art can best utilize the disclosure and use various embodiments with different modifications to suit the specific applications contemplated.

Claims

1. A multimodal imaging method based on acoustic-thermal coupling, characterized in that, Including: Obtaining thermal radiation data and ultrasonic reflection data of a target device area; Performing feature representation learning on the thermal radiation data and the ultrasonic reflection data respectively, and correspondingly obtaining a thermal radiation embedding feature and an ultrasonic embedding feature; Performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, and correspondingly obtaining thermal field compensation reference features of the target device area in the thermal radiation data at multiple feature compensation stages; Performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding feature in sequence; wherein, the feature compensation stages of the multi-stage ultrasonic feature compensation processing are the same as the feature compensation stages of the multi-stage thermal radiation feature compensation processing; between every two consecutive stages of ultrasonic feature compensation processing, performing feature coupling processing on the ultrasonic feature compensation processing result obtained from the previous stage of ultrasonic feature compensation processing in the two-stage ultrasonic feature compensation processing and the thermal field compensation reference feature of the corresponding feature compensation stage; when performing the latter stage of ultrasonic feature compensation processing in the two-stage ultrasonic feature compensation processing, performing ultrasonic feature compensation processing on the ultrasonic embedding feature and the feature coupling output after the feature coupling is completed; Performing imaging reconstruction on the acoustic-thermal coupling feature obtained after the multi-stage ultrasonic feature compensation processing, and obtaining an acoustic-thermal coupling imaging result including the thermal distribution feature of the target device area and the reflection spectrum characteristics of the ultrasonic reflection data.

2. The method according to claim 1, wherein The method is implemented by a pre-trained multi-modal imaging model; the multi-modal imaging model includes a core imaging unit and a thermal feature calibration unit; the core imaging unit at least includes an ultrasonic feature mapper and a feature coupling unit, and the thermal feature calibration unit includes a thermal feature mapper and an acoustic-thermal feature calibrator; The multi-modal imaging model is obtained by the following method, including: Loading thermal radiation data instances in a preset training set into the thermal feature calibration unit, and performing feature mapping processing on the thermal radiation data instances through the thermal feature mapper in the thermal feature calibration unit to obtain thermal radiation data instance thermal radiation embedding features; Performing multi-stage thermal radiation feature compensation processing on the thermal radiation data instance thermal radiation embedding features through the acoustic-thermal feature calibrator in the thermal feature calibration unit, and correspondingly obtaining thermal radiation data instance thermal field compensation reference features of the target device area instances in the thermal radiation data instances at multiple feature compensation stages; Loading ultrasonic reflection data instances in a preset training set into the core imaging unit, and performing feature mapping processing on the ultrasonic reflection data instances through the ultrasonic feature mapper in the core imaging unit to obtain sample ultrasonic embedding features; Performing multi-stage ultrasonic feature compensation processing on the sample ultrasonic embedding features in sequence through the feature coupling unit in the core imaging unit to obtain acoustic-thermal coupling feature instances; Performing imaging reconstruction on the acoustic-thermal coupling feature instances, and obtaining acoustic-thermal coupling imaging result instances including the thermal distribution features of the target device area instances and the reflection spectrum characteristics of the ultrasonic reflection data instances; Take the mean absolute error between the acoustic-thermal coupling imaging result instance and the thermal radiation data instance as the consistency error value; Perform imaging reconstruction on the thermal field compensation reference feature of the thermal radiation data instance to obtain an imaging reconstruction image; Extract thermal field features from the imaging reconstruction image and the acoustic-thermal coupling imaging result instance respectively to obtain a first feature vector and a second feature vector correspondingly; Take the vector difference degree between the first feature vector and the second feature vector as the thermal field feature calibration error value; Take the sum of the consistency error value and the thermal field feature calibration error value as the error value of the multimodal imaging model; Optimize the architecture parameters in the multimodal imaging model based on the error value to obtain a trained multimodal imaging model.

3. The method according to claim 1, characterized in that The multi-stage thermal radiation feature compensation process for the thermal radiation embedding feature correspondingly obtains the thermal field compensation reference feature of the target device area in the thermal radiation data under multiple feature compensation stages, including: Perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedding feature through multiple intermediate layers in the trained acoustic-thermal feature calibrator to correspondingly obtain the thermal field compensation reference feature of the target device area in the thermal radiation data under multiple feature compensation stages; Among them, when performing the first-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, the first intermediate layer among the multiple intermediate layers is used to perform the first-stage thermal radiation feature compensation processing on the thermal radiation embedding feature to obtain the thermal field compensation reference feature of the target device area in the first stage in the thermal radiation data; When performing the target-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, the target intermediate layer among the multiple intermediate layers is used to perform the target-stage thermal radiation feature compensation processing on the thermal field compensation reference feature of the previous stage to obtain the thermal field compensation reference feature of the target device area in the target stage in the thermal radiation data, and the previous stage is the previous stage of the target stage.

4. The method according to claim 3, characterized in that, The intermediate layer is a first feature compensation component, and the first feature compensation component includes a feature compensation operator; The multi-stage thermal radiation feature compensation process for the thermal radiation embedding feature through multiple intermediate layers in the trained acoustic-thermal feature calibrator to correspondingly obtain the thermal field compensation reference feature of the target device area in the thermal radiation data under multiple feature compensation stages, including: Perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedding feature by invoking the feature compensation operators in each first feature compensation component among the multiple intermediate layers to correspondingly obtain the thermal field compensation reference feature of the target device area in the thermal radiation data under multiple feature compensation stages.

5. The method according to claim 4, characterized in that, The sequential multi-stage ultrasonic feature compensation process for the ultrasonic embedding feature includes: When performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the first second feature compensation component among the multiple second feature compensation components is used to perform the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, and the first-stage output feature after the first-stage ultrasonic feature compensation processing of the ultrasonic reflection data is obtained; After performing the target-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the target cross-correlation feature alignment component in the feature coupling unit is used to perform the feature coupling on the target-stage output feature after the target-stage ultrasonic feature compensation processing and the thermal field compensation reference feature generated by the target first feature compensation component in the acoustic-thermal feature calibrator, and the target feature coupling output is obtained; When performing the target-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the target second feature compensation component among the multiple second feature compensation components is used to perform the target-stage ultrasonic feature compensation processing on the ultrasonic embedded feature and the previous feature coupling output generated by the previous cross-correlation feature alignment component, and the target-stage output feature after the target-stage ultrasonic feature compensation processing of the ultrasonic reflection data is obtained; The output feature obtained after the last-stage ultrasonic feature compensation processing of the multi-stage ultrasonic feature compensation processing is used as the acoustic-thermal coupling feature obtained after the multi-stage ultrasonic feature compensation processing; Wherein, the feature coupling unit includes a plurality of cascaded second feature compensation components, and a cross-correlation feature alignment component is arranged between every two second feature compensation components with a superior-subordinate relationship; the number of the second feature compensation components is the same as the number of the first feature compensation components in the acoustic-thermal feature calibrator.

6. The method according to claim 5, characterized in that, The multiple second feature compensation components include: a second feature compensation component for performing dimensionality reduction processing on the ultrasonic embedded feature, and a second feature compensation component for performing dimensionality increase processing on the ultrasonic embedded feature; Wherein, the number of the second feature compensation components for performing dimensionality reduction processing on the ultrasonic embedded feature is the same as the number of the second feature compensation components for performing dimensionality increase processing on the ultrasonic embedded feature; and the second feature compensation component for performing dimensionality increase processing on the ultrasonic embedded feature is arranged after the second feature compensation component for performing dimensionality reduction processing on the ultrasonic embedded feature.

7. The method according to claim 5, wherein Before performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the method further includes: Generating a simulation perturbation feature of the feature coupling unit through a preset acoustic-thermal perturbation simulation module; Using the simulation perturbation feature and the ultrasonic embedded feature as the current processing features and loading them into the feature coupling unit; The step of using the first second feature compensation component among the multiple second feature compensation components to perform the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature and obtaining the first-stage output feature after the first-stage ultrasonic feature compensation processing of the ultrasonic reflection data includes: Using the first second feature compensation component to perform the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature and the simulation perturbation feature, and obtaining the first-stage output feature.

8. The method according to claim 5, characterized in that, The feature dimension of the output features of the target stage is consistent with the feature dimension of the thermal field compensation reference features generated by the target first feature compensation component; The feature coupling of the output features of the target stage after the target stage ultrasonic feature compensation process and the thermal field compensation reference features generated by the target first feature compensation component in the acoustic-thermal feature calibrator is performed through the target cross-correlation feature alignment component in the feature coupling unit, and the target feature coupling output is obtained, including: Through the target cross-correlation feature alignment component in the feature coupling unit, the output features of the target stage and the thermal field compensation reference features generated by the target first feature compensation component are subjected to feature merging processing in the energy dimension to obtain merged features; Peak feature compression processing and mean feature compression processing are respectively performed on the merged features to correspondingly obtain peak compression features and mean compression features; The peak compression features and the mean compression features are used as the compression features; Convolution operation is performed on the compression features to obtain the acoustic-thermal coupling modulation coefficient between the output features of the target stage and the thermal field compensation reference features generated by the target first feature compensation component; Based on the acoustic-thermal coupling modulation coefficient, feature coupling processing calculation is performed on the merged features to obtain the target feature coupling output.

9. The method according to claim 1, characterized in that, The thermal field compensation reference features include a thermal radiation feature representation for indicating the thermal field distribution features of the target device area; The thermal field distribution features include: the position of the thermal anomaly area, the shape of the thermal anomaly, and the contour of the device area.

10. A server system, characterized in that, It includes a server, and the server is used to execute the method described in any one of claims 1-9.

Images