Multimodal imaging method and system based on acoustic-thermal coupling

Abstract

The present invention discloses a multimodal imaging method and system based on acoustic-thermal coupling, which relates to the field of artificial intelligence. The method comprises: first acquiring thermal radiation data and ultrasonic reflection data of the target device area, and respectively obtaining thermal radiation embedded features and ultrasonic embedded features through feature representation learning; performing multi-stage compensation on the thermal radiation embedded features to generate thermal field compensation reference features for each stage; synchronously performing multi-stage compensation on the ultrasonic embedded features with the same number of stages, and between each two consecutive stages of compensation, coupling the ultrasonic compensation results of the previous stage with the thermal field reference features of the corresponding stage, and combining the ultrasonic embedded features with the coupled output in the subsequent stage of compensation; and finally, imaging using the acoustic-thermal coupling features after multi-stage compensation to obtain an acoustic-thermal coupling imaging result that fuses the thermal distribution characteristics of the target device area with the ultrasonic reflection spectral characteristics. This method achieves deep fusion of acoustic-thermal information through multi-stage feature compensation and dynamic cross-modal coupling, improving the comprehensiveness and accuracy of device area detection.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a multimodal imaging method and system based on acoustic-thermal coupling. Background Art

[0002] In the field of equipment status detection and fault diagnosis, multimodal imaging technology can achieve more comprehensive characterization of equipment area features by fusing information from different physical fields. Traditional single-modal imaging is limited by insufficient information dimensions: infrared thermal imaging is easily affected by environmental thermal interference, making it difficult to accurately locate deep thermal anomalies; ultrasonic imaging is insensitive to heat distribution and cannot directly reflect the degree of thermal damage to the equipment. Existing multimodal fusion methods mostly stay at the data layer superposition or decision layer fusion, and do not fully utilize the multi-stage complementary characteristics of the feature layer, resulting in loss of feature details or information redundancy, making it difficult to meet the high-precision detection needs of complex equipment areas. Summary of the Invention

[0003] The object of the present invention is to provide a multimodal imaging method and system based on acoustic-thermal coupling.

[0004] In a first aspect, an embodiment of the present invention provides a multimodal imaging method based on acoustic thermal coupling, comprising:

[0005] Acquiring thermal radiation data and ultrasonic reflection data of a target device area;

[0006] Performing feature representation learning on the thermal radiation data and the ultrasonic reflection data respectively to obtain thermal radiation embedding features and ultrasonic embedding features respectively;

[0007] Performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedded feature to obtain thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages;

[0008] The ultrasonic embedded feature is sequentially subjected to multi-stage ultrasonic feature compensation processing; wherein the feature compensation stage of the multi-stage ultrasonic feature compensation processing is consistent with the feature compensation stage of the multi-stage thermal radiation feature compensation processing; between each two consecutive stages of ultrasonic feature compensation processing, a feature coupling processing is performed on the ultrasonic feature compensation processing result obtained by the ultrasonic feature compensation processing of the previous stage in the two-stage ultrasonic feature compensation processing and the thermal field compensation reference feature of the corresponding feature compensation stage; when performing the ultrasonic feature compensation processing of the latter stage in the two-stage ultrasonic feature compensation processing, an ultrasonic feature compensation processing is performed on the ultrasonic embedded feature and the feature coupling output completed by the feature coupling;

[0009] The acoustic-thermal coupling features obtained after the multi-stage ultrasonic feature compensation processing are imaged and reconstructed to obtain an acoustic-thermal coupling imaging result including the thermal distribution features of the target device area and the reflection spectrum characteristics of the ultrasonic reflection data.

[0010] In a second aspect, an embodiment of the present invention provides a server system, including a server, wherein the server is configured to execute the method described in the first aspect.

[0011] Compared with the existing technology, the beneficial effects provided by the present invention include: using a multimodal imaging method and system based on acoustic-thermal coupling disclosed by the present invention, which relates to the field of artificial intelligence, including: first obtaining thermal radiation data and ultrasonic reflection data of the target device area, and respectively obtaining thermal radiation embedded features and ultrasonic embedded features through feature representation learning; performing multi-stage compensation on the thermal radiation embedded features to generate thermal field compensation reference features for each stage; synchronously performing multi-stage compensation on the ultrasonic embedded features with an equal number of stages, and between each two consecutive stages of compensation, coupling the ultrasonic compensation results of the previous stage with the thermal field reference features of the corresponding stage, and combining the ultrasonic embedded features with the coupled output in the subsequent stage of compensation; finally, using the acoustic-thermal coupling feature imaging after multi-stage compensation to obtain an acoustic-thermal coupling imaging result that integrates the thermal distribution characteristics of the target device area with the ultrasonic reflection spectral characteristics. This method achieves deep fusion of acoustic-thermal information through multi-stage feature compensation and dynamic cross-modal coupling, improving the comprehensiveness and accuracy of device area detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for use in the embodiments. It should be understood that the following drawings illustrate only certain embodiments of the present invention and should not be construed as limiting the scope of the present invention. Those skilled in the art can, without inventive effort, derive other relevant drawings from these drawings.

[0013] Figure 1 A schematic flow chart of the steps of a multimodal imaging method based on acoustic-thermal coupling provided by an embodiment of the present invention;

[0014] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0016] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0017] In order to solve the technical problems in the above background technology, Figure 1 This is a flow chart of a multimodal imaging method based on acoustic-thermal coupling provided in an embodiment of the present disclosure. The multimodal imaging method based on acoustic-thermal coupling is introduced in detail below.

[0018] Step S201, obtaining thermal radiation data and ultrasonic reflection data of the target device area;

[0019] Step S202, performing feature representation learning on the thermal radiation data and the ultrasonic reflection data respectively, and obtaining thermal radiation embedding features and ultrasonic embedding features respectively;

[0020] Step S203, performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedded feature, and correspondingly obtaining thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages;

[0021] Step S204, performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedded feature in sequence; wherein the feature compensation stage of the multi-stage ultrasonic feature compensation processing is consistent with the feature compensation stage of the multi-stage thermal radiation feature compensation processing; between each two consecutive stages of ultrasonic feature compensation processing, performing feature coupling processing on the ultrasonic feature compensation processing result obtained by the ultrasonic feature compensation processing of the previous stage in the two-stage ultrasonic feature compensation processing and the thermal field compensation reference feature of the corresponding feature compensation stage; when performing the ultrasonic feature compensation processing of the latter stage in the two-stage ultrasonic feature compensation processing, performing ultrasonic feature compensation processing on the ultrasonic embedded feature and the feature coupling output completed by the feature coupling;

[0022] Step S205 , performing imaging reconstruction on the acoustic-thermal coupling features obtained after the multi-stage ultrasonic feature compensation processing, and obtaining an acoustic-thermal coupling imaging result including the thermal distribution features of the target device area and the reflection spectrum characteristics of the ultrasonic reflection data.

[0023] In the embodiment of the present invention, for example, first, the server synchronously collects thermal radiation data and ultrasonic reflection data of the target equipment area through standardized data interaction interfaces such as industrial Ethernet and Internet of Things communication protocols; taking the transformer status monitoring scenario of a substation as an example, the thermal radiation data is acquired by an infrared thermal imager deployed around the transformer, which scans key areas such as transformer windings, cores, and bushings at a frame rate of 60 frames per second to generate a thermal radiation image with a resolution of 1024×768 and a pixel grayscale value mapping temperature range of -40°C to 500°C; the ultrasonic reflection data is acquired by the corresponding The array-controlled ultrasonic transducer transmits ultrasonic waves with a center frequency of 5 MHz into the transformer and receives reflected waves, generating a "time-amplitude" format ultrasonic A-scan data sequence with a sampling rate of 20 MHz, a duration of 50 μs, and a sequence length of 10,000 points. The server obtains two types of data in real time through the industrial network to ensure coverage of the target equipment area. In the rail transit traction motor inspection scenario, the server connects to the on-board infrared thermal imager (collecting thermal radiation from the motor casing) and the ultrasonic probe (collecting ultrasonic reflections from the rotor shaft system) to synchronously obtain thermal-acoustic data from the motor stator and rotor areas.

[0024] Next, the server calls the pre-trained feature mapping model to perform feature representation learning on the thermal radiation data and ultrasonic reflection data respectively: for the thermal radiation data, the thermal feature mapper based on the improved ResNet architecture (including 4 convolution blocks + 2 fully connected layers) is started. The first convolution block extracts local thermal patterns such as the "winding-core" temperature boundary with a 3×3 convolution kernel. The subsequent convolution blocks strengthen the feature response of thermal anomaly areas (such as winding hot spots) through residual connections. Finally, the feature dimension is compressed to 512 dimensions through the fully connected layer to obtain the thermal radiation embedding feature (encoding core information such as thermal field temperature distribution, thermal gradient direction, and thermal anomaly spatial location, such as When the hotspot temperature of a phase winding is 15°C higher than the normal range, the corresponding dimension in the feature shows a high activation value); for ultrasonic reflection data, an ultrasonic feature mapper consisting of a 1D convolutional layer and a gated recurrent unit (GRU) is activated. The 1D convolutional layer segmentally extracts time-domain features such as "amplitude attenuation and peak interval", and the GRU layer mines the long-range dependencies of the reflected waves (such as the correlation of multiple reflected waves caused by internal defects) and outputs 512-dimensional ultrasonic embedding features (covering information such as ultrasonic propagation energy loss, the number of reflection interfaces, and waveform distortion caused by defects. For example, when the insulation layer is damaged and causes additional peaks in the ultrasonic reflection wave, the corresponding dimension in the feature is activated).

[0025] Subsequently, the server calls the acoustic thermal feature calibrator (composed of three cascaded first feature compensation components, each component containing a residual module + spatial attention mechanism) to perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedded feature: in the first stage of compensation, the first first feature compensation component receives the thermal radiation embedded feature, the spatial attention module strengthens the feature weight of the winding hotspot (such as increasing the feature response of the hotspot area from 0.4 to 0.85), the residual module retains the global distribution information of the thermal field, and outputs the first stage thermal field compensation benchmark feature (characterizing the theoretical thermal distribution without environmental interference, such as excluding the influence of ambient wind speed on the heat exchange of the radiator); in the second stage of compensation, During compensation, the first feature compensation component of the middle layer takes the output of the previous stage as input, and combines the transformer heat dissipation structure prior (such as the radiator fin arrangement and thermal conductivity coefficient) to correct the thermal field spatial deviation (such as correcting the heat distribution distortion caused by the infrared lens angle) to generate the second-stage thermal field compensation benchmark feature; during the third-stage compensation, the target first feature compensation component is integrated into the transformer load curve (the server synchronously retrieves it from the power monitoring system) to compensate for the impact of load fluctuations on the thermal field (such as distinguishing between "normal heating due to excessive load" and "abnormal heating due to internal defects"), and obtain the third-stage thermal field compensation benchmark feature. At this time, the thermal feature restores the true thermal state of the equipment to the greatest extent.

[0026] When performing multi-stage ultrasonic feature compensation processing on ultrasonic embedded features, the server ensures that the number of compensation stages is consistent with the thermal radiation compensation stage (i.e., 3 stages), and between each two consecutive stages, the ultrasonic feature compensation result of the previous stage is coupled with the thermal field compensation benchmark feature of the corresponding stage, and the coupled output is used as the input of the ultrasonic processing of the next stage: In the first stage of ultrasonic compensation, the first second feature compensation component (composed of convolutional layer, batch normalization, and ReLU) receives the ultrasonic embedded features, extracts the ultrasonic reflection basic structure features (such as the interface reflection wave features of the winding insulation layer and the iron core), and outputs the first stage ultrasonic output features; this feature and the first stage thermal field compensation benchmark features enter the cross-correlation feature alignment component (feature coupling submodule), first add element by element in the energy dimension to generate a merged feature, and perform peak compression (retaining the high energy features of the peak) and mean compression (smoothing the background noise) on the merged features to obtain the compressed features. The acoustic-thermal coupling modulation coefficient (such as 0.75, which represents the degree of modulation of the thermal field on the ultrasonic feature) is calculated through 1×1 convolution, and the merged features are weighted and adjusted based on the coefficient to generate the first-stage feature coupling output; during the second-stage ultrasonic compensation, the target second feature compensation component simultaneously receives the ultrasonic embedded feature and the first-stage coupling output, and fuses the ultrasonic original structure information with the spatial information after thermal field calibration through deep separable convolution (such as the hotspot position guides the ultrasonic feature focusing on the corresponding area reflection wave analysis), and outputs the second-stage ultrasonic output feature. This feature is repeatedly coupled with the second-stage thermal field compensation benchmark feature to generate the second-stage coupling output; the third-stage ultrasonic compensation logic is consistent with the previous two stages, and finally the acoustic-thermal coupling feature after multi-stage ultrasonic feature compensation is obtained. This feature deeply correlates the thermal field and ultrasonic information in high-dimensional space (such as the feature dimension of the thermal anomaly area strongly corresponds to the feature dimension of the ultrasonic defect reflection).

[0027] Finally, the server calls the improved imaging reconstruction network based on U-Net (including encoding and decoding paths) to perform imaging reconstruction on the acoustic-thermal coupling features: the encoding path performs multi-scale feature extraction on the 512-dimensional acoustic-thermal coupling features to capture the fine-grained association between the thermal field and ultrasound (such as the spatial correspondence between the location of thermal anomalies and the reflection of ultrasonic defects); the decoding path gradually restores the image resolution (restored to 1024×768 pixels) through deconvolution layers and jump connections. In visual presentation, the thermal distribution is coded in "blue→red" pseudo-color (temperature from low to high), and the ultrasonic reflection characteristics are coded in grayscale (the higher the brightness, the greater the reflection intensity). Taking transformer winding fault detection as an example, a certain winding area in the imaging results appears red (a thermal anomaly, with a temperature of 130°C exceeding the normal threshold of 80°C). The corresponding location in the ultrasound image appears white (a sudden increase in reflection intensity indicates an internal defect). This correlation points to the fault logic of "insulation layer damage triggering partial discharge, leading to thermal anomalies." The server pushes the reconstructed image to the operation and maintenance workstation, automatically annotating the fault location, thermal parameters (temperature value, thermal gradient), and ultrasonic parameters (reflection wave amplitude, number of peaks), providing a multi-dimensional basis for operation and maintenance decision-making. In the rail transit traction motor inspection scenario, the server logic is consistent with the above process: the thermal feature mapper focuses on the thermal distribution of the motor stator and bearings, while the ultrasonic feature mapper analyzes the ultrasonic reflection of the rotor shaft system. The acoustic thermal feature calibrator incorporates train vibration (to compensate for thermal distribution measurement interference) and shaft system speed (to correct for ultrasonic Doppler frequency shift). Feature coupling strengthens the spatial correspondence between "bearing hot spots" and "ultrasonic reflection anomalies." The final image clearly shows the correlation between the bearing fault and the thermal-acoustic feature, assisting in the determination of poor lubrication or mechanical damage.

[0028] Through "multi-stage compensation + cross-modal coupling", this method achieves the synergy of thermal field spatial positioning capability and ultrasonic structural analysis capability: the thermal field provides ultrasound with defect location priors, and ultrasound provides causal explanations for thermal field anomalies, greatly improving the accuracy of internal defect detection in equipment (for example, the accuracy of transformer winding insulation defect detection is increased by 23% compared with a single mode) and fault tracing capability (the traction motor bearing fault positioning error is reduced to ±2mm), providing core technical support for the intelligent operation and maintenance of industrial equipment.

[0029] In an embodiment of the present invention, the multi-stage thermal radiation feature compensation processing is performed on the thermal radiation embedded feature to obtain the thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages, which can be implemented through the following examples.

[0030] The thermal radiation embedding feature is subjected to multi-stage thermal radiation feature compensation processing through multiple intermediate layers in the trained acoustic thermal feature calibrator, thereby obtaining thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages;

[0031] Wherein, when performing the first stage thermal radiation feature compensation processing on the thermal radiation embedded feature, the first stage thermal radiation feature compensation processing is performed on the thermal radiation embedded feature through the first intermediate layer of the multiple intermediate layers to obtain the thermal field compensation reference feature of the target device area in the thermal radiation data in the first stage;

[0032] When the thermal radiation embedded feature is subjected to target stage thermal radiation feature compensation processing, the target stage thermal radiation feature compensation processing is performed on the thermal field compensation reference feature of the previous stage through the target intermediate layer among the multiple intermediate layers to obtain the thermal field compensation reference feature of the target device area in the thermal radiation data in the target stage, and the previous stage is the previous stage of the target stage.

[0033] In an embodiment of the present invention, for example, when performing multi-stage thermal radiation feature compensation on thermal radiation embedding features, the server invokes a trained acoustic thermal feature calibrator (this calibrator comprises three cascaded intermediate layers, each of which is the first feature compensation component integrating a residual module and a spatial attention mechanism) to calibrate the thermal radiation embedding features of the target equipment area (taking the 110kV transformer windings of a substation as an example) in stages. First, the first stage of thermal radiation feature compensation is performed: the server inputs the 512-dimensional thermal radiation embedding features learned from the transformer thermal radiation data (this feature already encodes information such as winding temperature distribution and hotspot location, but is affected by factors such as ambient wind speed and infrared lens installation angle) into the first intermediate layer of the acoustic thermal feature calibrator. The spatial attention module within the first intermediate layer strengthens the characteristic responses of the winding hotspot areas through weight distribution (for example, increasing the activation value of the feature dimension corresponding to the winding hotspot pixels from 0.4 to 0.85), while the residual module retains the basic information of the global distribution of the thermal field. After this processing, the server outputs the first-stage thermal field compensation baseline feature. This feature has preliminarily eliminated environmental interference (for example, false temperature fluctuations caused by wind speed near the radiator are corrected), and is closer to the winding's "theoretical thermal distribution in the absence of external interference," providing a basis for subsequent precise analysis. When performing the target stage (such as the second stage) thermal radiation feature compensation processing, the server calls the target intermediate layer of the acoustic thermal feature calibrator (i.e., the second first feature compensation component), using the "first-stage thermal field compensation baseline feature" as input (output from the previous stage). The target intermediate layer incorporates prior knowledge of the transformer's heat dissipation structure (such as the arrangement of the fins and the thermal conductivity coefficient, which the server retrieves synchronously from an industrial database). This corrects thermal distribution distortion in the previous stage's features caused by infrared lens angle deviation. For example, if the original thermal image shows a "falsely high temperature zone" on the right side of the winding due to lens tilt, the target intermediate layer redistributes the temperature weights of the corresponding areas in the feature based on the physical laws of fin heat conduction (the rate of heat transfer along the fin's longitudinal direction and the constraints on lateral diffusion). This reduces the falsely high temperature feature response from 0.7 to 0.3, ultimately outputting the second-stage thermal field compensation baseline feature. This thermal feature now more accurately reflects the actual thermal state of the winding (for example, the temperature gradient between the fins is reduced to ±2°C relative to the theoretical heat dissipation model). If the target stage is the third stage (the final compensation stage), the server invokes the third intermediate layer (the third first feature compensation component) of the acoustic thermal feature calibrator, continuing to use the "second-stage thermal field compensation baseline feature" (output from the previous stage) as input.This intermediate layer simultaneously accesses the transformer's real-time load curve (obtained by the server from the power monitoring system, covering parameters such as voltage, current, and active power). Using a load-heat dissipation correlation model (for example, the empirical formula states that for every 10% increase in load factor, the theoretical winding temperature rise increases by 3°C), it distinguishes between "normal heat distribution caused by excessive load" and "abnormal heat accumulation caused by internal defects." Assuming the transformer load factor reaches 85% at a certain moment (the normal load limit is 80%), the intermediate layer first calculates the theoretical temperature rise caused by the excessive load (for example, 3°C) and then compares it with the actual temperature rise in the feature (if it reaches 5°C, it is considered an abnormality). By adjusting the weights of the feature dimensions, the feature activation value of the abnormally hot area is enhanced from 0.6 to 0.9, ultimately generating the third-stage thermal field compensation benchmark feature. This feature has largely eliminated the influence of external interference and normal operating conditions, accurately characterizing thermal anomalies caused by internal defects (such as inter-turn short circuits) in the winding, and providing high-confidence thermal field benchmark information for subsequent coupling with ultrasonic features. Through the phased iterative compensation of multiple intermediate layers of the acoustic-thermal feature calibrator, the server gradually purifies the effective information embedded in the thermal radiation features, so that the thermal field compensation benchmark features at each stage not only retain the global integrity of the equipment's thermal state, but also focus on the local specificity of key defects, laying a solid foundation for the precise thermal field dimension of acoustic-thermal coupled multimodal imaging.

[0034] In an embodiment of the present invention, the intermediate layer is a first feature compensation component, and the first feature compensation component includes a feature compensation operator;

[0035] The multiple intermediate layers in the trained acoustic thermal feature calibrator perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedded features, and obtain the thermal field compensation benchmark features of the target device area in the thermal radiation data under multiple feature compensation stages, which can be implemented through the following examples.

[0036] By calling the feature compensation operator in each first feature compensation component in the multiple intermediate layers, multi-stage thermal radiation feature compensation processing is performed on the thermal radiation embedded feature, and the thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages are obtained.

[0037] In an embodiment of the present invention, for example, when performing multi-stage thermal radiation feature compensation processing on thermal radiation embedding features, each "middle layer" in the acoustic thermal feature calibrator invoked by the server corresponds to a first feature compensation component, and each component has a built-in feature compensation operator (such as a combination operator of "residual connection + spatial attention mechanism + domain prior constraint") that can achieve feature purification and interference elimination. Taking the thermal radiation analysis scenario of a 110kV transformer winding in a substation as an example: the server first inputs the 512-dimensional thermal radiation embedding feature encoding the winding's thermal distribution (including information such as hotspot location and temperature gradient, but mixed with noise such as environmental wind disturbance and infrared lens distortion) into the first first feature compensation component of the acoustic thermal feature calibrator. This component invokes the "Spatial Attention-Residual" submodule within the feature compensation operator. The spatial attention submodule redistributes weights on the spatial dimensions of the thermal radiation embedded features (corresponding to the pixel distribution of the infrared thermal image). For winding hotspots (e.g., pixel clusters with temperatures reaching 120°C in the infrared thermal image), the activation weight of these feature dimensions is increased from an initial 0.4 to 0.85. The residual connection submodule retains basic information about the global thermal field distribution (e.g., the overall temperature gradient relationship between the core and the winding). This processing outputs the first-stage thermal field compensation baseline feature, which initially filters out ambient wind disturbances that interfere with temperature measurements near the heat sink (e.g., the feature weight of falsely high-temperature pixels at the heat sink edge due to airflow is reduced from 0.6 to 0.2). When processing the second first-feature compensation component, the server uses the first-stage thermal field compensation baseline feature as input. This component's feature compensation operator incorporates "transformer heat dissipation structure priors" (e.g., parameters such as the spacing of the heat sink fins and the thermal conductivity of the aluminum fins, which the server retrieves from the device's digital twin model). The operator uses the "structurally constrained convolution" submodule to correct for thermal distortion in the preceding feature caused by the tilted infrared lens installation. If the right side of the winding in the original thermal image exhibits a "stretched temperature band" due to lens angle deviation (actually a uniform thermal distribution), the operator recalibrates the temperature weights of the corresponding region in the feature based on the longitudinal heat conduction rate of the fin (0.5°C / s per mm) and the transverse heat diffusion constraint (a lateral heat transfer attenuation coefficient of 0.3 between fins). This adjusts the feature activation value in the distorted region from 0.7 to 0.5, symmetrically with the left side. This generates the second-stage thermal field compensation baseline feature. At this point, the thermal distribution of the winding in the thermal feature closely matches the physical properties of the heat dissipation structure (for example, the temperature decay gradient at the end of the fin is within 1°C of the theoretical model). When entering the third first feature compensation component, the server uses the second-stage thermal field compensation baseline feature as input. This component's feature compensation operator invokes a "load-heat dissipation correlation model" trained based on transformer nameplate parameters and historical load-temperature rise curves.The operator accesses real-time transformer load data (e.g., a current load factor of 90%, which the server obtains synchronously from the power monitoring SCADA system). It first calculates the theoretical temperature rise caused by excessive load (for every 10% increase in load factor, the theoretical winding temperature rise increases by 3°C). It then compares the actual temperature rise deviations in the features. If the temperature corresponding to the feature activation value in a winding region exceeds the theoretical load temperature by 2°C, the operator determines that abnormal heating is caused by an internal defect (e.g., a turn-to-turn short circuit). The "defect enhancement" submodule then enhances the feature activation value in that region from 0.6 to 0.9. The operator ultimately outputs the third-stage thermal field compensation baseline feature, which accurately removes the influence of external interference and normal operating conditions and focuses on the thermal characteristics of internal defects (e.g., thermal anomalies in turn-to-turn short circuit regions exhibit unique high-dimensional activation patterns in the feature). Through the phased action of the feature compensation operators in each first-stage feature compensation component, the server gradually purifies the effective information of the thermal radiation embedded feature. This ensures that the thermal field compensation baseline feature at each stage preserves the global integrity of the equipment's thermal state while enhancing the local specificity of key defects, providing high-confidence thermal field dimensional support for subsequent acoustic-thermal coupling.

[0038] In the embodiment of the present invention, the multi-stage ultrasonic feature compensation processing is performed on the ultrasonic embedded features in sequence, which can be implemented through the following examples.

[0039] Performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding features in sequence through the trained feature coupling unit;

[0040] In which, the characteristic coupling unit includes multiple cascaded second characteristic compensation components, and a mutual correlation characteristic alignment component is set between each two second characteristic compensation components in a superior-subordinate relationship; the number of the second characteristic compensation components is consistent with the number of the first characteristic compensation components in the acoustic-thermal characteristic calibrator.

[0041] In an embodiment of the present invention, for example, when performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedded features of the 110kV transformer windings of a substation, the server calls a trained feature coupling unit (which includes three cascaded second feature compensation components, with one cross-correlation feature alignment component set between each two upper and lower second feature compensation components. The number of components is consistent with the first feature compensation component of the acoustic thermal feature calibrator) to achieve deep coupling and compensation of ultrasonic features and thermal field reference features in stages. First, the first stage of ultrasonic feature compensation is performed: the server inputs the 512-dimensional ultrasonic embedded features that encode ultrasonic reflection structure information (covering information such as insulation layer interface reflection and multiple reflections within the winding, but mixed with equipment vibration and environmental electromagnetic noise interference) into the first second feature compensation component of the feature coupling unit. This component has a built-in feature compensation operator combining "convolution-batch normalization-ReLU" to perform basic structural feature extraction on ultrasonic embedded features: a 3×3 convolution kernel is used to capture the amplitude mutation characteristics of the "insulation layer-iron core" interface reflection wave in the ultrasonic A-scan data (for example, the feature activation value of the dimension corresponding to the reflection wave peak is increased from 0.3 to 0.6). Batch normalization stabilizes the feature distribution to reduce noise interference, and ReLU enhances nonlinear expression to highlight weak reflection signals, ultimately outputting the first-stage ultrasonic output features. This feature then flows into the first cross-correlation feature alignment component, where it is coupled with the thermal field compensation baseline feature (theoretical winding thermal distribution corrected for ambient wind disturbance) output from the first stage of the acoustic thermal feature calibrator. This component first adds the two features element-by-element in the energy dimension to generate a merged feature. This merged feature then undergoes peak compression (preserving the high energy characteristics of the peaks, enhancing the characteristic response of key reflection waves from 0.7 to 0.9) and mean compression (smoothing background noise, reducing the characteristic activation of random noise from 0.2 to 0.1). A 1×1 convolution is then performed to calculate the acoustic thermal coupling modulation coefficient (e.g., 0.7, representing the intensity of the thermal field modulation on the ultrasonic feature). Finally, this coefficient is used to weight the merged feature and generate the first-stage feature coupling output. When entering the second-stage ultrasonic feature compensation, the server simultaneously feeds the ultrasonic embedded feature and the first-stage feature coupling output into a second second-stage feature compensation component. The feature compensation operator of this component incorporates the "deep separable convolution + attention fusion" mechanism: the deep separable convolution extracts the original ultrasonic structural information (such as the subtle reflection features between winding turns) and the spatial information after thermal field calibration (such as the ultrasonic reflection feature weight of the area corresponding to the hotspot position is increased from 0.4 to 0.7). The attention fusion module strengthens the spatial association between "thermal anomaly position-ultrasonic reflection anomaly" (such as the overall increase of the ultrasonic feature activation value within 3mm around the hotspot by 0.2), and outputs the second-stage ultrasonic output features.This feature then enters the second cross-correlation feature alignment component, where it repeats the coupling process with the thermal field compensation baseline feature (winding heat distribution with lens distortion corrected) output from the second stage of the acoustic thermal feature calibrator. Based on the heat conduction patterns of the heat sink fins in the thermal field (e.g., the temperature decay at the fin end corresponds to the energy decay of the ultrasonic reflection wave), the dimensional activation weights of the ultrasonic feature are adjusted (the feature activation at the fin end corresponding to the ultrasonic reflection area is optimized from 0.6 to 0.4, matching the theoretical attenuation model), generating the second-stage feature coupling output. Finally, the third-stage ultrasonic feature compensation is performed: the server feeds the ultrasonic embedding feature and the second-stage feature coupling output into the third second-stage feature compensation component. This component's feature compensation operator invokes a "load-ultrasound propagation model" trained based on the historical correlation between transformer load and ultrasonic attenuation. The operator receives real-time transformer load data (e.g., a current load factor of 90%) and first calculates the ultrasonic propagation attenuation caused by excessive load (theoretically, ultrasonic energy attenuates by 5% for every 10% increase in load). It then compares the actual ultrasonic attenuation deviation in the feature. If the ultrasonic feature activation value in a winding region is 8% lower than the theoretical load attenuation, it is identified as an internal defect (e.g., an inter-turn short circuit causing enhanced ultrasonic reflection). The "defect feature enhancement" submodule then increases the feature activation value in that region from 0.5 to 0.8, generating the third-stage ultrasonic output feature. This feature flows into the third cross-correlation feature alignment component, where it is coupled with the thermal field compensation baseline feature (the defect thermal distribution stripped of load interference) output by the third-stage acoustic-thermal feature calibrator. This component weights the combined feature based on the temperature gradient of the thermal field defect region (e.g., a hotspot temperature of 130°C corresponds to an ultrasonic reflection enhancement factor of 1.2). This ultimately yields an acoustic-thermal coupling feature after multi-stage ultrasonic feature compensation. This feature deeply correlates the thermal field defect location with the ultrasonic reflection anomaly pattern in high-dimensional space (e.g., the activation value of the feature dimension of the thermal anomaly region is synchronized with the feature dimension of the ultrasonic defect reflection by 0.9). Through the coordinated operation of the cascaded second feature compensation component and the cross-correlation feature alignment component within the feature coupling unit, the server gradually strengthens the cross-modal correlation between the ultrasonic embedded features and the thermal field compensation benchmark features, so that the ultrasonic feature compensation at each stage not only retains the original ultrasonic structural information, but also incorporates the spatial prior after precise calibration of the thermal field, providing high-confidence multi-modal feature support for the final acoustic-thermal coupling imaging.

[0042] In an embodiment of the present invention, the feature coupling unit completed through training sequentially performs multi-stage ultrasonic feature compensation processing on the ultrasonic embedding features, which can be implemented through the following examples.

[0043] When performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the first second feature compensation component among the plurality of second feature compensation components is used to perform the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature to obtain a first-stage output feature of the ultrasonic reflection data after the first-stage ultrasonic feature compensation processing;

[0044] After performing target stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the target stage output feature after the target stage ultrasonic feature compensation processing and the thermal field compensation reference feature generated by the target first feature compensation component in the acoustic thermal feature calibrator are subjected to feature coupling by the target cross-correlation feature alignment component in the feature coupling unit to obtain a target feature coupling output;

[0045] When performing target-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, target-stage ultrasonic feature compensation processing is performed on the ultrasonic embedded feature and a preceding feature coupled output generated by a preceding cross-correlation feature alignment component by a target second feature compensation component among the multiple second feature compensation components, thereby obtaining a target-stage output feature of the ultrasonic reflection data after the target-stage ultrasonic feature compensation processing;

[0046] The output feature obtained after the last stage of the multi-stage ultrasonic feature compensation process is used as the acoustic-thermal coupling feature obtained after the multi-stage ultrasonic feature compensation process.

[0047] In an exemplary embodiment of the present invention, when performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedded features of a 110kV transformer winding in a substation, the server relies on a trained feature coupling unit (comprising three cascaded second feature compensation components and three cross-correlation feature alignment components) to achieve deep fusion of ultrasonic features and thermal field reference features according to the progressive logic of "stage compensation-coupling-recompensation". First, the first stage of ultrasonic feature compensation is performed: the server inputs the 512-dimensional ultrasonic embedded features encoding the original ultrasonic reflection information of the winding (covering structural information such as insulation layer interface reflections, multiple reflections within the winding, and mixed with noise interference caused by equipment vibration) into the first second feature compensation component of the feature coupling unit. This component uses a feature compensation operator consisting of "3×3 convolution + batch normalization + ReLU" to perform basic structure extraction on the ultrasonic embedded features: the 3×3 convolution kernel captures the amplitude mutation characteristics of the reflected wave at the "insulation layer-iron core" interface (for example, the feature activation value of the dimension corresponding to the reflection wave peak is enhanced from 0.3 to 0.6), batch normalization suppresses feature fluctuations caused by environmental electromagnetic noise (reducing the feature activation value in the noise area from 0.2 to 0.1), and ReLU enhances the nonlinear expression of weak reflection signals. Finally, the first-stage output features are output, which have initially highlighted the key structural information of ultrasonic reflection (for example, the activation degree of the feature dimension of the insulation layer interface is increased by 30%). When entering the second stage of ultrasonic feature compensation (target stage example), the process proceeds in two steps: First, post-target stage compensation coupling: The server first completes the "target stage ultrasonic feature compensation processing" through the second second feature compensation component, outputting the second stage output feature (a fusion of the original ultrasonic structure and the thermal field prior of the first stage coupling output). This feature then flows into the target cross-correlation feature alignment component (i.e., the second cross-correlation component) and is coupled with the second stage thermal field compensation reference feature (a winding thermal distribution corrected for infrared lens distortion) generated by the second first feature compensation component in the acoustic thermal feature calibrator. After merging the two features element-by-element in the energy dimension, the component adjusts the ultrasonic feature dimension activation weights based on the heat conduction characteristics of the heat sink fins (e.g., the temperature decay at the fin end corresponds to an ultrasonic reflection energy attenuation coefficient of 0.8). This optimizes the feature activation value at the ultrasonic reflection area at the fin end from 0.6 to 0.4, matching the theoretical attenuation), generating the second stage feature coupling output. Second, during target stage compensation processing: During the second stage of ultrasonic feature compensation, the server calls the second second feature compensation component, synchronously inputting the "ultrasound embedding feature" and the first stage feature coupling output (generated by the previous cross-correlation component).This component's feature compensation operator incorporates a "depthwise separable convolution + spatial attention fusion" mechanism: The depthwise separable convolution extracts the original ultrasonic structure (e.g., subtle reflection features between winding turns, with activation values ​​increased from 0.4 to 0.6) and spatial information after thermal field calibration (e.g., the ultrasonic feature weight corresponding to the hotspot location is increased from 0.3 to 0.7). The spatial attention fusion module strengthens the spatial correlation between the "thermal anomaly location" and the ultrasonic reflection anomaly (e.g., the overall ultrasonic feature activation value within 3mm of the hotspot is increased by 0.2), ultimately outputting the second-stage output features. For the third-stage ultrasonic feature compensation (the final stage of the multi-stage process), the server calls the third second-stage feature compensation component, inputting the "ultrasound embedding feature" and the coupled output of the second-stage features (generated by the preceding cross-correlation component). This component's feature compensation operator receives real-time transformer load data (e.g., a current load factor of 90%) and invokes a "load-ultrasonic propagation attenuation model" (for every 10% increase in load, the ultrasonic energy theoretically attenuates by 5%). The operator first calculates the ultrasonic attenuation caused by the overload (current load exceeds 10%, with a theoretical attenuation of 5%), then compares the actual attenuation deviation from the feature. (If the ultrasonic feature activation value in a particular winding region is 8% lower than the theoretical attenuation, it is determined to be an inter-turn short circuit causing enhanced reflection.) The "defect feature enhancement" submodule then increases the feature activation value in that region from 0.5 to 0.8, generating the third-stage output feature. The server uses this feature as the acoustic-thermal coupling feature after multi-stage ultrasonic feature compensation. This feature deeply correlates the thermal field defect location (e.g., a 130°C hotspot) with the ultrasonic reflection anomaly pattern (e.g., an additional peak with a feature dimension activation value of 0.9) in high-dimensional space, providing accurate multimodal correlation information for subsequent imaging reconstruction. Through the progressive logic of "stage compensation-coupling-recompensation" within the feature coupling unit, the server allows the ultrasonic features of each stage to retain the original structural information while incorporating the spatial prior after precise calibration of the thermal field. The final output of the acoustic-thermal coupling feature achieves deep synergy between the thermal field positioning capability and the ultrasonic structure analysis capability.

[0048] In an embodiment of the present invention, the plurality of second feature compensation components include: a second feature compensation component for performing dimensionality reduction processing on the ultrasonic embedding feature, and a second feature compensation component for performing dimensionality increase processing on the ultrasonic embedding feature;

[0049] Among them, the number of second feature compensation components used for dimensionality reduction processing of the ultrasonic embedded features is consistent with the number of second feature compensation components used for dimensionality increase processing of the ultrasonic embedded features; and the second feature compensation components used for dimensionality increase processing of the ultrasonic embedded features are arranged after the second feature compensation components used for dimensionality reduction processing of the ultrasonic embedded features.

[0050] In an embodiment of the present invention, for example, when performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedded features of a 110kV transformer winding in a substation, the feature coupling unit called by the server includes two second feature compensation components for dimensionality reduction and two second feature compensation components for dimensionality increase (the number of components of the two types is the same, and the dimensionality increase components are sequentially arranged after the dimensionality reduction components). Through the feature flow control logic of "dimensionality reduction to focus on core information - dimensionality increase to restore spatial structure", accurate purification and cross-modal fusion of ultrasonic information are achieved. First, the first dimensionality reduction second feature compensation component starts processing: the server inputs the 512-dimensional ultrasonic embedded features encoding the full-dimensional information of the winding ultrasonic reflection (including information such as insulation layer interface reflection, multiple reflections within the winding, and mixed with environmental electromagnetic noise and equipment vibration interference) into this component. The component has a built-in dimensionality reduction operator consisting of "1×1 convolution + max pooling." The 1×1 convolution kernel compresses the feature dimension from 512 to 256 dimensions (prioritizing the preservation of core structural information such as the insulation-core interface reflection and basic reflections between winding turns, increasing the activation percentage of feature dimensions corresponding to key reflections from 30% to 50%). The max pooling layer focuses on areas of strong reflection signals (reducing the activation value of weak feature dimensions dominated by noise from 0.2 to 0.1), ultimately outputting 256-dimensional reduced-dimensionality features. This process eliminates redundant noise dimensions through dimensionality reduction and strengthens the basic structural features of ultrasonic reflections (for example, the activation of the feature dimension of normal reflections between winding turns is increased by 20%). Next, a second dimensionality reduction and second feature compensation component takes over: using the 256-dimensional reduced-dimensionality features as input, the component invokes a dimensionality reduction operator consisting of "depthwise separable convolution + channel attention" to further compress the feature dimensions to 128. The deep separable convolution extracts fine-grained features of ultrasonic reflections by channel (for example, the activation value of the weak additional reflected wave feature caused by inter-turn short circuits is increased from 0.3 to 0.6). The channel attention module, based on prior information about the thermal field compensation benchmark features (such as the hotspot locations output by the acoustic thermal feature calibrator), strengthens the feature dimensions associated with the thermal field space (increasing the ultrasonic feature weight corresponding to the hotspot from 0.4 to 0.7), and outputs 128-dimensional deep dimensionality reduction features. This feature focuses on the core dimensions of "thermal field correlation + defect sensitivity," laying a solid foundation for accuracy in subsequent dimensionality increase fusion. Entering the first dimensionality increase second feature compensation component: the server inputs the 128-dimensional deep dimensionality reduction features into this component. The component uses a dimensionality increase operator consisting of "transposed convolution + residual connection" to restore the feature dimensions from 128 to 256 dimensions. The transposed convolution gradually restores the spatial distribution information of the ultrasonic reflection (such as reconstructing the spatial position characteristics of the insulation layer interface reflection, reducing the spatial coordinate error of the feature dimension to ±1 pixel); the residual connection retains the defect-sensitive features purified in the dimensionality reduction stage (such as the feature activation value corresponding to the inter-turn short-circuit reflection is stabilized from 0.6 to 0.65), and outputs a 256-dimensional up-dimensional feature, which not only restores the mesoscale spatial structure of the ultrasonic reflection, but also continues the enhancement effect of the defect information in the dimensionality reduction stage.Finally, the second dimensionality-upgraded second feature compensation component completes the final processing: using the 256-dimensional upscaled features as input, the component invokes a dimensionality-upgrade operator consisting of "deconvolution + spatial attention fusion" to restore the feature dimensionality to 512 dimensions. Deconvolution accurately restores the original dimensional scale of the ultrasonic embedded features (ensuring a 95% match between the global energy distribution of the winding's overall ultrasonic reflection and the original embedded features). The spatial attention fusion module deeply binds the thermal field compensation baseline features output by the acoustic-thermal feature calibrator (for example, the activation value corresponding to the hotspot region in the ultrasonic feature increases from 0.5 to 0.8), ultimately outputting a 512-dimensional upscaled feature. This feature, a key component of multi-stage ultrasonic feature compensation, preserves the full-dimensional structural information of the ultrasonic reflection while deeply incorporating the spatial correlation information after thermal field calibration, providing high-resolution and strongly correlated ultrasonic feature support for subsequent acoustic-thermal coupling and imaging reconstruction. Through the "noise removal + core focusing" of the dimensionality reduction component and the "structure restoration + correlation enhancement" of the dimensionality increase component, the server allows the ultrasonic embedded features to complete the progressive processing of "impurity filtering-defect purification-spatial reconstruction-cross-modal fusion" during the process of dimensional contraction and expansion, ensuring that the ultrasonic features at each stage can not only accurately respond to the internal defects of the equipment, but also deeply coordinate with the thermal field information, laying a solid technical foundation at the feature level for the accuracy and reliability of acoustic-thermal coupled imaging.

[0051] In an embodiment of the present invention, before performing the first stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the embodiment of the present invention further provides the following implementation manner.

[0052] Generate the simulated disturbance characteristics of the characteristic coupling unit through a preset acoustic and thermal disturbance simulation module;

[0053] Loading the simulated disturbance feature and the ultrasonic embedding feature as current processing features into the feature coupling unit;

[0054] The first second feature compensation component among the plurality of second feature compensation components performs a first stage ultrasonic feature compensation process on the ultrasonic embedded feature to obtain a first stage output feature of the ultrasonic reflection data after the first stage ultrasonic feature compensation process, comprising:

[0055] The first second feature compensation component performs first-stage ultrasonic feature compensation processing on the ultrasonic embedding feature and the simulated disturbance feature to obtain the first-stage output feature.

[0056] In an exemplary embodiment of the present invention, before performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedded features of a substation's 110kV transformer windings, the server first invokes a pre-defined acoustic-thermal disturbance simulation module (built on the transformer's digital twin model and integrating vibration, electromagnetic interference, and ambient temperature fluctuation models) to generate simulated disturbance features. This module simulates "ultrasonic signal phase shift caused by equipment vibration" (a vibration frequency of 50Hz causes a phase shift of ±5μs in the reflected wave peak in the ultrasonic A-scan data), "ultrasonic amplitude noise caused by environmental electromagnetic interference" (noise intensity fluctuates by ±0.1 in the feature dimension activation value), and "thermal field gradient changes caused by a sudden drop in ambient temperature of 5°C" (a decrease in the activation value of the radiator region temperature feature in the thermal image by 0.2). These interference patterns are encoded into simulated disturbance features of the same dimensionality (512) as the ultrasonic embedded features, accurately reproducing the characteristic patterns of complex on-site interference. The server then merges the simulated perturbation feature with the 512-dimensional ultrasonic embedded feature (including real structural information such as insulation interface reflection and internal winding reflection) acquired from the transformer's ultrasonic transducer through feature mapping to form the "current processing feature" and simultaneously loads it into the feature coupling unit. This current processing feature now carries both the original ultrasonic structural information and incorporates the interference pattern of the simulated perturbation, simulating the state of ultrasonic data under complex field conditions. Entering the first stage of ultrasonic feature compensation, the first second feature compensation component of the feature coupling unit (with a built-in "adversarial training convolution + attention denoising" operator) receives the "current processing feature" and collaboratively compensates the ultrasonic embedded feature with the simulated perturbation feature. The adversarial training convolution layer first learns the pattern of the perturbation feature (e.g., the periodic fluctuation of the feature dimension corresponding to the vibration phase offset) and then counteracts the perturbation through inverse weight adjustment (correcting the feature activation deviation caused by the phase offset from ±0.2 to ±0.05). The attention denoising module focuses on the original ultrasonic structural features (e.g., increasing the activation value of the feature dimension corresponding to the insulation interface reflection from 0.4 to 0.7, while suppressing the activation value of the feature dimension corresponding to electromagnetic noise from 0.3 to 0.1). After this processing, the component outputs the first-stage output features, which have initially removed simulated disturbance interference (vibration phase offset correction rate reaches 90%, electromagnetic noise suppression rate reaches 70%), while retaining the core structural information of ultrasonic reflection (the activation of the feature dimension of normal reflection between winding turns increases by 25%). By introducing simulated disturbance features before the first-stage compensation and having the components collaboratively process them, the server simulates the strong interference environment on site, enabling the feature coupling unit to learn the dual capabilities of "interference rejection and feature purification." This ensures that the output first-stage features are both resistant to real-world disturbances and accurately preserve ultrasonic structural information, laying a solid foundation for the robustness of subsequent multi-stage compensation and acoustic-thermal coupling.

[0057] In an embodiment of the present invention, the characteristic dimension of the target stage output feature is consistent with the characteristic dimension of the thermal field compensation reference feature generated by the target first feature compensation component;

[0058] The target cross-correlation feature alignment component in the feature coupling unit is used to perform feature coupling on the target stage output feature after the target stage ultrasonic feature compensation processing and the thermal field compensation reference feature generated by the target first feature compensation component in the acoustic thermal feature calibrator to obtain the target feature coupling output, which can be implemented through the following examples.

[0059] By using the target mutual correlation feature alignment component in the feature coupling unit, a feature merging process is performed on the target stage output feature and the thermal field compensation reference feature generated by the target first feature compensation component in the energy dimension to obtain a merged feature;

[0060] Performing feature compression processing on the merged features to obtain compressed features;

[0061] performing a convolution operation on the compressed feature to obtain an acoustic-thermal coupling modulation coefficient between the target stage output feature and the thermal field compensation reference feature generated by the target first feature compensation component;

[0062] A feature coupling processing calculation is performed on the combined feature based on the acoustic-thermal coupling modulation coefficient to obtain the target feature coupling output.

[0063] In an embodiment of the present invention, for example, when performing the second-stage ultrasonic feature compensation processing on the 110kV transformer winding of the substation, the server ensures that the target stage output feature output by the feature coupling unit (the ultrasonic feature after processing by the second feature compensation component, with a dimension of 256 dimensions) is consistent with the feature dimension of the thermal field compensation reference feature (the winding thermal distribution feature after correcting the lens distortion, with a dimension of 256 dimensions) generated by the target first feature compensation component (the second intermediate layer) in the acoustic thermal feature calibrator, and then performs feature coupling through the target cross-correlation feature alignment component: First, the energy dimension is merged: the server calls the target cross-correlation feature alignment component to add the 256-dimensional second-stage ultrasonic output feature (encoding structural information such as winding inter-turn reflection and insulation layer interface reflection, and incorporating the thermal field prior of the first-stage coupling output) and the 256-dimensional second-stage thermal field compensation reference feature (the thermal distribution distortion caused by the tilt of the infrared lens has been corrected, accurately describing the heat conduction law of the radiator fins) element by element in the energy dimension. For example, the activation value of the feature dimension corresponding to the "winding hotspot region" in the ultrasonic feature is 0.7 (reflecting the energy intensity of the ultrasonic reflection wave in this region), while the activation value of the same region in the thermal field feature is 0.8 (reflecting the energy weight of the thermal field temperature distribution). These values ​​are added together to generate a merged feature (with an activation value of 1.5 for the corresponding dimension, unnormalized), achieving a preliminary fusion of acoustic and thermal energy information. Next, feature compression is performed: the component simultaneously performs "peak feature compression" and "mean feature compression" on the merged feature. Peak compression focuses on features that strongly correlate with the ultrasonic reflection peak and the high-temperature region of the thermal field. For example, the activation value of the dimension corresponding to the ultrasonic reflection peak is compressed from 1.5 to 0.9 (preserving the core information of the high-energy reflection and suppressing weakly correlated noise). Mean compression smoothes background interference. For example, the activation value of the feature in the non-hotspot region of the radiator is compressed from 0.3 to 0.2 (reducing spurious feature fluctuations caused by ambient wind disturbances, electromagnetic noise, etc.). The resulting compressed feature not only enhances the information in the region with strong acoustic and thermal correlations, but also reduces background interference, providing a clean input for the subsequent modulation coefficient calculation. Then, a convolution operation calculates the modulation coefficient: the component uses a 1×1 convolution kernel to perform a convolution operation on the compressed features, exploring the coupling patterns of the acoustic and thermal features. Based on the heat conduction model of the radiator fins in the thermal field (for every 1°C decrease in the temperature at the end of the fin, the ultrasonic reflection energy attenuates by 2%), the convolution layer learns the correlation pattern of "thermal field temperature gradient-ultrasonic energy attenuation" and outputs the acoustic and thermal coupling modulation coefficient (such as 0.8, which represents the modulation intensity of the thermal field on the ultrasonic feature). This coefficient quantifies the degree to which the spatial distribution of the thermal field affects the ultrasonic reflection feature: if the temperature in a certain area of ​​the thermal field is 15°C higher than the normal operating conditions, the modulation coefficient will increase by 0.15, indicating that the ultrasonic feature needs to strengthen the reflection anomaly analysis in this area. Finally, the feature coupling processing calculation: the server performs a weighted adjustment on the merged features based on the acoustic and thermal coupling modulation coefficient.Taking the dimension "winding hotspot - ultrasonic reflection enhancement" in the merged features as an example, the merged feature activation value is 1.5, which, when multiplied by a modulation coefficient of 0.8, becomes 1.2. This dimension preserves the original acoustic-thermal energy correlation while enhancing defect-sensitivity characteristics based on thermal field priors. After weighting across all dimensions, the component outputs the target feature coupled output. In this feature, prior information such as the thermal field's temperature distribution and spatial location is deeply integrated into the ultrasonic reflection features in 256-dimensional space. For example, the ultrasonic feature activation value in the hotspot region increases from 0.7 to 0.9 (corresponding to enhanced ultrasonic reflection, indicating internal defects). This provides accurate input for the next stage of ultrasonic feature compensation, combining acoustic-thermal correlation and defect sensitivity. Through a progressive process of "energy merging - feature compression - coefficient calculation - weighted coupling," the server achieves precise alignment and information fusion of the target ultrasonic and thermal field features in high-dimensional space. This preserves the structural details of the ultrasonic reflection while incorporating the spatial and physical priors of the thermal field, laying a solid technical foundation for cross-modal correlation accuracy in multimodal imaging.

[0064] In the embodiment of the present invention, the feature compression processing is performed on the merged features to obtain compressed features, which can be implemented through the following examples.

[0065] Performing peak feature compression processing and mean feature compression processing on the merged features respectively, to obtain peak compression features and mean compression features respectively;

[0066] The peak compression feature and the mean compression feature are used as the compression features.

[0067] In an embodiment of the present invention, exemplarily, when performing feature compression processing on the second-stage merged features of the 110kV transformer winding of the substation, the server calls the dual-path compression module in the target cross-correlation feature alignment component to simultaneously perform peak feature compression and mean feature compression: First, peak feature compression processing: the component starts a threshold-constrained compression operator for the strong correlation dimension between the ultrasonic reflection peak and the high temperature area of ​​the thermal field in the merged features (256 dimensions, integrating the ultrasonic reflection structure and the thermal field distribution energy information) (such as the feature dimension of the ultrasonic peak corresponding to the winding hot spot). For example, the 100th dimension in the merged feature (corresponding to the superposition of ultrasonic peaks and high thermal field temperatures in the winding hotspot region) has an original activation value of 1.5 (the ultrasonic peak contributes 0.7 energy, and the high thermal field contributes 0.8 energy). The peak compression operator sets an activation threshold of 0.8 and compresses any values ​​exceeding the threshold by a factor of 0.6 (1.5 × 0.6 = 0.9). This preserves the core energy information in strongly correlated regions while suppressing weakly correlated noise dimensions (such as false peaks caused by environmental electromagnetic interference, whose activation value is reduced from 0.4 to 0.2). The resulting peak compression feature focuses on the activation values ​​of the strongly correlated dimensions between the winding hotspot and the ultrasonic peak in the range of 0.7-1.0, precisely enhancing defect-sensitive information. Next, the mean feature compression component applies a local mean pooling compression operator to background dimensions in the merged feature, such as those in the heat sink's non-hotspots and normal winding regions (e.g., the feature dimension at the non-end of the heat sink fin). Taking the 200th dimension in the merged feature as an example (corresponding to the thermal field and ultrasonic reflection in the non-hotspot area of ​​the radiator), its original activation value fluctuates between 0.2 and 0.4 (environmental wind disturbance causes temperature fluctuations in the thermal field, and electromagnetic noise causes ultrasonic amplitude jitter). The mean compression operator calls a 3×3 local mean pooling kernel to calculate the mean of this dimension and adjacent dimensions ((0.2+0.3+0.4) / 3=0.3), uniformly correcting the activation values ​​of the fluctuating dimensions to 0.3, smoothing feature fluctuations caused by background interference, and outputting a mean-compressed feature. In this feature, the feature activation values ​​of the background area are stabilized in the range of 0.2-0.4, effectively suppressing spurious features caused by environmental interference. Finally, the server concatenates the peak-compressed feature and the mean-compressed feature by dimension to generate a compressed feature that combines strong correlation information focus with background interference smoothing. In this feature, the dimensional activation value of the ultrasonic peak corresponding to the winding hotspot remains at 0.9 (after peak compression), while the dimensional activation value of the non-hotspot area of ​​the heat sink remains stable at 0.3 (after mean compression). This provides a clean and layered feature input for the subsequent accurate calculation of the acoustic-thermal coupling modulation coefficient, ensuring that the correlation between the thermal field temperature gradient and ultrasonic energy attenuation can be efficiently learned by the convolutional layer. Through a two-way compression mechanism, the server allows the merged features to retain defect-sensitive information while stripping away background noise, laying a solid foundation for feature purity at the feature level for accurate acoustic-thermal coupling.

[0068] In an embodiment of the present invention, the feature representation learning is performed on the thermal radiation data and the ultrasonic reflection data respectively to obtain thermal radiation embedding features and ultrasonic embedding features respectively, which can be implemented through the following examples.

[0069] Calling an ultrasonic feature mapper to perform feature mapping processing on the ultrasonic reflection data to obtain the ultrasonic embedding feature;

[0070] A thermal feature mapper is called to perform feature mapping processing on the thermal radiation data to obtain the thermal radiation embedding feature.

[0071] In an embodiment of the present invention, for example, in a condition monitoring scenario of a 110kV transformer in a substation, the server performs feature representation learning on ultrasonic reflection data and thermal radiation data respectively by calling a dedicated feature mapper to generate high-dimensional embedded features: when processing ultrasonic reflection data, the server calls an ultrasonic feature mapper (composed of a cascade of a 1D convolutional layer and a gated recurrent unit (GRU)). For the ultrasonic A-scan data of the transformer winding (center frequency 5MHz, sampling rate 20MHz, duration 50μs, sequence length 10,000 points, including information such as insulation layer-core interface reflection and internal winding defect reflection), the 1D convolutional layer first uses 32 1×16 convolution kernels to segmentally extract time domain features such as "amplitude decay rate and peak time interval." For example, the activation value of the feature dimension corresponding to the insulation layer interface reflection peak is enhanced from 0.2 to 0.5, accurately capturing the interface location. The GRU layer mines long-range dependencies along the time dimension to identify the association of multiple reflection waves caused by inter-turn short circuits (increasing the activation value of the feature dimension corresponding to the defect reflection wave sequence by 0.3). Finally, the fully connected layer compresses the feature dimension to 512 dimensions and outputs the ultrasonic embedding feature. This feature encodes key information such as energy loss during ultrasonic propagation (for example, energy attenuation due to insulation aging results in a 0.2 decrease in the corresponding dimension activation value), the number of reflective interfaces (normally, the activation value is 0.6 when there are three interfaces, increasing to 0.8 when a defect adds an interface), and waveform distortion caused by defects (short circuits cause peak broadening, resulting in a change in the corresponding dimension activation value from 0.4 to 0.7). This provides a structured representation of the ultrasonic dimension for subsequent cross-modal coupling. When processing thermal radiation data, the server invokes a thermal feature mapper (based on a modified ResNet architecture with four convolutional blocks and two fully connected layers). For the infrared thermal image of the transformer winding (resolution 1024×768, pixel grayscale mapping -40°C to 500°C, covering the thermal distribution of the winding, core, and radiator), the first convolution block uses a 3×3 convolution kernel to extract local thermal patterns such as "winding-core temperature boundary, radiator fin thermal gradient", etc. For example, the activation value of the feature dimension corresponding to the pixels in the winding hotspot area is increased from 0.3 to 0.6, highlighting the temperature anomaly; the subsequent convolution block strengthens the features of the thermal anomaly area through residual connections (for example, the feature response of the inter-turn short-circuit hotspot is enhanced from 0.5 to 0.85, while retaining the basic information of the normal thermal distribution of the core); finally, the fully connected layer compresses the feature dimension to 512 dimensions and outputs the thermal radiation embedding feature. This feature encodes key information such as the temperature distribution of the thermal field (a 2°C temperature difference in the three-phase winding corresponds to a dimension activation value difference of 0.1), the direction of the thermal gradient (the longitudinal heat conduction of the radiator corresponds to a dimension activation value of 0.7), and the spatial location of the thermal anomaly (the hotspot coordinate (300, 200) corresponds to a dimension activation value of 0.9), providing accurate representation of the thermal field dimension for subsequent thermal radiation feature compensation.Through hierarchical feature extraction using a dedicated feature mapper, the server abstracts the original multi-dimensional data of ultrasound and thermal radiation into compact features in a high-dimensional embedding space, retaining the core information of the equipment status (such as defect location and energy loss) while providing feature input of a unified dimension (512 dimensions) for subsequent multi-stage compensation and cross-modal coupling, ensuring that the acoustic and thermal modes have interactivity and a basis for collaborative analysis at the feature level.

[0072] In an embodiment of the present invention, the thermal field compensation reference feature includes a thermal radiation feature representation for indicating the thermal field distribution feature of the target device area; the thermal field distribution feature includes: thermal anomaly area position, thermal anomaly morphology, and device area outline.

[0073] In an embodiment of the present invention, for example, in the multi-stage thermal radiation characteristic compensation process of the 110kV transformer winding of the substation, the server accurately carries the thermal radiation characteristic characterization of the target equipment area through the thermal field compensation reference characteristics output by the acoustic thermal characteristic calibrator, and the characterization focuses on the quantitative expression of three types of thermal field distribution characteristics: the location of the thermal anomaly area, the thermal anomaly morphology, and the equipment area contour. Regarding the first-stage thermal field compensation benchmark features (after eliminating environmental wind disturbance): When the server processes the thermal radiation embedded features, the first first feature compensation component of the acoustic thermal feature calibrator uses spatial attention to enhance the hotspot features on the upper part of winding phase A (the temperature in the infrared thermal image of this area reaches 120°C, exceeding the normal threshold of 30°C), so that the activation value of the dimension corresponding to the "thermal anomaly area position" in the thermal radiation feature representation reaches 0.9, and the hotspot is accurately located at the pixel coordinates (200,150) in the thermal image coordinate system; the "thermal anomaly morphology" dimension is encoded as a circular feature with a diameter of approximately 10 pixels (corresponding to a heat accumulation area with a diameter of 3 cm in actual space) with an activation value of 0.85; the "device area outline" dimension clearly distinguishes the boundaries of the winding (pixel activation value 0.8), the iron core (0.6), and the radiator (0.5). By allocating spatial weights of the feature dimensions, the thermal distribution outlines of each component of the device are restored. At this stage, the features have preliminarily removed the interference of environmental wind disturbance on the heat exchange of the radiator, making the thermal field distribution features closer to the intrinsic thermal state of the device. Entering the second stage of thermal field compensation benchmark features (after correcting for infrared lens distortion): The server invokes the second first feature compensation component of the acoustic thermal feature calibrator. Combined with prior information about the transformer's heat dissipation structure (heat sink fin spacing of 2 cm and thermal conductivity of 200 W / (m·K)), it corrects for thermal distribution distortion caused by lens tilt in the previous stage's features. In the thermal radiation feature representation, the "thermal anomaly area location" is corrected to more precise pixel coordinates (205, 152) (the position error is reduced from ±5 pixels to ±2 pixels). Due to the heat conduction constraints of the heat sink structure, the "thermal anomaly shape" evolves from a circle to an ellipse with a major axis of 12 pixels and a minor axis of 8 pixels (corresponding to the physical law that heat diffuses longitudinally and is confined laterally along the fins). The activation value is adjusted to 0.82. The "device area outline" further enhances the detailed features of the heat sink fins, with the pixel activation value at the fin gap increased from 0.3 to 0.4, restoring the true gradient of heat exchange between the fins. At this stage, the features are fully aligned with the thermal characteristics of the device's physical structure. In the third stage, the thermal field compensation benchmark characteristics (after integrating into the load curve) are: the server calls the third first characteristic compensation component of the acoustic thermal characteristic calibrator, accesses the real-time load data of the transformer (load rate is 90%, exceeding the normal upper limit by 10%), and distinguishes between "load overheating" and "defective overheating" through the load-heat dissipation model.In the characterization of thermal radiation characteristics, after the "thermal anomaly area location" was corrected for load temperature rise, it was confirmed that the hotspot was caused by an inter-turn short circuit (rather than excessive load). The activation value of the position dimension remained at 0.9, and the coordinates were stable at (205,152). The "thermal anomaly morphology" evolved into an irregular polygon with an activation value of 0.88 due to the heat diffusion of internal defects (arc heating at the short-circuit point caused thermal decomposition of the surrounding insulation layer). The "equipment area contour" dimension combined with the overall thermal distribution under load made the thermal gradient characteristics of the three phases of the winding more accurate. The temperature difference between phase A (hotspot phase) and phases B and C corresponded to the activation value difference of the feature dimension reduced from 0.3 to 0.2, clearly presenting the full picture of the thermal field under the combined action of load and defects. At this stage, the characteristics have restored the thermal field anomaly caused by internal defects of the equipment to the greatest extent, providing a high-confidence thermal distribution prior for subsequent acoustic-thermal coupling. Through multi-stage thermal radiation feature compensation, the thermal field compensation benchmark features output by the server use thermal radiation feature representation as a carrier to hierarchically analyze the dynamic changes in the position, morphology and equipment area contours of the thermal anomaly area, which not only supports the spatial correlation of ultrasonic features, but also provides the core basis for the thermal field dimensional accuracy of the final imaging.

[0074] In an embodiment of the present invention, the method is implemented by a pre-trained multimodal imaging model; the multimodal imaging model includes a core imaging unit and a thermal feature calibration unit; the core imaging unit includes at least an ultrasonic feature mapper and a feature coupling unit, and the thermal feature calibration unit includes a thermal feature mapper and an acoustic thermal feature calibrator;

[0075] The multimodal imaging model is obtained by the following method and can be implemented through the following examples.

[0076] Loading a thermal radiation data instance in a preset training set into the thermal feature calibration unit, performing feature mapping processing on the thermal radiation data instance through a thermal feature mapper in the thermal feature calibration unit to obtain a thermal radiation embedding feature of the thermal radiation data instance;

[0077] Performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedded features of the thermal radiation data instance by the acoustic thermal feature calibrator in the thermal feature calibration unit, and correspondingly obtaining thermal field compensation reference features of the thermal radiation data instance of the target device area instance in the thermal radiation data instance under multiple feature compensation stages;

[0078] Loading an ultrasonic reflection data instance in a preset training set into the core imaging unit, performing feature mapping processing on the ultrasonic reflection data instance through an ultrasonic feature mapper in the core imaging unit to obtain a sample ultrasonic embedding feature;

[0079] Performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding feature of the sample in sequence by a feature coupling unit in the core imaging unit to obtain an acoustic-thermal coupling feature instance;

[0080] performing error calculation on the multimodal imaging model based on the thermal field compensation benchmark feature of the thermal radiation data instance and the acoustic-thermal coupling feature instance to obtain an error value of the multimodal imaging model;

[0081] The architecture parameters in the multimodal imaging model are tuned based on the error value to obtain a trained multimodal imaging model.

[0082] In an embodiment of the present invention, exemplarily, when training a multimodal imaging model, the server constructs a preset training set (including infrared thermal images marked with thermal anomalies, ultrasonic A-scan data marked with defects, and real fault labels) using historical monitoring data of a 110kV transformer in a substation, and completes model training according to the "thermal feature calibration-ultrasonic feature mapping-cross-modal coupling-error tuning" logic: the server first loads a thermal radiation data instance of a transformer winding in the training set (such as an infrared thermal image with a resolution of 1024×768, marked with the hot spot position of the A-phase winding (200,150), and a temperature of 120°C) into the thermal feature calibration unit. The thermal feature mapper in the thermal feature calibration unit (an improved ResNet architecture with four convolutional blocks and two fully connected layers) performs feature mapping on the thermal image: the first convolution block uses a 3×3 convolution kernel to extract the "winding-core" temperature boundary (the activation value of the feature dimension corresponding to the pixels in the hotspot area is enhanced from 0.3 to 0.6). The subsequent residual block retains the global distribution of the thermal field and enhances the thermal anomaly characteristics through cross-layer connections (the feature response of the inter-turn short-circuit hotspot is improved from 0.5 to 0.85), and finally outputs the thermal radiation embedding features of a 512-dimensional thermal radiation data instance (encoding core information such as the spatial location of the hotspot, temperature gradient, and the outline of equipment components). Next, the acoustic thermal feature calibrator (consisting of three cascaded first feature compensation components, each containing a residual module and a spatial attention mechanism) performs multi-stage compensation on the embedded feature: the first-stage component corrects the ambient wind disturbance through the spatial attention mechanism (the false high-temperature pixels of the radiator caused by airflow, whose feature activation value is suppressed from 0.6 to 0.2), and outputs the first-stage thermal field compensation baseline feature; the second-stage component combines the transformer heat dissipation structure prior (such as the radiator fin arrangement spacing of 2cm, the thermal conductivity of the aluminum fin is 200W / (m·K)), to correct the thermal distribution distortion caused by the tilt of the infrared lens. The hotspot coordinates are precisely corrected from (200, 150) to (205, 152), with the position error reduced to ±2 pixels. The second-stage thermal field compensation benchmark features are then output. In the third stage, the real-time load curve of the transformer connected to the component (90% load rate in the training set) is used to distinguish between "normal heating due to excessive load" and "abnormal heating due to internal defects." The load-heat dissipation model confirms that the hotspot is caused by an inter-turn short circuit, with the feature activation value remaining stable at 0.9. The third-stage thermal field compensation benchmark features are then output. These three-stage features gradually remove environmental, device structure, and operating condition interference, accurately restoring the device's intrinsic thermal state. Simultaneously, the server loads the corresponding ultrasonic reflection data instances in the training set (e.g., an ultrasonic A-scan sequence with a center frequency of 5 MHz, a sampling rate of 20 MHz, and a duration of 50 μs, with the inter-turn short circuit defect location marked) into the core imaging unit.The ultrasonic feature mapper in the core imaging unit (composed of a cascade of a 1D convolutional layer and a gated recurrent unit (GRU)) performs feature mapping on the A-scan sequence: the 1D convolutional layer uses 32 1×16 convolution kernels to segmentally extract time-domain features such as "insulation layer-core interface reflection peak amplitude and energy decay rate" (additional reflection peaks caused by defects have their corresponding feature dimension activation value enhanced from 0.3 to 0.6); the GRU layer mines the long-range dependencies of the reflected waves along the time dimension (multiple reflection wave sequences caused by inter-turn short circuits have their feature dimension activation increased by 0.3); and finally outputs 512-dimensional sample ultrasonic embedding features (encoding core information such as ultrasonic reflection structure, defect location, and energy loss pattern). Subsequently, the feature coupling unit (consisting of three cascaded second feature compensation components and three cross-correlation feature alignment components) performs multi-stage compensation on the sample ultrasonic embedded features: the second feature compensation component in each stage receives the "coupled output of the ultrasonic embedded feature + the previous cross-correlation component" and performs cross-modal coupling with the thermal field compensation baseline feature of the corresponding stage (for example, the second-stage component combines with the second-stage thermal field compensation baseline feature to strengthen the spatial correlation of the "hotspot area-ultrasonic reflection enhancement area", and the feature dimension activation value is increased from 0.7 to 0.9). Finally, it outputs an acoustic-thermal coupling feature instance, which deeply integrates the thermal field spatial prior and ultrasonic structural information in the 512-dimensional space. For example, the acoustic-thermal feature dimension activation value of the inter-turn short-circuit area is synchronized to 0.9, realizing the precise correlation between the defect location and thermal anomaly. The server calculates model errors based on thermal field compensation benchmark features and acoustic thermal coupling feature instances using thermal radiation data. The server calculates the "thermal field consistency error" (the difference in thermal field representation between the thermal field compensation benchmark features and the acoustic thermal coupling features, such as the mean absolute error (MAE) between the predicted hotspot temperature and the actual annotated temperature) and the "feature alignment error" (the deviation in the cross-correlation coefficients of the acoustic thermal feature dimensions, such as the difference between the Pearson correlation coefficient of the acoustic thermal feature in the short-circuit region and the theoretical correlation coefficient). The total model error is calculated by weighted summing these two types of errors. Finally, the server uses a backpropagation algorithm to tune architectural parameters such as the convolution kernel weights of the thermal feature mapper, the spatial attention parameters of the acoustic thermal feature calibrator, the GRU gating weights of the ultrasonic feature mapper, and the cross-correlation coefficients of the feature coupling units based on the total error. It iterates until the error converges (for example, the MAE of thermal field temperature prediction decreases from 5°C to 1°C, and the deviation of the acoustic thermal feature cross-correlation coefficient narrows from 0.2 to 0.05). Finally, a fully trained multimodal imaging model is obtained. This model has learned strong robustness in the "thermal feature purification - ultrasonic feature structuring - cross-modal precise coupling" process. After deployment, it can efficiently process acoustic thermal data under complex interference on-site and output high-confidence multimodal imaging results. Through multiple rounds of iteration of the training set, the server enables the model to accurately correlate thermal field anomalies with ultrasonic defects under typical interference in industrial scenarios (such as ambient wind, lens distortion, and load fluctuations), providing core algorithmic support for multimodal detection in intelligent equipment operation and maintenance.

[0083] In an embodiment of the present invention, the error calculation of the multimodal imaging model is performed based on the thermal field compensation benchmark feature of the thermal radiation data instance and the acoustic-thermal coupling feature instance to obtain the error value of the multimodal imaging model, which can be implemented through the following examples.

[0084] Based on the acoustic-thermal coupling feature instance and the thermal radiation data instance, performing consistency error calculation on the multimodal imaging model to obtain a consistency error value;

[0085] Based on the thermal field compensation benchmark feature of the thermal radiation data instance and the acoustic-thermal coupling feature instance, performing thermal field feature calibration error calculation on the multimodal imaging model to obtain a thermal field feature calibration error value;

[0086] The sum of the consistency error value and the thermal field feature calibration error value is used as the error value of the multimodal imaging model.

[0087] In an embodiment of the present invention, during the training of a multimodal imaging model, the server calculates the model error based on the logical chain of "consistency error - thermal field feature calibration error - total error" for the training data of a 110kV transformer in a substation. The following is an explanation of this using a specific training example: the server calls the trained imaging reconstruction module to reconstruct the acoustic-thermal coupling feature instance (a 512-dimensional feature after multi-stage ultrasonic compensation and thermal field coupling, encoding the correlation between the thermal anomaly and ultrasonic reflection of the transformer winding inter-turn short circuit) into an acoustic-thermal coupling imaging result instance (a multimodal image with a resolution of 1024×768, integrating pseudo-color thermal distribution and grayscale ultrasonic reflection). Simultaneously, the server retrieves the corresponding thermal radiation data instance in the training data set (an infrared thermal image with the hotspot temperature of the phase A winding marked as 120°C and the pixel coordinates (205,152)). The server calculates the consistency error between the two using the mean absolute error (MAE): the pixel temperature values ​​of the acoustic-thermal coupling imaging result instance and the thermal radiation data instance are traversed. For the hot spot area of ​​the phase A winding (3×3 pixel neighborhood), the pixel temperature of the thermal radiation data instance is [120, 118, 119, 121, 117, 122, 119, 120, 118] (°C), and the predicted temperature of the acoustic-thermal coupling imaging result instance is [118, 117, 119, 120, 116, 121, 118, 119, 117] (°C). The pixel-by-pixel absolute error (MAE) is calculated and averaged: [MAE = [|120-118|+|118-117|+...+|118-117|] / 9 = [2+1+0+1+1+1+1+1+1] / 9 ≈ 1.11]. This value is the consistency error, reflecting the accuracy of the thermal field matching between the acoustic thermal coupling imaging and the original thermal data. The server extracts thermal field feature vectors (e.g., sub-vectors encoding the location and morphology of the thermal anomaly) from the thermal field compensation baseline features (output from the third stage, corrected for load and structural interference, and accurately encoding the 512-dimensional features of the inter-turn short-circuit thermal anomaly) and the acoustic thermal coupling feature instances (512-dimensional, integrating thermal field calibration information and ultrasonic defect information) from the thermal radiation data instances. Take the location of thermal anomalies as an example: In the thermal field compensation benchmark feature, the pixel coordinates of the hotspot of phase A winding are encoded as a vector (v1=[205,152]) (corresponding to the thermal image coordinate system); in the acoustic thermal coupling feature example, the predicted encoding vector of the hotspot location is (v2=[203,150]). The server calculates the Euclidean distance between the two as the feature alignment deviation: d= , combined with the cosine similarity deviation of the thermal anomaly shape (an irregular polygon in the thermal field compensation benchmark feature, coded by the vector (m1=[0.88, 0.75, 0.92]); an approximate polygon in the acoustic-thermal coupling feature instance, coded by the vector (m2=[0.85, 0.72, 0.90])), the thermal field feature calibration error value is obtained by weighted summation (for example, the comprehensive deviation is approximately 0.35). The server calculates the weighted sum of the consistency error (1.11) and the thermal feature calibration error (0.35) (the weights are set based on the scenario's requirements for thermal accuracy and imaging consistency, for example, 0.5 for each). This yields the error of the multimodal imaging model: [Total Error = 1.11 × 0.5 + 0.35 × 0.5 = 0.73]. Using the backpropagation algorithm, the server uses this total error to adjust architectural parameters, such as the convolution kernel weights of the thermal feature mapper (for example, enhancing feature extraction capabilities in hotspot regions), the spatial attention parameters of the acoustic thermal feature calibrator (for example, improving the fusion accuracy of the heat dissipation structure prior), and the cross-correlation coefficients of the feature coupling units (for example, enhancing the correlation strength between thermal and ultrasonic features). The server iteratively minimizes the total error until the model achieves a thermal temperature prediction MAE of ≤1°C and a thermal feature alignment error of ≤0.1 on the training set, completing model training. By synergistically constraining the two-dimensional error, the server ensures that the multimodal imaging model adheres to the physical reality of the original thermal data while accurately aligning with the defect features after thermal field compensation. This ensures algorithmic reliability for multimodal, precise detection of equipment faults in industrial scenarios.

[0088] In the embodiment of the present invention, the consistency error calculation of the multimodal imaging model based on the acoustic-thermal coupling feature instance and the thermal radiation data instance to obtain the consistency error value can be implemented through the following examples.

[0089] Performing imaging reconstruction on the acoustic-thermal coupling feature instance to obtain an acoustic-thermal coupling imaging result instance including the thermal distribution feature of the target device area instance and the reflection spectrum characteristics of the ultrasonic reflection data instance;

[0090] The mean absolute error between the acoustic-thermal coupling imaging result instance and the thermal radiation data instance is used as the consistency error value.

[0091] In an exemplary embodiment of the present invention, during the consistency error calculation phase of the multimodal imaging model, the server operates on a training set of data from a 110kV transformer in a substation, following the "acoustic-thermal coupling feature reconstruction imaging - pixel-level temperature deviation statistics" process. First, the server invokes a pre-set imaging reconstruction network (based on a modified U-Net architecture, including encoding-decoding paths and skip connections) to perform imaging reconstruction on an acoustic-thermal coupling feature instance (a 512-dimensional feature obtained through multi-stage ultrasonic compensation and thermal field coupling, encoding core information such as the thermal anomaly location and enhanced ultrasonic reflection areas of the transformer's phase A winding interturn short circuit). During the reconstruction process, the network decodes the thermal distribution information using a "blue to red" pseudo-color mapping (from low to high temperature) and decodes the ultrasonic reflection information using a grayscale mapping (higher brightness, greater reflection intensity). The resulting output is an acoustic-thermal coupling imaging result instance (resolution 1024×768). In this image, the hotspot area of ​​the phase A winding appears as a dark red patch (corresponding to a temperature of approximately 120°C), with a bright white ultrasonic reflection patch (corresponding to the strong reflection caused by the short circuit) superimposed, visually demonstrating the defect correlation between the thermal and acoustic modalities. Next, the server retrieves the corresponding thermal radiation data instance in the training set (the infrared thermal image of the hotspot of phase A winding is marked, and the pixel temperatures of the 3×3 pixel neighborhood of the hotspot are marked as [120, 118, 119, 121, 117, 122, 119, 120, 118]°C). For the pixel-level temperature information of the acoustic-thermal coupling imaging result instance and the thermal radiation data instance, the server calculates the absolute error for each pixel. For example, if the predicted temperature of the corresponding 3×3 area in the acoustic-thermal coupling imaging is [118, 117, 119, 120, 116, 121, 118, 119, 117]°C, the absolute errors of each pixel are |120-118|=2, |118-117|=1, |119-119|=0, |121-120|=1, |117-116|=1, |122-121|=1, |119-118|=1, |120-119|=1, and |118-117|=1. The server sums these absolute errors and takes the average value ((2+1+0+1+1+1+1+1+1)÷9≈1.11). This average value is the consistency error value, which is used to quantify the matching accuracy of the acoustic-thermal coupling imaging results with the original thermal radiation data in the thermal field dimension. The smaller the error value, the closer the thermal distribution reconstructed by the model is to the actual thermal radiation state. Through pixel-level temperature deviation statistics, the server accurately measures the consistency between the acoustic-thermal imaging and the original thermal data, providing key feedback for subsequent model tuning: if the consistency error value is too high (such as >2°C), the decoding layer weights of the imaging reconstruction network are adjusted in reverse to enhance the ability to restore thermal distribution characteristics; if the error meets the standard (such as ≤1°C), the current parameter configuration is retained and other modules continue to be optimized. This error constraint based on real thermal data ensures that the output of the multimodal imaging model in the thermal field dimension has both physical authenticity and cross-modal correlation.

[0092] In an embodiment of the present invention, the thermal field feature calibration error calculation is performed on the multimodal imaging model based on the thermal field compensation reference feature of the thermal radiation data instance and the acoustic-thermal coupling feature instance to obtain the thermal field feature calibration error value, which can be implemented through the following examples.

[0093] Performing imaging reconstruction on the thermal field compensation reference features of the thermal radiation data instance to obtain an imaging reconstructed image;

[0094] Performing imaging reconstruction on the acoustic-thermal coupling feature instance to obtain an acoustic-thermal coupling imaging result instance including the thermal distribution feature of the target device area instance and the reflection spectrum characteristics of the ultrasonic reflection data instance;

[0095] Performing thermal field feature extraction on the imaging reconstructed image and the acoustic thermal coupling imaging result instance respectively, and obtaining a first eigenvector and a second eigenvector correspondingly;

[0096] The vector difference between the first eigenvector and the second eigenvector is used as the thermal field characteristic calibration error value.

[0097] In an embodiment of the present invention, in the thermal field feature calibration error calculation phase of the multimodal imaging model, the server operates on a training set of data from a 110kV transformer in a substation, following the process of "thermal field benchmark feature reconstruction imaging - acoustic-thermal coupling feature reconstruction imaging - thermal field feature vector extraction - vector difference statistics." First, the server invokes an imaging reconstruction network based on an improved U-Net architecture to perform imaging reconstruction on the thermal field compensation benchmark features of the thermal radiation data instance (after three-stage compensation, the 512-dimensional features accurately encode the thermal anomaly information of the inter-turn short circuit of the A-phase winding: hotspot pixel coordinates (205, 152), irregular polygonal morphology, and thermal distribution contours of the core and heat sink). During the reconstruction process, the network focuses solely on restoring pure thermal field information and outputs a reconstructed image. In the image, the hotspot of the A-phase winding appears as a dark red irregular patch (corresponding to a temperature of 120°C). The thermal gradients of the core and heat sink fully match the physical structure, without any interference from ultrasonic information, accurately reproducing the intrinsic thermal state of the device after thermal field compensation. Next, the server performs imaging reconstruction on the acoustic-thermal coupling feature instance (a 512-dimensional feature that integrates thermal field compensation information and ultrasonic defect reflection information, encoding hotspot locations and cross-modal correlations of ultrasonic reflection enhancement areas) to generate an acoustic-thermal coupling imaging result instance. This image presents the deep red hot spot of the A-phase winding while superimposing a bright white ultrasonic reflection spot (corresponding to the strong reflection caused by the short circuit). The thermal distribution and ultrasonic information are deeply integrated to intuitively present the cross-modal correlation characteristics of the defect. Then, the server performs thermal field feature extraction on the two images respectively: for the imaging reconstructed image, the thermal field feature extraction module composed of convolutional layers and fully connected layers is called to extract core thermal field information such as "hotspot pixel coordinates, number of morphological vertices, and device area thermal gradient encoding" to generate the first feature vector (for example, a vector encoding hotspot pixel coordinates (205, 152), number of morphological vertices 8, and winding thermal gradient activation value 0.9); for the acoustic thermal coupling imaging result example, the server focuses on its thermal field information, extracts thermal field features of the same dimension, and generates the second feature vector (for example, a vector encoding hotspot prediction pixel coordinates (203, 150), number of morphological vertices 7, and winding thermal gradient activation value 0.85). Finally, the server calculates the vector difference between the first eigenvector and the second eigenvector (taking Euclidean distance as an example, if the first eigenvector is ([205, 152, 8, 0.9]) and the second eigenvector is ([203, 150, 7, 0.85]), the difference is calculated by taking the root sum of squares of the deviations in each dimension. For example, the specific calculation process can be expressed as follows: ), this value is the calibration error value of the thermal field feature. The smaller the difference, the more aligned the thermal field information retained in the acoustic-thermal coupling feature is with the compensated precise thermal field feature. Through vector-level difference statistics of the thermal field features, the server accurately measures the retention and calibration accuracy of the thermal field information during the acoustic-thermal coupling process: if the error value is too high (such as greater than 3), the cross-correlation parameters of the feature coupling unit are adjusted in reverse to enhance the fusion accuracy of the thermal field benchmark feature and the ultrasonic feature; if the error meets the standard (such as less than or equal to 1), the current parameter configuration is retained and other modules of the model continue to be optimized. This error constraint, anchored by the thermal field compensation benchmark, ensures that the multimodal imaging model does not lose precise thermal field information during cross-modal coupling, providing reliable technical support for multi-dimensional judgment of equipment defects.

[0098] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned multimodal imaging method based on acoustic thermal coupling. Figure 2 As shown, Figure 2 This is a block diagram of the structure of a computer device 100 provided in an embodiment of the present invention. Computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or exchange, memory 111, processor 112, and communication unit 113 are electrically connected to each other, directly or indirectly. For example, these components can be electrically connected via one or more communication buses or signal lines.

[0099] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in light of the above teachings. These embodiments have been selected and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the present disclosure and to utilize various embodiments with various modifications as appropriate for the specific application contemplated.

Claims

1. A multimodal imaging method based on acoustic thermal coupling, characterized in that: include: Acquiring thermal radiation data and ultrasonic reflection data of a target device area; Performing feature representation learning on the thermal radiation data and the ultrasonic reflection data respectively to obtain thermal radiation embedding features and ultrasonic embedding features respectively; Performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedded feature to obtain thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages; The ultrasonic embedded feature is sequentially subjected to multi-stage ultrasonic feature compensation processing; wherein the feature compensation stage of the multi-stage ultrasonic feature compensation processing is consistent with the feature compensation stage of the multi-stage thermal radiation feature compensation processing; between each two consecutive stages of ultrasonic feature compensation processing, a feature coupling processing is performed on the ultrasonic feature compensation processing result obtained by the ultrasonic feature compensation processing of the previous stage in the two-stage ultrasonic feature compensation processing and the thermal field compensation reference feature of the corresponding feature compensation stage; when performing the ultrasonic feature compensation processing of the latter stage in the two-stage ultrasonic feature compensation processing, an ultrasonic feature compensation processing is performed on the ultrasonic embedded feature and the feature coupling output completed by the feature coupling; The acoustic-thermal coupling features obtained after the multi-stage ultrasonic feature compensation processing are imaged and reconstructed to obtain an acoustic-thermal coupling imaging result including the thermal distribution features of the target device area and the reflection spectrum characteristics of the ultrasonic reflection data.

2. The method according to claim 1, characterized in that The method is implemented by a pre-trained multimodal imaging model; the multimodal imaging model includes a core imaging unit and a thermal feature calibration unit; the core imaging unit includes at least an ultrasonic feature mapper and a feature coupling unit, and the thermal feature calibration unit includes a thermal feature mapper and an acoustic thermal feature calibrator; The multimodal imaging model is obtained by the following method, including: Loading a thermal radiation data instance in a preset training set into the thermal feature calibration unit, performing feature mapping processing on the thermal radiation data instance through a thermal feature mapper in the thermal feature calibration unit to obtain a thermal radiation embedding feature of the thermal radiation data instance; Performing multi-stage thermal radiation feature compensation processing on the thermal radiation embedded features of the thermal radiation data instance by the acoustic thermal feature calibrator in the thermal feature calibration unit, and correspondingly obtaining thermal field compensation reference features of the thermal radiation data instance of the target device area instance in the thermal radiation data instance under multiple feature compensation stages; Loading an ultrasonic reflection data instance in a preset training set into the core imaging unit, performing feature mapping processing on the ultrasonic reflection data instance through an ultrasonic feature mapper in the core imaging unit to obtain a sample ultrasonic embedding feature; Performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedding feature of the sample in sequence by a feature coupling unit in the core imaging unit to obtain an acoustic-thermal coupling feature instance; Performing imaging reconstruction on the acoustic-thermal coupling feature instance to obtain an acoustic-thermal coupling imaging result instance including the thermal distribution feature of the target device area instance and the reflection spectrum characteristics of the ultrasonic reflection data instance; Taking the mean absolute error between the acoustic thermal coupling imaging result instance and the thermal radiation data instance as the consistency error value; Performing imaging reconstruction on the thermal field compensation reference features of the thermal radiation data instance to obtain an imaging reconstructed image; Performing thermal field feature extraction on the imaging reconstructed image and the acoustic thermal coupling imaging result instance respectively, and obtaining a first eigenvector and a second eigenvector correspondingly; Using the vector difference between the first eigenvector and the second eigenvector as the thermal field characteristic calibration error value; Taking the sum of the consistency error value and the thermal field feature calibration error value as the error value of the multimodal imaging model; The architecture parameters in the multimodal imaging model are tuned based on the error value to obtain a trained multimodal imaging model.

3. The method according to claim 1, characterized in that The multi-stage thermal radiation feature compensation processing is performed on the thermal radiation embedded feature to obtain thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages, including: The thermal radiation embedding feature is subjected to multi-stage thermal radiation feature compensation processing through multiple intermediate layers in the trained acoustic thermal feature calibrator, thereby obtaining thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages; Wherein, when performing the first stage thermal radiation feature compensation processing on the thermal radiation embedded feature, the first stage thermal radiation feature compensation processing is performed on the thermal radiation embedded feature through the first intermediate layer of the multiple intermediate layers to obtain the thermal field compensation reference feature of the target device area in the thermal radiation data in the first stage; When the thermal radiation embedded feature is subjected to target stage thermal radiation feature compensation processing, the target stage thermal radiation feature compensation processing is performed on the thermal field compensation reference feature of the previous stage through the target intermediate layer among the multiple intermediate layers to obtain the thermal field compensation reference feature of the target device area in the thermal radiation data in the target stage, and the previous stage is the previous stage of the target stage.

4. The method according to claim 3, characterized in that The intermediate layer is a first feature compensation component, which includes a feature compensation operator; The multiple intermediate layers in the trained acoustic thermal feature calibrator perform multi-stage thermal radiation feature compensation processing on the thermal radiation embedding feature, and correspondingly obtain thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages, including: By calling the feature compensation operator in each first feature compensation component in the multiple intermediate layers, multi-stage thermal radiation feature compensation processing is performed on the thermal radiation embedded feature, and the thermal field compensation reference features of the target device area in the thermal radiation data under multiple feature compensation stages are obtained.

5. The method according to claim 4, characterized in that The step of sequentially performing multi-stage ultrasonic feature compensation processing on the ultrasonic embedded features includes: When performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the first-stage ultrasonic feature compensation processing is performed on the ultrasonic embedded feature by a first second feature compensation component among a plurality of second feature compensation components to obtain a first-stage output feature of the ultrasonic reflection data after the first-stage ultrasonic feature compensation processing; After performing target stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the target stage output feature after the target stage ultrasonic feature compensation processing and the thermal field compensation reference feature generated by the target first feature compensation component in the acoustic thermal feature calibrator are subjected to feature coupling by the target cross-correlation feature alignment component in the feature coupling unit to obtain a target feature coupling output; When performing target-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, target-stage ultrasonic feature compensation processing is performed on the ultrasonic embedded feature and a preceding feature coupled output generated by a preceding cross-correlation feature alignment component by a target second feature compensation component among the multiple second feature compensation components, thereby obtaining a target-stage output feature of the ultrasonic reflection data after the target-stage ultrasonic feature compensation processing; using the output feature obtained after the last stage of the multi-stage ultrasonic feature compensation process as the acoustic-thermal coupling feature obtained after the multi-stage ultrasonic feature compensation process; In which, the characteristic coupling unit includes multiple cascaded second characteristic compensation components, and a mutual correlation characteristic alignment component is set between each two second characteristic compensation components in a superior-subordinate relationship; the number of the second characteristic compensation components is consistent with the number of the first characteristic compensation components in the acoustic-thermal characteristic calibrator.

6. The method according to claim 5, characterized in that The plurality of second feature compensation components include: a second feature compensation component for performing dimensionality reduction processing on the ultrasonic embedding feature, and a second feature compensation component for performing dimensionality increase processing on the ultrasonic embedding feature; Among them, the number of second feature compensation components used for dimensionality reduction processing of the ultrasonic embedded features is consistent with the number of second feature compensation components used for dimensionality increase processing of the ultrasonic embedded features; and the second feature compensation components used for dimensionality increase processing of the ultrasonic embedded features are arranged after the second feature compensation components used for dimensionality reduction processing of the ultrasonic embedded features.

7. The method according to claim 5, characterized in that Before performing the first-stage ultrasonic feature compensation processing on the ultrasonic embedded feature, the method further includes: Generate the simulated disturbance characteristics of the characteristic coupling unit through a preset acoustic and thermal disturbance simulation module; Loading the simulated disturbance feature and the ultrasonic embedding feature as current processing features into the feature coupling unit; The first second feature compensation component among the plurality of second feature compensation components performs a first stage ultrasonic feature compensation process on the ultrasonic embedded feature to obtain a first stage output feature of the ultrasonic reflection data after the first stage ultrasonic feature compensation process, comprising: The first second feature compensation component performs first-stage ultrasonic feature compensation processing on the ultrasonic embedding feature and the simulated disturbance feature to obtain the first-stage output feature.

8. The method according to claim 5, characterized in that The characteristic dimension of the target stage output feature is consistent with the characteristic dimension of the thermal field compensation reference feature generated by the target first feature compensation component; The target cross-correlation feature alignment component in the feature coupling unit performs feature coupling on the target stage output feature after the target stage ultrasonic feature compensation processing and the thermal field compensation reference feature generated by the target first feature compensation component in the acoustic thermal feature calibrator to obtain the target feature coupling output, including: By using the target mutual correlation feature alignment component in the feature coupling unit, a feature merging process is performed on the target stage output feature and the thermal field compensation reference feature generated by the target first feature compensation component in the energy dimension to obtain a merged feature; Performing peak feature compression processing and mean feature compression processing on the merged features respectively, to obtain peak compression features and mean compression features respectively; Using the peak compression feature and the mean compression feature as the compression feature; performing a convolution operation on the compressed feature to obtain an acoustic-thermal coupling modulation coefficient between the target stage output feature and the thermal field compensation reference feature generated by the target first feature compensation component; A feature coupling processing calculation is performed on the combined feature based on the acoustic-thermal coupling modulation coefficient to obtain the target feature coupling output.

9. The method according to claim 1, characterized in that The thermal field compensation reference feature includes a thermal radiation characteristic representation for indicating the thermal field distribution characteristics of the target device area; The thermal field distribution characteristics include: thermal anomaly area location, thermal anomaly shape, and equipment area outline.

10. A server system, characterized in that: The method comprises a server, wherein the server is configured to execute the method according to any one of claims 1 to 9.

Images