Temperature field reconstruction method based on video diffusion pseudo-view enhancement and detail layering

Abstract

The application relates to a temperature field reconstruction method based on video diffusion pseudo-view angle enhancement and detail layering. The method comprises the following steps: inputting an infrared edge condition graph, a depth edge condition graph, an optional thermal anomaly graph, a pseudo-view angle depth graph and a real infrared image in a structure condition graph of a target power transformation equipment into a video diffusion model to output a predicted infrared image corresponding to the pseudo-view angle; and reconstructing a temperature field of the target power transformation equipment according to a repaired infrared image corresponding to the predicted infrared image and the real infrared image to obtain a three-dimensional temperature field model of the target power transformation equipment; and the three-dimensional temperature field model comprises a Gaussian cell set of at least one detail level. The model can distinguish and display different Gaussian levels according to thermal radiation characteristics, the problem that the model cannot display different Gaussian levels is solved, the distinguishability of a high-temperature key area under different observation distances is improved, and the positioning time of an abnormal area during manual screening is reduced.

Description

Technical Field

[0001] This application relates to the field of intelligent inspection technology for power equipment, and in particular to a temperature field reconstruction method based on video diffusion pseudo-view enhancement and detail layering. Background Technology

[0002] The technology of UAV infrared inspection can obtain the surface temperature distribution of substations, bushings, insulators, busbar joints and main transformers without interrupting power supply. It is an important means of identifying local overheating, poor contact and insulation abnormalities.

[0003] Traditional methods involve densely collecting hundreds of infrared images around the equipment to obtain a complete three-dimensional temperature model. While this approach provides accurate 3D modeling of the equipment, it also introduces problems such as long flight times for UAV infrared inspections, long data transmission times, and long manual screening times. Furthermore, the use of a single Gaussian representation after modeling from multiple dense infrared images lacks the ability to organize the data at different observation scales. Summary of the Invention

[0004] Therefore, it is necessary to provide a temperature field reconstruction method based on video diffusion pseudo-viewpoint enhancement and detail layering that can solve the above-mentioned technical problems.

[0005] This application provides a temperature field reconstruction method based on video diffusion pseudo-viewpoint enhancement and detail layering. The method includes: Acquire real infrared images of the target substation from multiple real viewpoints, as well as structural condition diagrams corresponding to multiple pseudo viewpoints; the camera poses corresponding to each pseudo viewpoint and the camera poses corresponding to each real viewpoint belong to the same camera trajectory. The infrared edge condition map, depth edge condition map, optional thermal anomaly map, and depth map of the pseudo-viewpoint in the structural condition map are input into the control branch of the video diffusion model, and multi-scale residuals are output. The multi-scale residual and the predicted continuous infrared image corresponding to the real infrared image are processed by the diffusion denoising network of the video diffusion model to obtain the predicted infrared image corresponding to the pseudo-viewpoint. The predicted infrared image is repaired to obtain the repaired infrared image of the pseudo-viewpoint; Based on the repaired infrared image and the real infrared image, the temperature field of the target substation is reconstructed to obtain a three-dimensional temperature field model of the target substation; the three-dimensional temperature field model contains at least one level of detail infrared layer image.

[0006] The aforementioned temperature field reconstruction method based on video diffusion pseudo-view enhancement and detail layering acquires real infrared images of the target substation from multiple real viewpoints, as well as structural condition maps corresponding to multiple pseudo-viewpoints. The camera poses corresponding to each pseudo-viewpoint and the camera poses corresponding to each real viewpoint belong to the same camera trajectory. By acquiring infrared images of the target substation from multiple real viewpoints and structural condition maps of pseudo-viewpoints under the same camera trajectory, a multimodal and geometrically aligned data foundation is provided for subsequent generation and reconstruction, reducing the dependence on high-density real sampling viewpoints, thereby effectively shortening the flight time and data transmission time of UAV infrared inspection.

[0007] By inputting the infrared edge condition map, depth edge condition map, optional thermal anomaly map, and pseudo-viewpoint depth map from the structural condition map into the control branch of the video diffusion model, and outputting multi-scale residuals, multi-level precise constraints on the generation process in terms of geometric boundaries, depth structure, and thermal field distribution are achieved. This significantly suppresses the unconstrained behavior of the diffusion model, improves the structural fidelity and thermal field consistency of pseudo-viewpoint infrared images, and reduces misjudgments and repeated screening time caused by image distortion during manual screening.

[0008] By using a diffusion denoising network based on a video diffusion model, multi-scale residuals and predicted continuous infrared images corresponding to real infrared images are processed to obtain predicted infrared images corresponding to pseudo-viewpoints. This results in temporally coherent and geometrically stable predicted infrared images of pseudo-viewpoints, solving the problems of image flicker and structural jumps caused by the randomness of generation between adjacent pseudo-viewpoints. This makes the intermediate observation data required for three-dimensional temperature field reconstruction more reliable, thereby further reducing the time required for repeated acquisition and post-processing to eliminate flicker.

[0009] By repairing the predicted infrared image, a repaired infrared image with a pseudo-viewpoint is obtained, which improves the data quality of the pseudo-viewpoint image.

[0010] Based on the restored infrared images and the original infrared images, the temperature field of the target substation is reconstructed to obtain a three-dimensional temperature field model of the target substation. The three-dimensional temperature field model contains infrared level images of at least one level of detail. This enables the three-dimensional temperature model to distinguish and display different Gaussian levels based on thermal radiation characteristics, while improving the identifiability of key high-temperature areas at different observation distances, further reducing the time required to locate abnormal areas during manual screening. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a temperature field reconstruction method based on video diffusion pseudo-viewpoint enhancement and detail layering in one embodiment. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0013] In one embodiment, such as Figure 1 As shown, a temperature field reconstruction method based on video diffusion pseudo-viewpoint enhancement and detail layering is provided, specifically including: S101, acquire the real infrared images of the target substation from multiple real perspectives, as well as the structural condition diagrams corresponding to multiple pseudo perspectives.

[0014] The target substation equipment refers to the substation facilities that require infrared temperature field detection and 3D reconstruction, including but not limited to power equipment such as transformers, circuit breakers, disconnect switches, and busbar connectors. This equipment is the object of infrared image acquisition and temperature field modeling, and its surface temperature distribution is used to assess the equipment's operating status.

[0015] For example, the target substation equipment includes porcelain bushings or insulators.

[0016] The true infrared viewpoint refers to the actual infrared observation viewpoint obtained by photographing the target power equipment from a specific location and orientation in physical space using an actual imaging device. The image corresponding to the true infrared viewpoint is a true infrared image, containing the actual thermal radiation information of the equipment from that viewpoint, serving as the reference data for 3D reconstruction. The actual imaging device can be a drone equipped with an infrared thermal imager, a handheld thermal imager, or an inspection drone, etc.

[0017] A pseudo-viewpoint refers to a virtual observation perspective synthesized through mathematical interpolation or generative models between or outside of real viewpoints, and is not actually captured by a physical camera. A pseudo-viewpoint itself does not contain a real infrared image; instead, it generates a predicted infrared image based on a structure condition map using a video diffusion model. This image is used to fill the observation gaps between real viewpoints, supporting the reconstruction of a dense three-dimensional temperature field.

[0018] The camera poses corresponding to each pseudo-viewpoint and the camera poses corresponding to each real-viewpoint belong to the same camera trajectory.

[0019] Camera pose represents the position and orientation of the imaging device in three-dimensional space, and is described by both the position vector and the rotation matrix.

[0020] A camera trajectory represents a set of continuous poses formed by a camera as its position changes over time or spatially. The camera trajectory links the poses of all real and pseudo-viewpoints onto the same geometric path, ensuring consistency in spatial geometry during generation and reconstruction, and serves as the mathematical basis for pseudo-viewpoint pose interpolation.

[0021] The structural condition graph represents a set of multimodal conditional information graphs used to constrain the video diffusion model to generate pseudo-viewpoint infrared images.

[0022] Specifically, in S101, obtaining the structure condition maps corresponding to multiple pseudo-viewpoints can include: obtaining the camera trajectory through interpolation based on each real infrared image and the corresponding camera intrinsic and extrinsic parameters; rendering the coarse infrared map, depth map, and visible Gaussian projection information corresponding to each pseudo-viewpoint in the camera trajectory based on the trained 3D Gaussian model; determining the coarse infrared gradient of the coarse infrared map and the depth gradient of the depth map based on the coarse infrared map, depth map, and visible Gaussian projection information; and constructing the structure condition maps corresponding to the pseudo-viewpoints based on the coarse infrared gradient and depth gradient.

[0023] By acquiring infrared images of the target substation from multiple real-view perspectives and structural condition maps of pseudo-view perspectives under the same camera trajectory, a multimodal and geometrically aligned data foundation is provided for subsequent generation and reconstruction. This reduces the dependence on high-density real-sampling perspectives, thereby effectively shortening the flight time and data transmission time of UAV infrared inspection.

[0024] S102 takes the infrared edge condition map, depth edge condition map, optional thermal anomaly map, and pseudo-viewpoint depth map from the structural condition map as inputs to the control branch of the video diffusion model and outputs multi-scale residuals.

[0025] The infrared edge condition map represents the edge feature map extracted from the infrared image from the true viewpoint. By processing the infrared image, the boundaries of regions where thermal radiation intensity abruptly changes can be extracted. This condition map is used to constrain the video diffusion model when generating pseudo-viewpoint infrared images, ensuring that the thermal field structure boundaries (such as device outlines and hotspot edges) of the generated image remain consistent with the geometry of the true infrared observation, thereby suppressing false edges generated by the diffusion model under unconstrained conditions.

[0026] The depth edge condition map represents the geometric edge feature map extracted from the depth map. This condition map reflects the abrupt changes in the 3D geometry of the scene (such as object edges and occlusion boundaries). It is used to constrain the pseudo-view infrared image generated by the diffusion model to maintain geometric consistency with the real 3D space, ensuring that the infrared edges are spatially aligned with the depth edges and preventing the generated thermal signals from appearing in incorrect 3D locations.

[0027] Optional thermal anomaly maps represent hotspot region identifiers generated based on infrared temperature threshold segmentation or anomaly detection algorithms (such as local outlier factor and temperature standard deviation analysis). These maps can be represented as binary mask maps or continuous heatmaps. This map is used to identify high-temperature or low-temperature anomaly regions in the scene whose temperatures significantly deviate from the background region. It is input as an optional condition into the control branch of the diffusion model to impose enhanced constraints on the thermal anomaly regions, ensuring that key thermal anomaly signals are not smoothly attenuated or lost during the generation process.

[0028] The pseudo-view depth map represents the 3D geometric depth information corresponding to the pseudo-view. Each pixel value in the depth map represents the distance from the target substation surface to the camera's optical center under that pseudo-view. It provides the diffusion model with a 3D geometric skeleton under the target view, which is used to control the spatial layout, perspective structure, and occlusion relationships of the generated image, ensuring that the generated infrared image remains geometrically continuous with adjacent real viewpoints.

[0029] The video diffusion model represents a generative neural network built upon a diffusion probability model. It progressively adds noise to the data through a forward diffusion process, and then recovers the target video frame sequence from the random noise through a reverse denoising process, thus possessing the ability to generate temporally coherent and multi-frame consistent image sequences.

[0030] The control branch represents a sub-network module in the video diffusion model that receives external conditional inputs and injects them into the main diffusion network.

[0031] Multi-scale residuals represent the differential feature map residuals output by the control branch at different resolution levels, with each scale corresponding to a resolution level. Resolution levels include original resolution, 1 / 2 resolution, 1 / 4 resolution, etc.

[0032] It achieves multi-level precise constraints on the generation process in terms of geometric boundaries, depth structure and thermal field distribution, significantly suppresses the unconstrained behavior of the diffusion model, improves the structural fidelity and thermal field consistency of pseudo-view infrared images, and reduces the time for misjudgment and repeated screening caused by image distortion during manual screening.

[0033] S103 uses a diffusion denoising network of a video diffusion model to process multi-scale residuals and predicted continuous infrared images corresponding to real infrared images to obtain predicted infrared images corresponding to pseudo-viewpoints.

[0034] The diffusion denoising network represents the module in the video diffusion model responsible for performing the inverse denoising process. The video diffusion model includes the diffusion denoising network, the video diffusion backbone, and the control branch.

[0035] Predicted continuous infrared image, representing the infrared image output by the video diffusion backbone of the video diffusion model.

[0036] Predicted infrared image refers to the pseudo-viewpoint infrared image finally output by the diffusion denoising network after the video diffusion model has completed all inverse denoising iterations.

[0037] Specifically, each real infrared image is input into the video diffusion backbone of the video diffusion model, and the predicted continuous infrared image of the real infrared image is output.

[0038] By obtaining temporally coherent and geometrically stable pseudo-viewpoint predicted infrared images, the problems of image flicker and structural jumps caused by the randomness of generation between adjacent pseudo-viewpoints are solved, making the intermediate observation data required for three-dimensional temperature field reconstruction more reliable, thereby further reducing the time required for repeated acquisition and post-processing to eliminate flicker.

[0039] S104, repair the predicted infrared image to obtain a repaired infrared image with a pseudo-viewpoint.

[0040] Among them, the repaired infrared image refers to the optimized infrared image obtained after post-processing correction of the predicted infrared image generated by the video diffusion model.

[0041] There are various repair methods, such as thermal intensity correction, edge and structure repair, and inter-frame consistency repair. Specifically, S104 includes: thermal intensity correction, which involves mapping the grayscale / temperature histogram of the predicted infrared image onto the histogram distribution of adjacent real infrared images to eliminate the overall thermal intensity shift; edge and structure repair, which involves using a pseudo-view depth map or infrared edge condition map as a guide image to perform edge-preserving filtering on the predicted infrared image, restoring clear edges while smoothing flat areas; and inter-frame consistency repair, which involves smoothing the temperature values ​​at the same pixel location in adjacent frames in the time dimension (such as Gaussian smoothing or Kalman filtering) to suppress high-frequency jumps between frames.

[0042] Specifically, S104 includes: performing consistency estimation on the predicted infrared image to obtain the low-frequency thermal field consistency, structural edge consistency, and adjacent viewpoint temporal consistency of the predicted infrared image; for each predicted infrared image, determining the pixel-level confidence mask corresponding to the predicted infrared image based on the low-frequency thermal field consistency, structural edge consistency, and adjacent viewpoint temporal consistency; and determining the repaired infrared image of the pseudo-viewpoint based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint.

[0043] By repairing the predicted infrared images, the data quality of the pseudo-viewpoint images was improved.

[0044] S105. Based on the repaired infrared image and the real infrared image, the temperature field of the target substation is reconstructed to obtain a three-dimensional temperature field model of the target substation.

[0045] Among them, the three-dimensional temperature field model represents the digital representation of the temperature distribution of the target power equipment in three-dimensional space, which is composed of a three-dimensional geometric model and temperature field information mapped on its surface.

[0046] The three-dimensional temperature field model contains infrared-level images at at least one level of detail.

[0047] The level of detail represents different levels of refinement in a three-dimensional temperature field model, based on spatial resolution, temperature gradient complexity, or importance of thermal information. It is used to express the information granularity of the model under different observation distances or different analysis requirements.

[0048] For example, the level of detail can be a basic thermal field layer (temperature gradient), a structural layer (coarse level), or a level of detail (fine level).

[0049] A Gaussian set represents the total number of Gaussian elements used in a 3D temperature field model to express the thermal radiation distribution and 3D geometry of a target power equipment. It is the basic set of data units constituting the 3D temperature field model. Gaussian set can be used to render infrared-level images; specifically, an infrared-level image is rendered based on a Gaussian set at a specific detail level.

[0050] An infrared tonal image represents a two-dimensional infrared image rendered from a specific viewpoint at a particular level of detail in a 3D temperature field model. This image is a projection of the temperature field information preserved at that level from that viewpoint, including the geometric accuracy and temperature sampling resolution corresponding to that level. Specifically, an infrared tonal image can be obtained by rendering a set of Gaussian units at a specific level of detail.

[0051] There are several ways to reconstruct the temperature field of the target substation. Specifically, S105 includes: reconstructing the temperature field of the target substation based on the repaired infrared image, the real infrared image, the image weight, the weight of the basic thermal field layer, the weight of the structural layer, and the weight of the detail layer, to obtain a three-dimensional temperature field model of the target substation.

[0052] The weights of the basic thermal field layer include: the weights of low-frequency thermal field, isothermal plateau smoothing, and thermal intensity statistics constraints; the weights of the structure layer include: the weights of structure region reconstruction loss and gradient consistency loss; and the weights of the detail layer include: the weights of detail region reconstruction loss and texture anchoring.

[0053] Specifically, before reconstructing the temperature field, it is necessary to train the aforementioned weights. If the loss function corresponding to the weights satisfies the iteration termination condition, the temperature field can be reconstructed based on the updated weights corresponding to the last iteration.

[0054] After obtaining the three-dimensional temperature field model, infrared layer images of different levels can be displayed according to the camera focal length. Specifically, the camera focal length and the target depth of the Gaussian elements on the three-dimensional temperature field model relative to the camera are obtained; the projection radius of the Gaussian elements is determined according to the camera focal length and target depth; the detail level adapted to the projection radius is determined according to the relationship between the first level threshold, the second level threshold and the projection radius; the first level threshold is less than the second level threshold; wherein, the detail level includes at least one of the basic thermal field layer, the structural layer or the detail layer, the basic thermal field layer forms a basic Gaussian with a coverage area greater than a preset range in the isothermal field region, the structural layer adds structural Gaussian in the thermal edge, depth discontinuity and thermal anomaly regions, and the detail layer adds detail Gaussian in the high-confidence real or pseudo-view detail regions; the infrared layer image corresponding to the detail level in the three-dimensional temperature field model is displayed.

[0055] Specifically, based on the relationship between the first-level threshold, the second-level threshold, and the projection radius, the detail level adapted to the projection radius is determined, including: when the projection radius is less than the first-level threshold, the detail level adapted to the projection radius is the basic thermal field layer; when the projection radius is greater than or equal to the first-level threshold and less than the second-level threshold, the detail level adapted to the projection radius includes the basic thermal field layer and the structural layer; when the projection radius is greater than or equal to the second-level threshold, the detail level adapted to the projection radius includes the basic thermal field layer, the structural layer, and the detail layer.

[0056] The values ​​of the first-level threshold and the second-level threshold can be adjusted according to the camera resolution, equipment size, and inspection distance.

[0057] In the aforementioned temperature field reconstruction method based on video diffusion pseudo-view enhancement and detail layering, real infrared images of the target substation from multiple real viewpoints are acquired, along with structural condition maps corresponding to multiple pseudo-viewpoints. The camera poses corresponding to each pseudo-viewpoint and the camera poses corresponding to each real viewpoint belong to the same camera trajectory. By acquiring infrared images of the target substation from multiple real viewpoints and structural condition maps of pseudo-viewpoints under the same camera trajectory, a multimodal, geometrically aligned data foundation is provided for subsequent generation and reconstruction. This reduces the dependence on high-density real sampling viewpoints, thereby effectively shortening the flight time and data transmission time of UAV infrared inspection.

[0058] By inputting the infrared edge condition map, depth edge condition map, optional thermal anomaly map, and pseudo-viewpoint depth map from the structural condition map into the control branch of the video diffusion model, and outputting multi-scale residuals, multi-level precise constraints on the generation process in terms of geometric boundaries, depth structure, and thermal field distribution are achieved. This significantly suppresses the unconstrained behavior of the diffusion model, improves the structural fidelity and thermal field consistency of pseudo-viewpoint infrared images, and reduces misjudgments and repeated screening time caused by image distortion during manual screening.

[0059] By using a diffusion denoising network based on a video diffusion model, multi-scale residuals and predicted continuous infrared images corresponding to real infrared images are processed to obtain predicted infrared images corresponding to pseudo-viewpoints. This results in temporally coherent and geometrically stable predicted infrared images of pseudo-viewpoints, solving the problems of image flicker and structural jumps caused by the randomness of generation between adjacent pseudo-viewpoints. This makes the intermediate observation data required for three-dimensional temperature field reconstruction more reliable, thereby further reducing the time required for repeated acquisition and post-processing to eliminate flicker.

[0060] By repairing the predicted infrared image, a repaired infrared image with a pseudo-viewpoint is obtained, which improves the data quality of the pseudo-viewpoint image.

[0061] Based on the restored infrared images and the original infrared images, the temperature field of the target substation is reconstructed to obtain a three-dimensional temperature field model of the target substation. The three-dimensional temperature field model contains infrared level images of at least one level of detail. This enables the three-dimensional temperature model to distinguish and display different Gaussian levels based on thermal radiation characteristics, while improving the identifiability of key high-temperature areas at different observation distances, further reducing the time required to locate abnormal areas during manual screening.

[0062] In one embodiment, obtaining the structure condition map corresponding to multiple pseudo-viewpoints includes: obtaining the camera trajectory by interpolation based on each real infrared image and the camera intrinsic and extrinsic parameters corresponding to each real infrared image; rendering the coarse infrared image, depth image, and visible Gaussian projection information corresponding to each of the multiple pseudo-viewpoints in the camera trajectory based on the trained 3D Gaussian model; determining the coarse infrared gradient of the coarse infrared image and the depth gradient of the depth image based on the coarse infrared image, depth image, and visible Gaussian projection information; and constructing the structure condition map corresponding to the pseudo-viewpoint based on the coarse infrared gradient and depth gradient.

[0063] The 3D Gaussian model represents a scene representation model constructed based on 3D Gaussian splashing technology. The 3D Gaussian model consists of a large number of 3D Gaussian ellipsoids with learnable parameters.

[0064] The coarse infrared image represents the initial infrared image directly rendered from the pseudo-viewpoint camera pose using a trained 3D Gaussian model. This image is the projection of the 3D Gaussian model onto the surface temperature radiation distribution of the target substation equipment at this viewpoint. The visible Gaussian projection information represents the projection attribute record of each pixel position on the two-dimensional image plane after all three-dimensional Gaussian volumes in the three-dimensional Gaussian model are rendered by differentiable rasterization under a pseudo-viewpoint.

[0065] The coarse infrared gradient represents the gradient magnitude map or gradient direction map obtained after applying a differential operator to the coarse infrared image, characterizing the degree of spatial variation in thermal radiation intensity in the coarse infrared image.

[0066] Depth gradient refers to the gradient magnitude map or gradient direction map obtained after applying a differential operator to a pseudo-view depth map, which characterizes the degree of drastic changes in spatial undulations of a three-dimensional geometric surface.

[0067] Understandably, differential operators include the Sobel operator, the Scharr operator, or the Laplace operator.

[0068] In this embodiment, firstly, by determining the coarse infrared gradient and depth gradient based on the coarse infrared image and depth image respectively, the thermal field structure boundary and three-dimensional geometric abrupt change region under the pseudo-viewpoint are extracted. The key structural priors required for the generation of the diffusion model are quantified in the form of gradients, providing clear edge constraint information for the construction of the structure condition map. Secondly, by constructing the structure condition map corresponding to the pseudo-viewpoint based on the coarse infrared gradient and depth gradient, the thermal field edge and geometric edge are fused into multimodal condition constraints. This allows the diffusion model to be guided by both the thermal radiation structure and the three-dimensional geometric structure simultaneously when generating pseudo-viewpoint infrared images, effectively suppressing the occurrence of erroneous edges in the generated images and improving the structural fidelity of the pseudo-viewpoint images. Thirdly, by using the visible Gaussian projection information rendered by the three-dimensional Gaussian model to participate in the construction of the structure condition map, the visibility prior of the contribution of each Gaussian point to imaging under the pseudo-viewpoint is introduced. This allows the condition map to distinguish which regions in the scene are supported by observation data and which regions are only inferred by interpolation, providing differentiated credibility guidance for the diffusion model and reducing the uncertainty of pure interpolation region generation.

[0069] In one embodiment, the method further includes: inputting each real infrared image into the video diffusion backbone of the video diffusion model, and outputting a predicted continuous infrared image of the real infrared images.

[0070] The video diffusion backbone represents the main network structure in the video diffusion model responsible for performing core denoising and feature transformation. Built on a 3D U-Net architecture, the video diffusion backbone consists of an encoder, bottleneck layer, decoder, and skip connections, and integrates temporal and spatial attention modules.

[0071] In this embodiment, firstly, the backbone network is used to perform spatiotemporal joint feature encoding on the input image, mapping the two-dimensional image information from the real viewpoint to a high-dimensional latent space. This provides a deep feature representation containing temporal distribution and spatial structure for subsequent diffusion iterations, laying the feature foundation for video-level coherent generation. Secondly, the coarsely rendered viewpoints between a small number of real infrared images are organized into a continuous sequence. The video diffusion backbone is used to supplement the cross-viewpoint appearance, and the control branch is used to inject the depth and edge constraints corresponding to the target pose, reducing the geometric jumps caused by frame-by-frame independent generation.

[0072] In one embodiment, repairing a predicted infrared image to obtain a repaired infrared image from a pseudo-viewpoint includes: performing consistency estimation on the predicted infrared image to obtain low-frequency thermal field consistency, structural edge consistency, and temporal consistency of adjacent viewpoints; for each predicted infrared image, determining a pixel-level confidence mask corresponding to the predicted infrared image based on the low-frequency thermal field consistency, structural edge consistency, and temporal consistency of adjacent viewpoints; and determining the repaired infrared image from the pseudo-viewpoint based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image from the pseudo-viewpoint.

[0073] Among them, low-frequency thermal field consistency refers to the measure that the predicted infrared image is consistent with the statistical law of thermal radiation in the low-frequency thermal radiation distribution of adjacent real infrared images or scenes.

[0074] Structural edge consistency is a measure of the consistency in spatial location and geometry between structural edges (including device outlines, hotspot boundaries, etc.) extracted from predicted infrared images and structural edges derived from real infrared images or depth maps.

[0075] Adjacent view temporal consistency is a measure of how smoothly adjacent frames in a predicted infrared image sequence change over time in terms of thermal field distribution and geometry.

[0076] A pixel-level confidence mask represents a confidence map with the same resolution as the predicted infrared image, where each pixel value indicates the confidence level of the generated result for that pixel. In the confidence mask, high-value areas indicate that the pixel is highly consistent with the actual observation or physical laws in three dimensions: thermal field, structure, and temporal sequence, and is suitable for direct use in 3D reconstruction; low-value areas indicate that the pixel has significant deviations in at least one dimension and needs to be corrected first in subsequent repair.

[0077] Specifically, low-frequency thermal field uniformity The expression is as follows: ; in, This represents the Gaussian low-pass operator. To predict infrared images, This is a coarse infrared image from a pseudo-viewpoint. This is a scale parameter for the consistency of low-frequency thermal fields, used to control the intensity of the impact of differences on credibility.

[0078] Structural edge consistency The expression is as follows: ; in, To predict the image gradient of an infrared image, This is a geometric edge map from a pseudo-perspective. The scale parameter for edge consistency.

[0079] Temporal consistency of adjacent viewpoints The expression is as follows: ; in, For predicted infrared images obtained through viewpoint alignment or inter-frame transformation operations. The scaling parameter for time-series consistency.

[0080] Specifically, based on the low-frequency thermal field consistency, structural edge consistency, and adjacent viewpoint temporal consistency of the predicted infrared image, a pixel-level confidence mask corresponding to the predicted infrared image is determined according to these parameters. The expression is as follows: ; Here, α, β, and γ are the weighting indices of the three confidence components, used to adjust the importance of thermal field, edge, and temporal factors.

[0081] Specifically, after obtaining the pixel-level confidence mask, the repaired infrared image for the pseudo-viewpoint is determined based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint. The repaired infrared image can be expressed by the following formula: ; Where ⊙ represents pixel-by-pixel multiplication. For pixel-level credibility mask, To predict infrared images, This is a coarse infrared image from a pseudo-perspective.

[0082] In this embodiment, firstly, by performing consistency estimation on the predicted infrared image, low-frequency thermal field consistency, structural edge consistency, and adjacent viewpoint temporal consistency are obtained respectively. This achieves a comprehensive quality assessment of the diffusion model generation results in three independent dimensions: macroscopic distribution of thermal radiation, geometric structure accuracy, and inter-frame dynamic coherence, providing a multi-dimensional quality measurement basis for subsequent differentiated repair. Secondly, by determining the repair infrared image based on the predicted infrared image, pixel-level confidence mask, and coarse infrared image from pseudo-viewpoints, the confidence mask guides the repair process: high confidence regions retain the original generation results of the diffusion model, while low confidence regions are weighted and replaced or interpolated using the structural prior of the coarse infrared image. This achieves differentiated and refined repair of generation defects, effectively eliminating thermal intensity shifts, erroneous edges, and inter-frame flicker while preserving the effective detail information generated by the diffusion model in the confidence region to the greatest extent. In one embodiment, determining the repaired infrared image of the pseudo-viewpoint based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint includes: determining the truth-free confidence fusion image of the pseudo-viewpoint based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint; inputting the confidence fusion sequence composed of the truth-free confidence fusion images of each pseudo-viewpoint into the spatiotemporal recovery head, and outputting the repaired infrared image of the pseudo-viewpoint.

[0083] Among them, the ground truth confidence fusion image refers to the intermediate optimized image generated by a confidence-driven pixel-level weighted fusion strategy based on the predicted infrared image, pixel-level confidence mask, and pseudo-viewpoint coarse infrared image, without the presence of a real infrared image as a reference benchmark.

[0084] A spatiotemporal recovery head represents a lightweight decoding network module, specifically comprising a recovery head, a temporal branch, and a spatial branch. The recovery head simultaneously receives the spatial features of the current frame and the alignment features of adjacent frames. The spatial branch is responsible for repairing local device textures and edges, while the temporal branch suppresses brightness jumps and structural flicker from adjacent viewpoints.

[0085] For example, the expression for the restored infrared image obtained by the pseudo-viewpoint output by the spatiotemporal restoration head is as follows: ; Among them, the training loss of the recovery head It is a weighted combination of four factors: pixel, structure, gradient, and temporal. ; Here, Hφ represents the spatiotemporal reconstruction head with parameter φ, and k is the neighborhood radius. These are high-quality pairing images from the training phase. The restored infrared image is a pseudo-viewpoint output by the spatiotemporal reconstruction head. For the training phase, high-quality reference images or real infrared images are used. SSIM, or structural similarity metric, is used to measure whether the structures of images are similar. The gradient of the output graph. To align with the previous frame restoration result from the current viewpoint, , , and These correspond to the pixel, structure, gradient, and temporal loss weights, respectively. In actual inference, a high-quality target image is no longer needed; the reconstruction head relies on the trained mapping to process the output of the Stable Video Diffusion (SVD) model and the control branch. The spatiotemporal reconstruction head can employ convolution, 3D convolution, recurrent networks, or temporal attention structures.

[0086] Under the condition of no target truth value during the inference stage, a pixel-level credibility mask is constructed by low-frequency thermal field, structural edge and adjacent view continuity, and further repaired by spatiotemporal recovery head, thereby reducing erroneous thermal distribution and pseudo-texture entering pseudo-view supervision.

[0087] In this embodiment, firstly, by determining a truth-free confidence fusion image based on the predicted infrared image, a pixel-level confidence mask, and a pseudo-viewpoint coarse infrared image, differential fusion processing of reliable and unreliable regions in the predicted infrared image is achieved based on the confidence mask, even without a real infrared image as a reference. This allows the restoration process to be completed independently in the actual inference stage where truth supervision is lacking, eliminating the dependence on the real infrared image as a restoration reference and expanding the applicability of the method in truth-free scenarios. Secondly, by constructing a confidence fusion sequence from the truth-free confidence fusion images of each pseudo-viewpoint and inputting it into the spatiotemporal recovery head, the joint refinement capability of the spatiotemporal recovery head in the time and spatial dimensions is utilized to further eliminate residual temperature jumps and structural jitter between adjacent frames on the basis of single-frame fusion restoration. This results in the output restored infrared image sequence having a smooth and coherent thermal field evolution and stable geometric structure throughout the entire sequence.

[0088] In one embodiment, the method further includes: acquiring the camera focal length and the target depth of Gaussian elements on the three-dimensional temperature field model relative to the camera; determining the projection radius of the Gaussian elements based on the camera focal length and target depth; determining a detail level adapted to the projection radius based on the relationship between a first-level threshold, a second-level threshold, and the projection radius; wherein the first-level threshold is less than the second-level threshold; wherein the detail level includes at least one of a basic thermal field layer, a structural layer, or a detail layer, wherein the basic thermal field layer forms a basic Gaussian element with a coverage area greater than a preset range in the isothermal field region, the structural layer adds structural Gaussian elements in the thermal edge, depth discontinuity, and thermal anomaly regions, and the detail layer adds detail Gaussian elements in the high-confidence real or pseudo-view detail regions; and displaying the infrared layer image corresponding to the detail level in the three-dimensional temperature field model.

[0089] The camera focal length refers to the distance from the principal point of the imaging device's optical system to the focal plane. The imaging device can be the shooting device on an inspection drone.

[0090] Gaussian elements are the basic units used in three-dimensional temperature field models to represent local temperature distribution and geometric structure.

[0091] Target depth represents the distance between the position of a certain Gaussian element in the three-dimensional temperature field model in the world coordinate system and the optical center of the camera, that is, the spatial distance of the element from the imaging device along the optical axis of the camera.

[0092] The projection radius represents the effective pixel range covered by a Gaussian element in a three-dimensional temperature field model after it is projected onto a two-dimensional imaging plane through perspective.

[0093] Specifically, the formula for the projection radius is as follows: ; in, Let be the screen projection radius of the j-th Gaussian under the current camera. For camera focal length, Let the j-th Gaussian be the target depth relative to the camera. This represents the maximum scale of the j-th Gaussian.

[0094] The first-level threshold represents a preset lower limit for the projection radius, used to distinguish the projection radius critical value between detail layers and structural layers. When the projection radius of a Gaussian primitive is less than the first-level threshold, it indicates that the primitive's coverage area on the imaging plane is too small, and its detailed information contributes little to the observer; it is classified as a basic thermal field layer or a structural layer. When the projection radius is greater than or equal to the first-level threshold, it indicates that the primitive has sufficient pixel coverage to carry detailed information; it can be classified as a structural layer or a detail layer.

[0095] The second-level threshold represents a preset upper limit for the projection radius, used to distinguish the critical values ​​of projection radii between the structure layer and the detail layer. When the projection radius of a Gaussian primitive is between the first-level and second-level thresholds, the primitive is assigned to the structure layer and is used to add structure Gaussians to hot edges, depth discontinuities, and hot anomaly regions. When the projection radius is greater than or equal to the second-level threshold, the primitive is assigned to the detail layer and is used to add detail Gaussians to high-confidence real or pseudo-view detail regions. The second-level threshold is greater than the first-level threshold, and the two thresholds together divide the Gaussian primitive into three different detail level intervals based on the projection radius.

[0096] It should be noted that: **Basic Gaussian:** This refers to large Gaussian primitives used in the 3D temperature field model to represent the basic thermal field distribution of a scene. They have a large spatial scale, wide coverage, and a projection radius typically smaller than the first-level threshold, belonging to the basic thermal field layer. **Structural Gaussian:** This refers to medium-sized Gaussian primitives used in the 3D temperature field model to represent key geometric structures and regions of thermal abrupt changes. They have a medium spatial scale, and a projection radius between the first and second-level thresholds, belonging to the structural layer. **Detail Gaussian:** This refers to small Gaussian primitives used in the 3D temperature field model to represent high-fidelity textures and fine thermal field details. They have a small spatial scale, localized coverage, and a projection radius greater than or equal to the second-level threshold, belonging to the detail layer.

[0097] An infrared layered image represents a two-dimensional infrared image of a three-dimensional temperature field model rendered from a specific viewpoint using differentiable Gaussian rasterization at a particular detail level. This image is a color (temperature) projection representation of the Gaussian unit set (basic Gaussian, structural Gaussian, or detail Gaussian) preserved at that level from that viewpoint, containing the geometric accuracy and thermal field sampling resolution corresponding to that level.

[0098] Specifically, based on the relationship between the first-level threshold, the second-level threshold, and the projection radius, a detail level adapted to the projection radius is determined, including: when the projection radius is less than the first-level threshold, the detail level adapted to the projection radius is the basic thermal field layer; when the projection radius is greater than or equal to the first-level threshold and less than the second-level threshold, the detail level adapted to the projection radius includes the basic thermal field layer and the structural layer; when the projection radius is greater than or equal to the second-level threshold, the detail level adapted to the projection radius includes the basic thermal field layer, the structural layer, and the detail layer.

[0099] Based on this, the infrared layer images corresponding to the detail level in the three-dimensional temperature field model are displayed, including: When the projection radius is less than the first level threshold, the basic thermal field layer of the three-dimensional temperature field model is displayed; when the projection radius is greater than or equal to the first level threshold and less than the second level threshold, the basic thermal field layer and structural layer of the three-dimensional temperature field model are displayed; when the projection radius is greater than or equal to the second level threshold, the detailed levels adapted to the projection radius include the basic thermal field layer, structural layer and detailed layer.

[0100] In this embodiment, firstly, by determining the detail level based on the relationship between the first-level threshold, the second-level threshold, and the projection radius, adaptive hierarchical classification of Gaussian elements according to the imaging coverage is achieved, directly linking the hierarchical division results to the observation distance and visual importance. Secondly, by displaying the infrared hierarchical images corresponding to the detail level in the three-dimensional temperature field model, observers can switch between different levels of visualization results according to diagnostic needs, switching on demand between an overview of the thermal field distribution and fine hotspot localization, balancing rendering efficiency and diagnostic accuracy.

[0101] In one embodiment, determining the detail level adapted to the projection radius based on the relationship between the first-level threshold, the second-level threshold, and the projection radius includes: when the projection radius is less than the first-level threshold, the detail level adapted to the projection radius is the basic thermal field layer; when the projection radius is greater than or equal to the first-level threshold and less than the second-level threshold, the detail level adapted to the projection radius includes the basic thermal field layer and the structural layer; when the projection radius is greater than or equal to the second-level threshold, the detail level adapted to the projection radius includes the basic thermal field layer, the structural layer, and the detail layer.

[0102] For example, the first-level threshold and the second-level threshold can be obtained by evaluating the distribution of the projection radius on the camera. The 30th and 70th quantiles of the projection distribution on the camera are evaluated and determined as the first-level thresholds, respectively. Second level threshold .

[0103] Furthermore, compare the projected radii. Compared with the first level threshold Second level threshold The size relationship between them is determined by the projection radius. Adapted detail levels .

[0104] Specifically, when the projected radius is less than the first-level threshold, i.e. The detail level adapted to the projection radius is the basic thermal field layer; When the projected radius is greater than or equal to the first-level threshold and less than the second-level threshold, i.e. The detailed levels for projection radius adaptation include the basic thermal field layer and the structural layer; When the projection radius is greater than or equal to the second-level threshold, i.e. The detail levels for projection radius adaptation include the basic thermal field layer, the structural layer, and the detail layer.

[0105] When the projection radius is small, only the basic thermal field layer G0 is rendered; when the projection radius is in the middle range, both the basic thermal field layer G0 and the structural layer G1 are rendered; when the projection radius is large, the basic thermal field layer G0, the structural layer G1, and the detail layer G2 are rendered. In this way, a stable thermal field can be maintained and the amount of computation can be reduced when viewed at a long distance, the edges of the device and the thermal anomaly structure can be highlighted at a medium distance, and high-confidence local details can be displayed when viewed at close distance.

[0106] In this embodiment, firstly, by determining the detail level based on the relationship between the first-level threshold, the second-level threshold, and the projection radius, adaptive hierarchical classification of Gaussian primitives according to the imaging coverage range is achieved, directly linking the hierarchical division results to the observation distance and visual importance. Secondly, by using a basic Gaussian representation with a coverage area larger than a preset range in the basic thermal field layer, the overall thermal field baseline of the device can be covered with only a small number of large-scale primitives, significantly reducing the computational overhead when rendering distant views. Thirdly, by adding structural Gaussian primitives in the structural layer to thermal edges, depth discontinuities, and thermal anomaly regions, key geometric boundaries and anomalous hotspots are enhanced in structural representation during mid-range observation, improving the recognition of key areas. Fourthly, by adding detailed Gaussian primitives in the detail layer to high-confidence real or pseudo-view detail regions, minute temperature differences and local thermal field gradients on the device surface are fully presented during close-range observation, supporting refined thermal state diagnosis.

[0107] In one embodiment, the temperature field of the target substation is reconstructed based on the repaired infrared image and the real infrared image to obtain a three-dimensional temperature field model of the target substation. This includes: reconstructing the temperature field of the target substation based on the repaired infrared image, the real infrared image, image weights, weights of the basic thermal field layer, weights of the structural layer, and weights of the detail layer to obtain a three-dimensional temperature field model of the target substation.

[0108] Here, image weight represents the respective weights of the restored infrared image and the original infrared image. For example, the weight of the original infrared image is 1, and the weight of the restored infrared image is between 0 and 1.

[0109] The weight of the basic thermal field layer represents the overall weight coefficient assigned to the loss term corresponding to the basic thermal field layer during the reconstruction of the three-dimensional temperature field.

[0110] The weight of the structural layer represents the overall weight coefficient assigned to the loss term corresponding to the structural layer during the reconstruction of the three-dimensional temperature field.

[0111] The weights of the detail layer are the overall weight coefficients assigned to the loss terms corresponding to the detail layer during the 3D temperature field reconstruction process.

[0112] By training a cumulative model comprising a basic thermal field layer, a structural layer, and a detail layer, various weights can be obtained. For example, the basic thermal field layer absorbs stable, large-scale thermal radiation distributions from a small number of real-viewpoints and a large number of low-weight pseudo-viewpoints. The weights of the real infrared images are set to 1, while the weights of the repaired infrared images are less than 1 and further multiplied by a pixel-level confidence mask. The basic thermal field layer introduces a low-frequency thermal field loss. Smoothing loss of heat plateau and heat intensity statistics loss Training loss of the basic thermal field layer The expression is as follows: ; in, For Gaussian blur or low-pass filtering, R is the Gaussian rendering operator. This represents the infrared image rendered using only the basic thermal field layer; I is the supervisory image. A thermal plateau mask represents a region where the temperature change is relatively gradual. This represents the gradient corresponding to the infrared image obtained by rendering the base thermal field layer. This represents the nth-order thermal intensity statistic, such as mean, variance, and skewness. , and These correspond to the weights of low-frequency thermal field, isothermal plateau smoothing, and thermal intensity statistical constraints, respectively. The function of this layer is to suppress and smooth false textures on the surface of the device and prevent the overall thermal field from being too bright, too dark, or skewed.

[0113] For the structural layer, a structural residual Gaussian is added to the basic thermal field layer. The addition is located at infrared thermal edges, depth discontinuity edges, thermal anomaly regions, and pseudo-viewpoint reliable regions. The expression defining the importance of the structural graph is as follows: ; in, This is a structural importance graph for the i-th viewpoint. For infrared intensity edges, For depth edge, This is a region of thermal anomaly with a relatively significant local background. is a pixel-level confidence mask, and a, b, c, and d are combination coefficients.

[0114] Training loss of structural layers The expression is as follows: ; in, As the basic thermal field layer, As a structural layer, This represents an image rendered using both the base thermal field layer and the structural layer. For the supervised image of the i-th viewpoint, express The corresponding gradient, express The corresponding gradient. For structural area reconstruction losses, Weights are used for gradient consistency loss. Gaussian appending or densification of structural layers is only allowed when the structural importance exceeds a threshold.

[0115] For the detail layer, after the basic thermal field layer and structural layer have formed a stable thermal field and the main equipment outline, the detail layer will only add a small amount of detail residual Gaussian in the real view region and in the pseudo view region that simultaneously satisfies high confidence and high structural importance. The expression for the detail supervision mask is as follows: ; in, For the detail supervision mask of the i-th view, This is a structural importance graph for the i-th viewpoint. It is a pixel-level credibility mask.

[0116] Training loss of detail layers The expression is as follows: ; in, For the detail supervision mask of the i-th view, To obtain the complete image using three layers of Gaussian rendering, For the supervised image of the i-th viewpoint, For texture or high-frequency detail constraints, they are typically used to preserve details in realistic viewpoints or reliable coarse model areas. Weighting loss for detailed region reconstruction. Weights are anchored to the texture. The detail layer is not responsible for changing the overall thermal field, but only for supplementing the internal texture and small contours of the device in areas that are recognizable at close range.

[0117] In this embodiment, firstly, by differentially configuring the weights of the basic thermal field layer, the structural layer, and the detail layer, a hierarchical gradient-driven approach is achieved for the macroscopic baseline, structural boundaries, and microscopic details during the 3D temperature field reconstruction process. This allows each Gaussian layer to perform its function independently during optimization, resolving the optimization conflict caused by the competition between low-frequency background and high-frequency details in traditional single-weight reconstruction. This improves the overall reconstruction quality and training stability of the 3D temperature field model. Secondly, the basic thermal field layer, structural layer, and detail layer respectively undertake the representation of stable thermal field, thermal discontinuity structure, and high-confidence details, and each layer is directly formed during the training phase. Thirdly, the image weight of the real infrared image is greater than that of the restored infrared image. The real infrared viewpoint always serves as the primary supervision, while the pseudo-viewpoint participates in different levels of the 3D temperature field model with a lower viewpoint weight and pixel-level confidence. This not only supplements the information missing from the sparse viewpoint but also suppresses the contamination of the real temperature distribution by the generated image.

[0118] In one embodiment, the weights of the base thermal field layer include: the weights of low-frequency thermal field, isothermal plateau smoothing, and thermal intensity statistics constraints; the weights of the structure layer include: the weights of structure region reconstruction loss and gradient consistency loss; and the weights of the detail layer include: the weights of detail region reconstruction loss and texture anchoring weights.

[0119] The low-frequency thermal field weight represents the sub-weight coefficient in the basic thermal field layer loss function that constrains the Gaussian elements to fit the overall trend of the macroscopic thermal radiation distribution of the target substation equipment. The low-frequency thermal field weight can drive the basic Gaussian to capture the large-scale smooth thermal field baseline of the equipment surface during the optimization process, such as the overall temperature level of the equipment and the macroscopic layout of cold and warm zones, without being disturbed by local noise or minor thermal field fluctuations.

[0120] The isothermal plateau smoothing weight represents a sub-weight coefficient in the loss function of the basic thermal field layer that constrains the smooth and continuous thermal field values ​​among Gaussian elements within the same isothermal plateau region. This weight acts on the second-order smoothing regularization term of the temperature values ​​of adjacent Gaussian elements in regions where the surface temperature changes gently (i.e., isothermal plateau regions, such as the large plane of a transformer tank or the main section of a bushing). It is used to suppress non-physical thermal field fluctuations generated by the basic Gaussian in textureless flat regions, ensuring that the basic thermal field layer outputs a smooth, continuous, and physically intuitive thermal field distribution in isothermal field regions.

[0121] The weight of the thermal intensity statistics constraint (also known as the weight of the thermal intensity statistics) represents a sub-weight coefficient in the basic thermal field layer loss function that constrains the overall thermal intensity statistical distribution of the basic Gaussian set to maintain consistency with the statistical distribution of the real infrared image. This weight acts on the constraint terms between the mean, variance, and temperature histogram distribution of the basic Gaussian element temperature value set and the corresponding statistics of the real infrared image. It is used to eliminate the overall thermal intensity offset that may be introduced during pseudo-viewpoint generation and optimization, ensuring that the overall temperature benchmark of the three-dimensional temperature field model matches the statistical characteristics of real infrared observations.

[0122] The structural region reconstruction loss weight represents a sub-weight coefficient in the structural layer loss function that constrains the structural Gaussian units to accurately reconstruct the repaired infrared image from the real infrared image in regions of thermal edges, depth discontinuities, and thermal anomalies. This weight acts on the mean square error reconstruction loss term of the pixel-level temperature values ​​in the above three types of key structural regions, driving the structural Gaussian units to obtain sufficient gradient updates in these regions, ensuring that the device outline, component boundaries, and anomalous hotspot edges are accurately located and faithfully represented in the three-dimensional temperature field model.

[0123] The gradient consistency loss weight represents a sub-weight coefficient in the structural layer loss function that ensures the spatial gradient of the infrared image generated by the structural Gaussian primitive projection remains consistent with the spatial gradients of the real infrared image and the depth edge condition map. This weight applies to the gradient magnitude loss term between the structural layer rendered image and the real infrared image at thermal edges, as well as the alignment loss term between the edge directions of the structural layer rendered image and the edge directions of the depth edge condition map. It is used to force the geometric boundaries generated by the structural Gaussian primitives to be strictly aligned with the 3D real spatial geometry, eliminating edge misalignment during the generation process.

[0124] The detail region reconstruction loss weight represents a sub-weight coefficient in the detail layer loss function that constrains the detail Gaussian units to perform high-fidelity reconstruction of the repaired infrared image and the real infrared image in detail regions of high-confidence real or pseudo-viewpoints. This weight acts on the mean square error reconstruction loss term of pixel-level temperature values ​​within high-confidence regions (regions where the pixel-level confidence mask value is close to 1), driving the detail Gaussians to densely arrange themselves in these regions and accurately fit local thermal field gradients, minute temperature differences, and surface texture features, achieving the high-resolution thermal field representation required for close-range fine diagnosis.

[0125] Texture anchoring weights represent sub-weight coefficients in the detail layer loss function that constrain the local texture features of the detail Gaussian units to maintain consistency with the local texture features of the corresponding regions in the restored infrared image and the real infrared image. These weights apply to the structural similarity loss term and the local binary pattern loss term between the detail layer rendered image and the real / restored infrared image within a local window. This ensures that the detail Gaussian retains the spatial patterns of local thermal field changes (such as hotspot shapes and temperature transition morphologies) while fitting the thermal field values, enabling the 3D temperature field model to possess texture features consistent with the real infrared image at the detail level, thus avoiding the texture drift problem where numerical accuracy is compromised but spatial pattern distortion is present.

[0126] In this embodiment, firstly, by setting the weights of the basic thermal field layer and participating in joint optimization, a relatively low weight coefficient constrains the basic Gaussian to only fit the macroscopic thermal radiation baseline and the large-scale smooth thermal field distribution. This allows large-scale primitives to serve as background support during reconstruction without interfering with the structured representation of other layers, ensuring the continuity of the thermal field and computational efficiency of the 3D temperature field model during long-distance rendering. Secondly, by setting the weights of the structural layer and participating in joint optimization, a higher weight coefficient strengthens the contribution ratio of thermal edges, depth discontinuities, and thermal anomaly regions in the loss function. This allows the structural Gaussian to obtain sufficient optimization driving force in gradient updates, ensuring the accurate geometric positioning of key equipment contours and abnormal hotspot boundaries in the 3D temperature field model, significantly improving the accuracy of identifying structural and thermal anomalies in substation equipment. Thirdly, by setting the weights of the detail layer and participating in joint optimization, a moderate weight coefficient establishes a balance between detail preservation and noise suppression. This allows the detail Gaussian to absorb effective thermal field details in high-confidence regions without amplifying pseudo-viewpoint generation errors, ensuring the thermal field resolution and texture fidelity of the 3D temperature field model during close-range fine diagnosis.

[0127] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages of other steps.

[0128] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A temperature field reconstruction method based on video diffusion pseudo-viewpoint enhancement and detail layering, characterized in that, The method includes: Acquire real infrared images of the target substation from multiple real viewpoints, as well as structural condition diagrams corresponding to multiple pseudo viewpoints; the camera poses corresponding to each pseudo viewpoint and the camera poses corresponding to each real viewpoint belong to the same camera trajectory. The infrared edge condition map, depth edge condition map, optional thermal anomaly map, and depth map of the pseudo-viewpoint in the structural condition map are input into the control branch of the video diffusion model, and multi-scale residuals are output. The multi-scale residual and the predicted continuous infrared image corresponding to the real infrared image are processed by the diffusion denoising network of the video diffusion model to obtain the predicted infrared image corresponding to the pseudo-viewpoint. The predicted infrared image is repaired to obtain the repaired infrared image of the pseudo-viewpoint; Based on the repaired infrared image and the real infrared image, the temperature field of the target substation is reconstructed to obtain a three-dimensional temperature field model of the target substation; the three-dimensional temperature field model contains at least one set of Gaussian elements at a detail level, and the set of Gaussian elements is used to render the infrared level image.

2. The method according to claim 1, characterized in that, Obtain the structural condition graphs corresponding to multiple pseudo-viewpoints, including: The camera trajectory is obtained by interpolation based on each real infrared image and the corresponding camera intrinsic and extrinsic parameters of each real infrared image. Based on the trained 3D Gaussian model, render the coarse infrared image, depth image and visible Gaussian projection information corresponding to each of the multiple pseudo-viewpoints in the camera trajectory; Based on the coarse infrared image, the depth image, and the visible Gaussian projection information, determine the coarse infrared gradient of the coarse infrared image and the depth gradient of the depth image; Based on the coarse infrared gradient and the depth gradient, construct the structural condition map corresponding to the pseudo-viewpoint.

3. The method according to claim 1, characterized in that, The method further includes: Each of the real infrared images is input into the video diffusion backbone of the video diffusion model, and the predicted continuous infrared image of the real infrared image is output.

4. The method according to claim 1, characterized in that, The step of repairing the predicted infrared image to obtain the repaired infrared image from the pseudo-viewpoint includes: Consistency estimation is performed on the predicted infrared image to obtain the low-frequency thermal field consistency, structural edge consistency, and temporal consistency of adjacent viewpoints of the predicted infrared image; For each of the predicted infrared images, a pixel-level confidence mask is determined based on the low-frequency thermal field consistency, structural edge consistency, and adjacent viewpoint temporal consistency. Based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint, the repaired infrared image of the pseudo-viewpoint is determined.

5. The method according to claim 4, characterized in that, The step of determining the repaired infrared image of the pseudo-viewpoint based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint includes: Based on the predicted infrared image, the pixel-level confidence mask, and the coarse infrared image of the pseudo-viewpoint, determine the truth-free fusion image of the pseudo-viewpoint; The confidence fusion sequence, composed of the truth-less confidence fusion images of each pseudo-viewpoint, is input into the spatiotemporal recovery head, and the repaired infrared image of the pseudo-viewpoint is output.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the camera focal length and the target depth of the Gaussian elements on the three-dimensional temperature field model relative to the camera; The projection radius of the Gaussian element is determined based on the camera focal length and the target depth. Based on the relationship between the first-level threshold, the second-level threshold, and the projection radius, a detail level that matches the projection radius is determined; the first-level threshold is less than the second-level threshold. The detail level includes at least one of a basic thermal field layer, a structural layer, or a detail layer. The basic thermal field layer forms a basic Gaussian with a coverage area greater than a preset range in the equal thermal field region. The structural layer adds structural Gaussian in the thermal edge, depth discontinuity, and thermal anomaly regions. The detail layer adds detail Gaussian in the high-confidence real or pseudo-viewpoint detail regions. Displays the infrared layer image corresponding to the detail level in the three-dimensional temperature field model.

7. The method according to claim 6, characterized in that, The step of determining the detail level adapted to the projection radius based on the relationship between the first-level threshold, the second-level threshold, and the projection radius includes: When the projected radius is less than the first level threshold, the detailed level adapted by the projected radius is the basic thermal field layer; When the projection radius is greater than or equal to the first level threshold and less than the second level threshold, the detailed level of the projection radius adaptation includes the basic thermal field layer and the structural layer. When the projection radius is greater than or equal to the second level threshold, the detail level adapted by the projection radius includes the basic thermal field layer, the structural layer, and the detail layer.

8. The method according to claim 1, characterized in that, The step of reconstructing the temperature field of the target substation based on the repaired infrared image and the real infrared image to obtain a three-dimensional temperature field model of the target substation includes: Based on the repaired infrared image, the real infrared image, image weights, the weights of the basic thermal field layer, the weights of the structural layer, and the weights of the detail layer, the temperature field of the target substation is reconstructed to obtain a three-dimensional temperature field model of the target substation.

9. The method according to claim 8, characterized in that, The weights of the basic thermal field layer include: the weights of low-frequency thermal field, isothermal plateau smoothing, and thermal intensity statistical constraints; The weights of the structural layer include: structural region reconstruction loss and gradient consistency loss weights; The weights of the detail layer include: detail region reconstruction loss weights and texture anchoring weights.

Images