Multi-modal large model driven inertial platform temperature field generation method

Abstract

The invention belongs to the technical field of precision measurement and intelligent calculation, and particularly relates to a multi-mode large model driven inertial platform temperature field generation method. Comprising the following steps: (1) data acquisition and preprocessing: acquiring a structure appearance image and a multi-working-condition thermodynamic distribution diagram of an inertial platform, and performing space alignment by using affine transformation or perspective transformation; (2) descriptive text generation: inputting a multi-angle RGB image of the inertial platform, extracting spatial features through a pre-trained visual large model, and generating structured text description to describe the inertial platform and the environment thereof in the image; (3) constructing a diffusion model: inputting an inertial platform image and a descriptive text, respectively extracting visual structure features and semantic description features, and controlling the generation direction of a temperature field by utilizing the characteristic of gradual denoising of the diffusion model and combining text semantic guidance; and (4) model training and verification: selecting an optimizer and a learning rate scheduling strategy, and verifying the rationality of a model concern area.

Description

Technical Field

[0001] The present invention belongs to the technical field of precision measurement and intelligent computing, and particularly relates to a method for generating a temperature field of an inertial platform driven by a multi-modal large model. Background Art

[0002] With the development of science and technology, inertial platforms have been widely used and their performance has been improved in many fields. In the aerospace field, inertial platforms provide high-precision navigation and attitude control for aircraft. In unmanned driving technology, inertial navigation is integrated with GPS to enable autonomous driving of vehicles in complex environments. In deep-sea exploration and underground drilling, inertial platforms can provide position and orientation references in environments without satellite signals. However, the performance of inertial platforms can vary significantly under different temperature conditions. Temperature changes can cause zero bias drift, sensitivity changes, and mechanical stress changes in inertial components, thereby affecting navigation accuracy. Therefore, by accurately and realistically reconstructing the temperature of the inertial platform, it is possible to determine in advance the impact of temperature on measurement accuracy, take timely compensation measures, and ensure that the platform can maintain accurate and stable working conditions in complex environments. This is of great significance to effectively ensuring the operational safety, measurement accuracy, and economic efficiency of the system.

[0003] Currently, temperature field reconstruction methods for inertial platforms can be categorized into three main types: sensor-based temperature field reconstruction, physical model-based temperature field reconstruction, and data-driven temperature field reconstruction. Sensor-based temperature field reconstruction methods primarily place temperature sensors at key locations on the inertial platform to directly acquire temperature data. Interpolation or extrapolation algorithms are then used to reconstruct the full-field temperature distribution. This method relies on physical sensor data, resulting in intuitive and reliable results. However, reconstruction accuracy is limited by the number, layout, and accuracy of sensors. Furthermore, a large number of sensors are required, resulting in poor adaptability to nonlinear temperature gradients or complex boundary conditions. Physical model-based temperature field reconstruction methods primarily employ finite element methods, finite volume methods, or boundary element methods to solve heat transfer equations. These methods are free from sensor limitations and are applicable to complex geometries and non-steady-state conditions. However, they require high-precision meshing, iterative solutions, and accurate material parameters, resulting in lengthy computational times and high resource consumption. Data-driven temperature field reconstruction methods are primarily categorized into machine learning and deep learning methods. These methods directly learn temperature field characteristics from data, are suitable for nonlinear problems, and can capture complex spatiotemporal correlations. However, they require a large amount of high-quality training data, are prone to overfitting when data is insufficient, and are poorly adaptable to operating conditions outside the training set. Summary of the Invention

[0004] The purpose of the present invention is to overcome the shortcomings of the existing technology and propose a temperature field generation method for an inertial platform driven by a multimodal large model. The core of this method is to construct a cascaded temperature field reconstruction framework based on a visual-linguistic multimodal large model and a diffusion model. By fusing platform visual features, generative text descriptions and simulated temperature data, end-to-end temperature field reconstruction is achieved.

[0005] The present invention is implemented by adopting the following technical solution: a method for generating a temperature field of an inertial platform driven by a multi-modal large model, comprising the following steps:

[0006] (1) Data acquisition and preprocessing: The structural appearance image of the inertial platform and its multi-operating condition thermal distribution map are collected. Affine transformation or perspective transformation is used to perform spatial alignment based on the field of view difference between the appearance image and the thermal distribution map to ensure that pixels at the same physical location correspond. The appearance image is normalized to the range of [0, 1], and the thermal distribution map linearly maps the original temperature value to the interval of [0, 255]. The structural appearance image of the inertial platform refers to the multi-angle RGB image of the inertial platform, and the multi-operating condition thermal distribution map corresponds one-to-one to the multi-angle RGB image.

[0007] (2) Descriptive text generation: Input multi-angle RGB images of the inertial platform, extract spatial features through the pre-trained visual model, and generate structured descriptive text to describe the inertial platform and its environment in the image;

[0008] (3) Diffusion model construction: Input the inertial platform image and descriptive text, and extract the visual structure features and semantic description features through the image feature extraction network and text encoding model respectively. The gradual denoising characteristics of the diffusion model are used, combined with the text semantic guidance, to control the generation direction of the temperature field;

[0009] (4) Model training and validation: Select the optimizer and learning rate scheduling strategy, define the hybrid loss function, enable the data augmentation module, perform grid search or random search on hyperparameters, evaluate the model stability through K-fold cross validation, adjust the data augmentation strength or network depth, visualize the attention weight map, and verify the rationality of the model's attention area.

[0010] Furthermore, step (1) is specifically as follows: after data acquisition is completed, the multi-condition thermal distribution map is aligned with the appearance image through affine transformation to ensure that the coordinates of the same physical point in the two modes are consistent; assuming that the pixel coordinates of the original thermal distribution map are (x, y), the corresponding appearance image coordinates after transformation are (x′, y′), and the affine transformation matrix is:

[0011]

[0012] in:

[0013] a 11、a 12 、a 21 、a 22 Control rotation, scaling, and shearing;

[0014] t x , t y Control translation;

[0015] Step (2) is as follows: Using the Vision Transformer visual encoder, each RGB image after affine transformation is cropped to H×W resolution, divided into n P×P image blocks, and linearly mapped to embedding vectors:

[0016] z i =E·x i +e pos , i=1,2,...,n

[0017] in:

[0018] is the flattened pixel vector of the i-th image block, P is the resolution of the image block,;

[0019] is the learnable embedding matrix, and D is the target dimension after embedding;

[0020] e pos ∈R D is the position encoding vector; R represents the matrix space composed of real number sets (different superscripts represent different dimensions);

[0021] For images of K perspectives, extract the CLS token features h of the last layer of Vision Transformer visual encoder respectively k ∈R D , through attention-weighted fusion:

[0022]

[0023] Among them, W Q ∈R D×D is the learnable weight matrix, which is randomly generated during model initialization and optimized on the training data through the back-propagation algorithm. D is the target dimension after embedding.

[0024] Next, the extracted Vision Transformer visual features are decoded into structured text. The text generation model architecture uses the Transformer decoder and autoregressive generation: fusion Generate a JSON text sequence for the condition, at time step t, based on the generated word sequence w 1:t-1 Predict the next word w tProbability of:

[0025] P(w t |w 1:t-1 ,h fusion )=softmax(W o ·d t )

[0026] where d t is the hidden state of the decoder at step t, W o is the output projection matrix;

[0027] The loss function uses the joint optimization of cross entropy loss and structural consistency loss:

[0028] L text =αL CE +βL struct , α, β are empirical parameters; L CE : Standard cross entropy loss, supervised word-level generation;

[0029] L struct : Contrastive loss based on field accuracy;

[0030] Step (3) is as follows: first, multimodal feature extraction is performed. The image feature extraction network uses the pre-trained ResNet-50, and the text encoding model uses the CLIP text encoder. The image feature extraction network uses the pre-trained ResNet-50, removes the classification head, and extracts the feature map of the last convolutional layer of ResNet-50:

[0031] C is the number of feature channels;

[0032] The text encoding model uses the CLIP text encoder to semantically embed structured description text:

[0033] F text =TextEncoder(S)∈R D

[0034] Where S is the JSON text generated in step 2;

[0035] Secondly, a temperature field that conforms to physical laws is generated through the diffusion process. Under the joint control of multimodal characteristics, the forward diffusion process gradually adds Gaussian noise to the true temperature field T0 for a total of T steps:

[0036]

[0037] where β t is the noise scheduling coefficient; where q() is the conditional probability distribution, that is, the transfer probability distribution of the forward diffusion process; N represents the Gaussian distribution; I is the unit matrix;

[0038] The reverse denoising process defines the denoising network ∈ θ Predicting noise, conditional injection is achieved through feature concatenation and cross attention:

[0039] 1) Image feature modulation: F img Injected into U-Net layers through spatial adaptive normalization:

[0040]

[0041] in Input features for the lth layer of U-Net; Output feature of the lth layer of U-Net; γ l , β l are learnable affine transformation parameters;

[0042] 2) Text Cross Attention: Add text conditional attention to the Bottleneck layer of U-Net:

[0043]

[0044] Where Q = h t W q ,K=F text W k ,V=F text W v ,h t is the current hidden state (the hidden state feature of U-Net in the tth step of the diffusion process), W q 、W k 、W v is a trainable weight matrix; It is a scaling factor to prevent the dot product result from being too large and causing the gradient to disappear;

[0045] Jointly optimize noise prediction:

[0046]

[0047] is the expectation of time step t, true temperature field T0, and noise ∈; P phys is the physical constraint projection matrix, ∈ θ Noise prediction network; T t is the noisy temperature field at step t; F img is the image feature; F text is the text feature;

[0048] Finally, the learning of generating a temperature field that conforms to physical laws is completed through the conditional generation process:

[0049] 1) Initialization: From pure noise T T~N(0,I) starts;

[0050] 2) Iterative denoising:

[0051] For t=T to 1

[0052]

[0053] where z~N(0,I),α t =1-β t , z is the Gaussian noise added by the back diffusion process (used to avoid mode collapse caused by deterministic prediction), α s is the noise scheduling coefficient; σ t To control the noise intensity, the mean absolute error loss function and the structural similarity loss function are used here:

[0054] Mean absolute error loss function L MAE :

[0055]

[0056] H, W are the height and width of the temperature field spatial resolution; T pred is the temperature field matrix predicted by the model; T gt is the real temperature field matrix;

[0057] Structural similarity loss function L SSIM :

[0058] L SSIM =1-SSIM(T pred ,T gt )

[0059] SSIM() is the structural similarity function;

[0060] Step (4) is as follows: the AdamW optimizer is used in the model training and verification phase. The initial learning rate is gradually increased through linear warm-up followed by cosine annealing scheduling to balance the convergence speed and stability. At the same time, synchronous geometric transformation and modality-specific enhancement are applied to the image and thermal data to improve the generalization ability. During the training process, the model robustness is evaluated based on K-fold cross-validation, and the optimal hyperparameter combination (such as learning rate, loss weight, text attenuation coefficient, noise scheduling initial value, etc.) is determined through random search. The total loss function is:

[0061] L total =λ1·L text +λ2·L MAE +λ3·L SSIM

[0062] Among them, λ1, λ2, λ3 are hyperparameters; Ltext is the text generation cross entropy loss; L MAE is the mean absolute error loss function of the diffusion model; L SSIM is the structural similarity loss function of the diffusion model.

[0063] Compared with most existing thermal field prediction methods based on traditional neural networks or statistical models, the multimodal large model-driven inertial platform temperature field generation method of the present invention has the following beneficial effects: it does not rely on single sensor data, introduces a large visual model and a text description generation module to construct cross-modal semantic associations, and significantly reduces the data requirement in small sample scenarios through pre-training knowledge transfer; adopts a dual-branch adaptive architecture to realize the dynamic fusion of image features and text semantics, enhancing the model's generalization ability for complex working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a diagram showing the steps for implementing the technical solution.

[0065] Figure 2 This is the overall model architecture diagram of the multi-modal large model driven inertial platform temperature field generation method proposed in this invention.

[0066] Figure 3 The visualization result diagram of the temperature field of the inertial platform generated for the model of the present invention. The first figure from left to right (initial stage): the colored noise is randomly distributed, reflecting the initial noise state of the diffusion model input; the second figure (aggregation stage): the gray-white tone dominates, and the noise migrates directionally to the potential high-temperature area to form a cluster structure; the third figure (gradient forming): a light green temperature gradient is superimposed on the gray-white base, and the spatial positioning of the core area (corresponding to high temperature) tends to be stable; the fourth figure (three-dimensional thermal): the two-dimensional prediction is finally converted into a three-dimensional temperature field, the red / yellow protrusions represent the high-temperature area, and the green gradient shows the heat diffusion boundary. The four figures coherently present the entire process of the model from noise denoising, physical law extraction to three-dimensional temperature field reconstruction, verifying the algorithm's ability to model complex heat conduction boundaries. DETAILED DESCRIPTION

[0067] A method for generating a temperature field of an inertial platform driven by a multi-modal large model comprises the following steps:

[0068] (1) Data acquisition and preprocessing: The structural appearance image of the inertial platform and its multi-operating condition thermal distribution map are collected. Affine transformation or perspective transformation is used to perform spatial alignment based on the field of view difference between the appearance image and the thermal distribution map to ensure that pixels at the same physical location correspond. The appearance image is normalized to the range of [0, 1], and the thermal distribution map linearly maps the original temperature value to the interval of [0, 255]. The structural appearance image of the inertial platform refers to the multi-angle RGB image of the inertial platform, and the multi-operating condition thermal distribution map corresponds one-to-one to the multi-angle RGB image.

[0069] (2) Descriptive text generation: Input multi-angle RGB images of the inertial platform, extract spatial features through the pre-trained visual model, and generate structured descriptive text to describe the inertial platform and its environment in the image;

[0070] (3) Diffusion model construction: Input the inertial platform image and descriptive text, and extract the visual structure features and semantic description features through the image feature extraction network and text encoding model respectively. The gradual denoising characteristics of the diffusion model are used, combined with the text semantic guidance, to control the generation direction of the temperature field;

[0071] (4) Model training and validation: Select the optimizer and learning rate scheduling strategy, define the hybrid loss function, enable the data augmentation module, perform grid search or random search on hyperparameters, evaluate the model stability through K-fold cross validation, adjust the data augmentation strength or network depth, visualize the attention weight map, and verify the rationality of the model's attention area.

[0072] Furthermore, step (1) is specifically as follows: after data acquisition is completed, the multi-condition thermal distribution map is aligned with the appearance image through affine transformation to ensure that the coordinates of the same physical point in the two modes are consistent; assuming that the pixel coordinates of the original thermal distribution map are (x, y), the corresponding appearance image coordinates after transformation are (x′, y′), and the affine transformation matrix is:

[0073]

[0074] in:

[0075] a 11 、a 12 、a 21 、a 22 Control rotation, scaling, and shearing;

[0076] t x , t y Control translation;

[0077] Step (2) is as follows: Using the Vision Transformer (ViT) visual encoder, each RGB image after affine transformation is cropped to H×W resolution, divided into n P×P image patches, and linearly mapped to embedding vectors:

[0078] z i =E·x i +e pos , i=1,2,...,n

[0079] in:

[0080] is the flattened pixel vector of the i-th image block, P is the resolution of the image block,;

[0081] is the learnable embedding matrix, and D is the target dimension after embedding;

[0082] e pos ∈R D is the position encoding vector; R represents the matrix space composed of real number sets (different superscripts represent different dimensions);

[0083] For images from K perspectives, extract the CLStoken features h of the last layer of the Vision Transformer (ViT) visual encoder k ∈R D , through attention-weighted fusion:

[0084]

[0085] Among them, W Q ∈R D×D is the learnable weight matrix, which is randomly generated during model initialization and optimized on the training data through the back-propagation algorithm. D is the target dimension after embedding.

[0086] Next, the extracted Vision Transformer (ViT) visual features are decoded into structured text. The text generation model architecture uses the Transformer decoder and autoregressive generation: fusion Generate a JSON text sequence for the condition, at time step t, based on the generated word sequence w 1:t-1 Predict the next word w t Probability of:

[0087] P(w t |w 1:t-1 ,h fusion )=softmax(W o ·d t )

[0088] where d t is the hidden state of the decoder at step t, W o is the output projection matrix;

[0089] The loss function uses the joint optimization of cross entropy loss and structural consistency loss:

[0090] L text =αL CE +βL struct , α, β are empirical parameters;

[0091] L CE: Standard cross entropy loss, supervised word-level generation;

[0092] L struct : Contrastive loss based on field accuracy;

[0093] Step (3) is as follows: first, multimodal feature extraction is performed. The image feature extraction network uses the pre-trained ResNet-50, and the text encoding model uses the CLIP text encoder. The image feature extraction network uses the pre-trained ResNet-50, removes the classification head, and extracts the feature map of the last convolutional layer of ResNet-50:

[0094] C is the number of feature channels;

[0095] The text encoding model uses the CLIP text encoder to semantically embed structured description text:

[0096] F text =TextEncoder(S)∈R D

[0097] Where S is the JSON text generated in step 2;

[0098] Secondly, a temperature field that conforms to physical laws is generated through the diffusion process. Under the joint control of multimodal features, the forward diffusion process gradually adds Gaussian noise to the true temperature field T0 for a total of T steps:

[0099]

[0100] where β t is the noise scheduling coefficient; where q() is the conditional probability distribution, that is, the transfer probability distribution of the forward diffusion process; N represents the Gaussian distribution; I is the unit matrix;

[0101] The reverse denoising process defines the denoising network ∈ θ Predicting noise, conditional injection is achieved through feature concatenation and cross attention:

[0102] 1) Image feature modulation: F img Injected into U-Net layers through spatial adaptive normalization:

[0103]

[0104] in Input features for the lth layer of U-Net; Output feature of the lth layer of U-Net; γ l , β l are learnable affine transformation parameters;

[0105] 2) Text Cross Attention: Add text conditional attention to the Bottleneck layer of U-Net:

[0106]

[0107] Where Q = h t W q ,K=F text W k ,V=F text W v ,h t is the current hidden state (the hidden state feature of U-Net in the tth step of the diffusion process), W q 、W k 、W v is a trainable weight matrix; It is a scaling factor to prevent the dot product result from being too large and causing the gradient to disappear;

[0108] Jointly optimize noise prediction:

[0109]

[0110] is the expectation of time step t, true temperature field T0, and noise ∈; P phys is the physical constraint projection matrix, ∈ θ Noise prediction network; T t is the noisy temperature field at step t; F img is the image feature; F text is the text feature;

[0111] Finally, the learning of generating a temperature field that conforms to physical laws is completed through the conditional generation process:

[0112] 1) Initialization: From pure noise T T ~N(0,I) starts;

[0113] 2) Iterative denoising:

[0114] For t=T to 1

[0115]

[0116] where z~N(0,I),α t =1-β t , z is the Gaussian noise added by the back diffusion process (used to avoid mode collapse caused by deterministic prediction), α s is the noise scheduling coefficient; σ t To control the noise intensity, the mean absolute error loss function and the structural similarity loss function are used here:

[0117] Mean absolute error loss function L MAE :

[0118]

[0119] H, W are the height and width of the temperature field spatial resolution; T pred is the temperature field matrix predicted by the model; T gt is the real temperature field matrix;

[0120] Structural similarity loss function L SSIM :

[0121] L SSIM =1-SSIM(T pred ,T gt )

[0122] SSIM() is the structural similarity function;

[0123] Step (4) is as follows: AdamW optimizer is used in the model training and verification phase. The initial learning rate is gradually increased through linear warm-up followed by cosine annealing scheduling to balance the convergence speed and stability. At the same time, synchronous geometric transformation (rotation, translation, scaling) and modality-specific enhancement (image color perturbation, thermal noise injection) are applied to the image and thermal data to improve generalization ability. During the training process, the model robustness is evaluated based on K-fold cross-validation, and the optimal hyperparameter combination (such as learning rate, loss weight, text attenuation coefficient, noise scheduling initial value, etc.) is determined through random search. The total loss function is:

[0124] L total =λ1·L text +λ2·L MAE +λ3·L SSIM

[0125] Among them, λ1, λ2, λ3 are hyperparameters; L text is the text generation cross entropy loss; L MAE is the mean absolute error loss function of the diffusion model; L SSIM is the structural similarity loss function of the diffusion model (the methods used in step 4 are all well-known methods in the art).

[0126] The scope of protection claimed by the present invention is not limited to the above specific embodiments. For those skilled in the art, the present invention may have various variations and modifications. Any modifications, improvements and equivalent substitutions made within the concept and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for generating temperature field of an inertial platform driven by a multi-modal large model, characterized in that: The steps include: (1) Data acquisition and preprocessing: The structural appearance image of the inertial platform and its multi-operating condition thermal distribution map are collected. Affine transformation or perspective transformation is used to perform spatial alignment based on the field of view difference between the appearance image and the thermal distribution map to ensure that pixels at the same physical location correspond. The appearance image is normalized to the range of [0, 1], and the thermal distribution map linearly maps the original temperature value to the interval of [0, 255]. The structural appearance image of the inertial platform refers to the multi-angle RGB image of the inertial platform, and the multi-operating condition thermal distribution map corresponds one-to-one to the multi-angle RGB image. (2) Descriptive text generation: Input multi-angle RGB images of the inertial platform, extract spatial features through the pre-trained visual model, and generate structured descriptive text to describe the inertial platform and its environment in the image; (3) Diffusion model construction: Input the inertial platform image and descriptive text, and extract the visual structure features and semantic description features through the image feature extraction network and text encoding model respectively. The gradual denoising characteristics of the diffusion model are used, combined with the text semantic guidance, to control the generation direction of the temperature field; (4) Model training and validation: Select the optimizer and learning rate scheduling strategy, define the hybrid loss function, enable the data augmentation module, perform grid search or random search on hyperparameters, evaluate the model stability through K-fold cross validation, adjust the data augmentation strength or network depth, visualize the attention weight map, and verify the rationality of the model's attention area.

2. The method for generating a temperature field of an inertial platform driven by a multi-modal large model according to claim 1, wherein: Step (1) is as follows: after data acquisition is completed, the multi-condition thermal distribution map is aligned with the appearance image through affine transformation to ensure that the coordinates of the same physical point in the two modes are consistent; assuming that the pixel coordinates of the original thermal distribution map are (x, y), the corresponding appearance image coordinates after transformation are (x′, y′), and the affine transformation matrix is: in: a 11 、a 12 、a 21 、a 22 Control rotation, scaling, and shearing; t x , t y Control translation; Step (2) is as follows: Using the Vision Transformer visual encoder, each RGB image after affine transformation is cropped to H×W resolution, divided into n P×P image blocks, and linearly mapped to embedding vectors: z i =E·x i +e pos ,i=1,2,…,n in: is the flattened pixel vector of the i-th image block, and P is the resolution of the image block; is the learnable embedding matrix, and D is the target dimension after embedding; e pos ∈R D is the position encoding vector; R represents the matrix space composed of real number sets; For images of K perspectives, extract the CLS token features h of the last layer of Vision Transformer visual encoder respectively k ∈R D , through attention-weighted fusion: Among them, W Q ∈R D×D It is a learnable weight matrix, which is randomly generated when the model is initialized and optimized on the training data through the back propagation algorithm; Next, the extracted Vision Transformer visual features are decoded into structured text. The text generation model architecture uses the Transformer decoder and autoregressive generation: fusion Generate a JSON text sequence for the condition, at time step t, based on the generated word sequence w 1:t-1 Predict the next word w t Probability of: P(w t |w 1:t-1 ,h fusion )=softmax(W o ·d t ) where d t is the hidden state of the decoder at step t, W o is the output projection matrix; The loss function uses the joint optimization of cross entropy loss and structural consistency loss: L text =αL CE +βL struct , α, β are empirical parameters; L CE : Standard cross entropy loss, supervised word-level generation; L struct : Contrastive loss based on field accuracy; Step (3) is as follows: first, multimodal feature extraction is performed. The image feature extraction network uses the pre-trained ResNet-50, and the text encoding model uses the CLIP text encoder. The image feature extraction network uses the pre-trained ResNet-50, removes the classification head, and extracts the feature map of the last convolutional layer of ResNet-50: C is the number of feature channels; The text encoding model uses the CLIP text encoder to semantically embed structured description text: F text =TextEncoder(S)∈R D Where S is the JSON text generated in step 2; Secondly, a temperature field that conforms to physical laws is generated through the diffusion process. Under the joint control of multimodal characteristics, the forward diffusion process gradually adds Gaussian noise to the true temperature field T0 for a total of T steps: where β t is the noise scheduling coefficient; where q() is the conditional probability distribution, that is, the transfer probability distribution of the forward diffusion process; N represents the Gaussian distribution; I is the unit matrix; The reverse denoising process defines the denoising network ∈ θ Predicting noise, conditional injection is achieved through feature concatenation and cross attention: 1) Image feature modulation: F img Injected into U-Net layers through spatial adaptive normalization: in Input features for the lth layer of U-Net; Output feature of the lth layer of U-Net; γ l , β l are learnable affine transformation parameters; 2) Text Cross Attention: Add text conditional attention to the Bottleneck layer of U-Net: Where Q = h t W q ,K=F text W k ,V=F text W v ,h t is the current hidden state, W q 、W k 、W v is a trainable weight matrix; It is a scaling factor to prevent the dot product result from being too large and causing the gradient to disappear; Jointly optimize noise prediction: is the expectation of time step t, true temperature field T0, and noise ∈; P phys is the physical constraint projection matrix, ∈ θ is the noise prediction network; T t is the noisy temperature field at step t; F img is the image feature; F text is the text feature; Finally, the learning of generating a temperature field that conforms to physical laws is completed through the conditional generation process: 1) Initialization: From pure noise T T ~N(0,I) starts; 2) Iterative denoising: For t=T to 1 where z~N(0,I),α t =1-β t , z is the Gaussian noise added in the back diffusion process to avoid mode collapse caused by deterministic prediction; α s is the noise scheduling coefficient; σ t To control the noise intensity, the mean absolute error loss function and the structural similarity loss function are used here: Mean absolute error loss function L MAE : H, W are the height and width of the temperature field spatial resolution; T pred is the temperature field matrix predicted by the model; T gt is the real temperature field matrix; Structural similarity loss function L SSIM : L SSIM =1-SSIM(T pred ,T gt ) SSIM() is the structural similarity function; Step (4) is as follows: the AdamW optimizer is used in the model training and validation phase. The initial learning rate is gradually increased through linear warm-up followed by cosine annealing scheduling to balance the convergence speed and stability. At the same time, synchronous geometric transformation and modality-specific enhancement are applied to the image and thermal data to improve the generalization ability. During the training process, the model robustness is evaluated based on K-fold cross-validation, and the optimal hyperparameter combination is determined through random search. The total loss function is: L total =λ1·L text +λ2·L MAE +λ3·L SSIM Among them, λ1, λ2, λ3 are hyperparameters; L text is the text generation cross entropy loss; L MAE is the mean absolute error loss function of the diffusion model; L SSIM is the structural similarity loss function of the diffusion model.

Images