Infrared image and visible light image composite temperature measurement method, system and device

Abstract

This invention relates to the field of intelligent image processing and temperature detection, and discloses a method, system, and device for composite temperature measurement using infrared and visible light images. The composite temperature measurement method includes the following steps: synchronously acquiring visible light images, infrared data, and wind speed / medium parameters through a hardware triggering mechanism; constructing a joint optimization objective including heat conduction equations and convection effects by combining environmental parameters; extracting texture / radiation features using a dual-stream encoder, and achieving multi-scale feature interaction through frequency domain encoding; decoupling network training and physical equation solving, and iteratively updating using a pre-conditional conjugate gradient method; calculating adaptive weights based on visible light / temperature field gradients, and outputting visualized results in conjunction with anomaly detection. This invention improves the accuracy of temperature field reconstruction and the sensitivity of anomaly detection through hardware synchronous acquisition, frequency domain feature fusion, block-based alternating optimization, and a dynamic weighting mechanism, achieving highly robust composite temperature measurement.

Description

Technical Field

[0001] This invention relates to the field of intelligent image processing and temperature detection technology, and in particular to a method, system and device for composite temperature measurement of infrared and visible light images. Background Technology

[0002] With the increasing demand for high-precision temperature field measurement in fields such as industrial inspection and power line inspection, composite temperature measurement technology based on the fusion of infrared and visible light has become a research hotspot. Factors such as dynamic thermal process monitoring in complex industrial scenarios, multi-physics coupling effects, and environmental interference place higher demands on the spatiotemporal consistency, physical rationality, and anomaly detection sensitivity of temperature field reconstruction, necessitating the development of novel fusion sensing and optimization computing methods.

[0003] Existing technical solutions mostly employ an asynchronous acquisition architecture using visible light and infrared sensors, achieving data synchronization through software timestamp alignment. In the feature fusion stage, a dual-stream convolutional network is typically constructed to extract visible light texture features and infrared radiation features separately, using a channel attention mechanism for spatial domain weighted fusion. Temperature field reconstruction is often based on end-to-end deep learning models, using infrared observation data as a supervisory signal to train network parameters. The introduction of physical constraints is mainly achieved by adding residual terms from the heat conduction equation to the loss function, using stochastic gradient descent for joint optimization. Anomaly detection is generally based on statistical thresholding of infrared temperature data, identifying thermal anomaly regions by setting a fixed gradient threshold.

[0004] While existing technologies have achieved preliminary fusion of multimodal data, some shortcomings remain: software-level synchronization schemes struggle to eliminate millisecond-level timing deviations due to differences in sensor hardware response delays and transmission link jitter, leading to the accumulation of spatial registration errors in multi-source data during motion scenarios; purely data-driven temperature reconstruction models lack explicit modeling of physical laws such as heat conduction and convection, easily generating anomalous solutions that violate thermodynamic laws in scenarios with abrupt changes in medium properties or strong wind interference; spatial domain feature fusion mechanisms struggle to effectively decouple high-frequency details from low-frequency contour information, easily causing effective signals to be submerged by noise in complex texture regions; end-to-end joint optimization strategies often cause oscillations or even divergence in the optimization process due to differences in the convergence characteristics of data terms and physical constraint terms, limiting the computational efficiency of large-scale scenarios; anomaly detection methods based on a single infrared threshold cannot distinguish between real thermal hazards and sensor noise or optical interference, resulting in a high misjudgment rate in typical industrial interference scenarios such as metal reflection and steam obstruction. Summary of the Invention

[0005] The purpose of this invention is to provide a composite temperature measurement method, system, and device for infrared and visible light images, which solves the problems of spatiotemporal mismatch of multi-source data, physical inaccuracy of temperature reconstruction, inefficiency of cross-modal feature fusion, difficulty in convergence of optimization process, and insufficient sensitivity of anomaly detection in the prior art.

[0006] To achieve the above objectives, the present invention provides a composite temperature measurement method using infrared and visible light images, comprising the following steps: Visible light images, infrared radiation data, and environmental parameters are acquired through a synchronous acquisition unit. Based on the environmental parameters, a joint optimization objective is constructed that includes the heat conduction equation and convection constraints, wherein the parameters of the heat conduction equation are determined by the medium properties in the environmental parameters; The visible light image and infrared radiation data are input into a dual-stream encoder to extract visible light texture features and infrared radiation features, and cross-modal joint feature representation is generated by frequency domain coding fusion. Based on the joint optimization objective, the cross-modal joint feature representation is solved by block-alternating optimization to obtain the preliminary temperature field distribution; Dynamic confidence weights are calculated based on the gradient information of the visible light image and the gradient distribution of the preliminary temperature field, and the final temperature field is generated and the visualization results are output.

[0007] This invention also provides a composite temperature measurement system combining infrared and visible light images, comprising: Multi-source synchronous acquisition module: integrates visible light sensor, infrared array and environmental detection unit; Physical constraint modeling module: Real-time calculation of heat conduction equations and convection constraint terms; Neural feature fusion module: realizes dual-stream coding and frequency domain fusion; Dynamic optimization solution module: Executes the block-alternating optimization algorithm; Temperature visualization module: Generates temperature distribution maps with confidence weights.

[0008] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the composite temperature measurement method of infrared image and visible light image as described above.

[0009] In summary, the present invention has at least one of the following beneficial technical effects: 1. This invention achieves high-precision synchronous acquisition of visible light, infrared and environmental parameters through hardware-level synchronous triggering and FPGA delay compensation technology, effectively solving the problem of multimodal data timing misalignment in traditional methods, and providing a spatiotemporally aligned input data foundation for subsequent fusion processing.

[0010] 2. This invention overcomes the shortcomings of purely data-driven methods that may violate the laws of thermodynamics by incorporating the heat conduction equation and convection constraints into the optimization objective and driving the neural network parameter update through physical laws, so that the reconstruction results have both data characteristics and physical consistency.

[0011] 3. This invention employs a frequency domain coding fusion strategy to achieve deep interaction between visible light texture features and infrared radiation features in the frequency domain space, effectively capturing complementary information between high-frequency details and low-frequency contours, and improving the ability to restore temperature field details in complex scenes.

[0012] 4. This invention decouples network training from physical equation solving through a block-based alternating optimization strategy, and accelerates the solution of linear systems using the preconditional conjugate gradient method, thereby significantly reducing computational complexity while ensuring the strictness of physical constraints.

[0013] 5. Based on the dynamic weighted fusion mechanism of visible light gradient and temperature gradient, this invention can adaptively identify abnormal areas such as measurement noise and occlusion. Through audible and visual alarms and visualization enhancement, it can realize the real-time location of thermal hazards and improve the practicality of the system in industrial inspection. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the module architecture of the present invention; Figure 3 This is a schematic diagram of the computer device structure of the present invention. Detailed Implementation

[0015] The following is in conjunction with the appendix Figure 1 -Appendix Figure 3 The present invention will be further described in detail below.

[0016] This invention provides a composite temperature measurement method using infrared and visible light images. By employing hardware synchronous acquisition, frequency domain feature fusion, block-based alternating optimization, and dynamic weighting mechanisms, it solves the problems of spatiotemporal mismatch of multi-source data, physical inaccuracy in temperature reconstruction, inefficiency of cross-modal feature fusion, difficulty in convergence during optimization, and insufficient sensitivity in anomaly detection in existing technologies.

[0017] like Figure 1 As shown, the combined infrared and visible light image temperature measurement method may include the following steps: S1. Acquire visible light images, infrared radiation data, and environmental parameters through the synchronous acquisition unit; In this embodiment, the synchronous acquisition unit achieves spatiotemporal aligned acquisition of multi-source data through a heterogeneous sensor fusion architecture. The visible light image acquisition module employs a high dynamic range imaging sensor, whose photosensitive unit array is arranged based on a Bayer filter structure, enabling simultaneous capture of visible light radiation information from the red, green, and blue channels. The sensor incorporates an automatic exposure control loop, dynamically adjusting the integration time and analog gain by analyzing the scene brightness histogram in real time, ensuring the acquisition of detailed visible light image sequences under both strong light and low-light conditions. ;in, Image height; Image width; For timestamps; Indicates the location place, time The visible light image intensity value. Preferably, the sensor output image resolution can be configured to 1280×720 to 3840×2160 pixels, with a frame rate adapted to 25-60fps to meet the real-time requirements of different application scenarios.

[0018] The infrared radiation data acquisition module employs an uncooled microbolometer array, whose sensing element operates based on the thermal resistance variation characteristics of a vanadium oxide thin film. Each thermistor corresponds to a pixel in the spatial sampling grid, generating a two-dimensional radiation distribution map by measuring the infrared radiation flux of the target object in the 8-14μm band. ;in, Indicates the location place, time The infrared radiant flux value. Preferably, the array and the visible light sensor achieve field-of-view matching through a coaxial optical design, and an active temperature compensation algorithm is used to eliminate the influence of ambient temperature fluctuations on measurement accuracy, ensuring that the mapping relationship between the radiant flux data and the real temperature field remains stable.

[0019] The environmental parameter monitoring unit comprises a three-dimensional wind speed detection subsystem, an environmental temperature and humidity detection subsystem, and a data synchronization control subsystem. The three-dimensional wind speed detection is based on the ultrasonic time-of-flight principle, arranging multiple pairs of ultrasonic transducers in an orthogonal coordinate system. The wind speed vector is calculated by measuring the time difference of sound wave propagation in the downwind and upwind directions. ;in, For a moment The three-dimensional wind speed vector; for Directional wind speed component; for Directional wind speed component; for Directional wind speed component. Preferably, the detection unit has a built-in turbulence filtering algorithm, which can effectively suppress the influence of instantaneous wind speed fluctuations on the measurement results and output smoothed wind speed component data.

[0020] Ambient temperature detection uses a contact temperature probe, whose sensing element is tightly attached to the heat-conducting substrate of the sensor housing, acquiring the temperature value of the air surrounding the device through heat conduction. Preferably, the probe is covered with a heat-insulating material layer to minimize the interference of sensor self-heating on the measurement results. Relative humidity detection is based on the capacitive sensing principle. The moisture-sensing medium is a polymer film, whose dielectric constant changes with ambient humidity. The relative humidity is obtained by measuring the change in capacitance. .

[0021] The timing synchronization control unit achieves strict synchronization of multi-sensor data acquisition through hardware-level trigger signals. The master clock signal of the visible light sensor is divided by a phase-locked loop circuit to generate a synchronization pulse. This pulse signal simultaneously triggers the integration cycle start of the infrared sensor and the sample-and-hold circuit of the environmental parameter acquisition module. Preferably, the synchronization controller uses an FPGA to achieve nanosecond-level precision delay compensation. The trigger timing offset of each sensor is dynamically adjusted through programmable logic units to ensure that the visible light image frame, infrared radiation data frame, and environmental parameter sampling points are aligned on the time axis, with the maximum synchronization error controlled within 5ms.

[0022] The data preprocessing unit standardizes the raw acquired signals. Visible light images are converted into linear RGB color space data after de-mosaicing and white balance correction, while infrared radiation data undergoes non-uniformity correction algorithms to eliminate pixel response differences. Environmental parameter data is filtered by moving average and then bound to spatiotemporal label information to form a multimodal data stream with a unified time reference. Preferably, the preprocessing process is completed in a dedicated image signal processor, with hardware acceleration ensuring real-time processing capabilities.

[0023] S2. Construct a joint optimization objective based on environmental parameters, which includes the heat conduction equation and convection constraints, wherein the parameters of the heat conduction equation are determined by the medium properties in the environmental parameters; In this embodiment, the construction process of the joint optimization objective deeply integrates the basic principles of thermodynamics with real-time environmental parameters. Through mathematical modeling, physical laws are transformed into calculable constraints, thereby scientifically guiding the temperature field reconstruction process. Specifically, the optimization objective consists of three parts: data fitting terms, heat conduction constraint terms, and convection constraint terms. Its design fully considers the physical nature of energy conservation and mass transport during the temperature field evolution process.

[0024] In practical implementation, the mathematical expression of the heat conduction constraint term originates from the partial differential equation form of Fourier's heat conduction law. Its discretized representation achieves physical consistency constraints through a combination of the spatial second derivative and the time first derivative, as shown below: ; in: This is the heat conduction loss term, used to constrain the physical consistency of the temperature field over time; For the Laplace operator of the temperature field; This is the partial derivative of temperature with respect to time; The density of the medium is obtained from environmental parameters; Specific heat capacity is obtained from environmental parameters; Thermal conductivity, obtained from environmental parameters; It is the square of the L2 norm.

[0025] The convection constraint term is constructed based on the convection-diffusion equation described by Euler, and its loss function is in the form of: ; in, This represents the heat convection loss term, limiting the impact of wind speed on temperature field migration; The measured three-dimensional wind speed vector; This is the first-order gradient vector of temperature, representing the direction and rate of temperature change; This is the scalar product of the convection terms. Preferably, to eliminate the influence of wind speed measurement noise on gradient calculation, the wind speed data is processed by Kalman filtering before input, and its state transition matrix is ​​dynamically updated according to the fluid continuity equation.

[0026] The joint optimization objective is formed by weighted fusion of observation data errors and physical constraints: in: This is the total loss function; This represents the temperature prediction value generated from multimodal features; The temperature value observed by the infrared sensor; Weights for the heat conduction term; For the weights of the convection term. Preferably, the weight coefficients are normalized using a soft-maximum function to ensure... + The constraint is ≤1.

[0027] The mechanism for introducing physical constraint terms is specifically manifested in the heat conduction loss during reverse propagation. The gradient information is applied to the temperature field prediction module through the chain rule, forcing the network output to satisfy... The physical laws governing convection losses; and convection losses The gradient then corrects the parameter update direction of the feature fusion layer, so that the predicted temperature field maintains dynamic consistency with the measured wind speed field in terms of spatial distribution.

[0028] Preferably, the joint optimization objective is solved using the alternating direction multiplier method based on ADMM, which decomposes the original problem into alternating iterations of a data-driven subproblem and a physical constraint subproblem. In each iteration, the physical constraint parameters are fixed first to update the neural network weights, then the network parameters are fixed to solve for the temperature field distribution that satisfies the partial differential equation, and finally global convergence is achieved by updating through Lagrange multipliers.

[0029] S3. Input visible light images and infrared radiation data into a dual-stream encoder, extract visible light texture features and infrared radiation features, and generate cross-modal joint feature representations through frequency domain coding fusion. In this embodiment, the dual-stream encoder employs a heterogeneous network architecture to achieve feature extraction and fusion of visible light and infrared modes, enhancing the complementarity of cross-modal features through frequency domain coding. The visible light branch is constructed based on a deep residual network, whose backbone network contains multiple residual modules, each consisting of a convolutional layer, a batch normalization layer, and skip connections. The input is a visible light image. After preprocessing, the data enters the network, where multi-scale texture features are extracted through layer-by-layer convolution operations, ultimately outputting a high-dimensional semantic feature vector. Preferably, the depth of the residual network can be configured to 18 to 34 layers. By adjusting the number of channels in the bottleneck layer, the computational efficiency and feature representation capability can be balanced, and the output dimension can be adjusted. It adapts to 256 to 512 dimensions to meet the needs of different scenarios.

[0030] The infrared branch uses a multilayer sensor structure to process radiative flux data. The network has 3 to 5 fully connected layers, with the number of nodes per layer adjustable between 128 and 256. After flattening the input data, radial features are extracted through layer-by-layer nonlinear transformation. The activation function uses ReLU to enhance the model's sparse representation capability. Preferably, the MLP introduces a layer normalization operation in the last layer to eliminate the influence of feature magnitude differences on the subsequent fusion process, ensuring... and They exist in a space with the same dimensions.

[0031] The frequency domain coding module performs spectral space mapping on the dual-stream features, which is mathematically expressed as follows: ; in: This is the feature representation after frequency domain coding; This is the original feature vector; For frequency domain coding levels; This represents a vector concatenation operation; The index of the current level for frequency domain coding; For the first The frequency coefficient of the level; Pi is a constant. Preferably, the frequency coefficient... The exponential growth allows the encoded features to capture both local details and global contextual information simultaneously.

[0032] The cross-modal fusion stage employs a weighted branch fusion strategy, the expression of which is: ; in: The final joint feature output is used for temperature prediction; For the first The weighting coefficients of each fusion branch; For the first The weight matrix of each fusion layer; For the first Bias terms for each fusion layer; This represents the concatenation operation of two vectors; The number of fully connected fusion layers; It is a linear rectification activation function.

[0033] The training process of the dual-stream encoder employs an end-to-end optimization strategy, with the parameters of the visible light branch and the infrared branch jointly updated through backpropagation. Preferably, during network initialization, the visible light branch is loaded with weights pre-trained on a large image dataset, while the infrared branch is initialized using Xavier methods to accelerate model convergence and improve the transferability of feature representations. The frequency domain coding parameters are fixed to a preset frequency basis function to avoid introducing additional trainable parameters that could lead to overfitting.

[0034] S4. Based on the joint optimization objective, the cross-modal joint feature representation is solved by block-alternating optimization to obtain the preliminary temperature field distribution; In this embodiment, the block-based alternating optimization solution process employs a variable decomposition-based optimization framework, breaking down the joint optimization problem into parallel subproblems for iterative solving. Specifically, the optimization variables are defined as triples. The network parameters ΘΘ include the weight matrix and bias terms of the two-stream encoder and the fusion layer, the temperature field TT is the two-dimensional spatial distribution matrix to be reconstructed, and the residual terms... Used to characterize systematic errors caused by unmodeled physical effects.

[0035] Network parameter update phase: fixed temperature field With residuals The observation loss is calculated using the backpropagation algorithm. For network parameters The gradient. In practice, the adaptive moment estimator optimizer (Adam) is used to perform parameter updates: ; in, For neural network model parameters; This refers to the current iteration round; The learning rate; For observation loss; To calculate the gradient of the network parameters, its learning rate is... The parameters are dynamically adjusted using a cosine annealing strategy. Preferably, gradient pruning is employed during gradient calculation to limit the parameter update step size to the range [−0.1, 0.1], preventing the optimization process from diverging due to gradient explosion.

[0036] Temperature field solution stage: fix the updated network parameters The heat conduction equation and convection constraint terms are discretized into a system of linear equations. The coefficient matrix It is composed of the discrete form of the Laplace operator for the heat conduction term and the upwind difference scheme for the convection term, with the right-hand vector... Includes infrared observation data and residuals The contribution of the system is discussed. The linear system is solved iteratively using the preconditional conjugate gradient method (PCG): ; in, The updated temperature field; This is the preconditional conjugate gradient method; This is the sparse coefficient matrix constructed by linearizing the heat conduction equation; This is the right-hand vector formed by combining the heat source term with the initial estimate.

[0037] Residual term update stage: based on the updated temperature field The residual terms are corrected using an explicit time integration method: ; in, For the first The residual tensor after the next iteration; This is the residual term, used to fit unmodeled physical errors; For the first The residual term tensor of the next iteration; For time step; The time derivative of the residual; This serves as the index for the iteration rounds. Preferably, a viscous damping term is introduced to suppress high-frequency oscillations in the residual term.

[0038] The block optimization strategy decouples the optimization objectives of data-driven and physical constraints, allowing network parameter updates to focus on feature representation learning, while the temperature field solution strictly follows thermodynamic laws. The residual term dynamically compensates for physical effects not covered by the model (such as radiative heat dissipation and latent heat of phase change), ultimately achieving high-fidelity temperature field reconstruction.

[0039] S5. Calculate the dynamic confidence weight based on the gradient information of the visible light image and the gradient distribution of the preliminary temperature field, generate the final temperature field and output the visualization results. In this embodiment, the calculation of dynamic confidence weights and the optimization of the temperature field are achieved through multi-scale gradient analysis and an adaptive fusion strategy, aiming to improve the spatial consistency and anti-interference capability of the temperature field reconstruction results. (Visible light image gradient magnitude) The calculation is implemented using the Sobel operator, and its horizontal and vertical gradient components are obtained through convolution kernel operations: ; in, The gradient components of the visible light image in the horizontal direction (x-axis) are calculated using a Sobel horizontal convolution kernel. The gradient components of the visible light image in the vertical direction (y-axis) are calculated using a Sobel vertical convolution kernel. The input visible light image data is represented as a two-dimensional matrix, where the matrix element values ​​are pixel brightness values ​​(range 0-255). This represents a two-dimensional discrete convolution operation.

[0040] Gradient magnitudes are synthesized using the Euclidean distance formula: ; in, Represents the visible light image in pixel coordinates The gradient magnitude at that location is used to quantify the intensity of the image edge at that position; The horizontal gradient component, calculated using a Sobel horizontal convolution kernel, reflects the gradient along the image direction. The rate of change of brightness along the horizontal axis; The vertical gradient component, calculated using the Sobel vertical convolution kernel, reflects the gradient along the image direction. The rate of change of brightness along the axis (vertical direction).

[0041] The Sobel operator preferably employs a split convolution optimization strategy, performing one-dimensional convolution in the horizontal direction first and then calculating in the vertical direction, which significantly reduces computational complexity.

[0042] Temperature field gradient magnitude The calculation method is as follows: ; in, The intensity of the temperature field gradient; Preliminary temperature field distribution; For the temperature field in Partial derivatives in the direction; For the temperature field in Partial derivatives in the direction.

[0043] Dynamic credibility weight function Its structure integrates visible light structural information with temperature field physical properties: ; in: Use the Sigmoid activation function; It is an exponential function; This represents the confidence weight of each pixel; For visible light images at position Gradient magnitude at; For the temperature field at position The gradient magnitude at that point.

[0044] The final temperature field is generated through spatial adaptive fusion: ; in, To smooth the temperature estimate for a local area; The final output temperature field; To make a preliminary estimate of the temperature field; This is a dynamic confidence weighting factor. The fusion strategy effectively suppresses abnormal fluctuations caused by measurement noise or model errors while preserving details in high-confidence regions.

[0045] Anomaly detection is based on the statistical characteristics of temperature gradient amplitude, when When it is determined to be an outlier, among which / pixel represents the temperature gradient threshold. Detected abnormal areas are highlighted with a flashing outline in the visualization interface. The outline width is preferably 2-4 pixels, and the flashing frequency is adapted to 2-5Hz to attract the operator's attention.

[0046] The visualization rendering pipeline employs pseudo-color mapping technology to map temperature values ​​to predefined color chromatograms (such as Jet chromatograms), and achieves real-time rendering through OpenGL shaders. The blinking effect of abnormal contours is achieved by alternately drawing framebuffer objects; odd-numbered frames display red contours (RGB(255,0,0)), and even-numbered frames display yellow contours (RGB(255,255,0)). Synchronously triggered audible and visual alarm signals are controlled via a GPIO interface. The preferred audible frequency is a 1-3kHz square wave, and the visual alarm uses synchronized blinking of high-brightness LEDs with a duty cycle set to 40%-60%.

[0047] The dynamic weighting mechanism perceives the complexity of scene structure through visible light gradients, assigning higher confidence to areas with rich textures (such as nameplates on device surfaces and heat sinks); at the same time, it detects physical anomalies through temperature gradients, reducing weights in areas with sudden changes in heat flow (such as hot spots and measurement blind spots), thereby achieving intelligent allocation of reconstruction accuracy in the spatial dimension.

[0048] The combined infrared and visible light image temperature measurement system described below can be referred to in correspondence with the combined infrared and visible light image temperature measurement method described above.

[0049] Please see the appendix Figure 2 The present invention also provides a composite temperature measurement system for infrared and visible light images, comprising: Multi-source synchronous acquisition module: integrates visible light sensor, infrared array and environmental detection unit; Physical constraint modeling module: Real-time calculation of heat conduction equations and convection constraint terms; Neural feature fusion module: realizes dual-stream coding and frequency domain fusion; Dynamic optimization solution module: Executes the block-alternating optimization algorithm; Temperature visualization module: Generates temperature distribution maps with confidence weights.

[0050] The system in this embodiment can be used to execute the above method embodiments, and its principle and technical effect are similar, so they will not be described again here.

[0051] Please see the appendix Figure 3 The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the above-described composite temperature measurement method of infrared image and visible light image.

[0052] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for combined infrared and visible light image temperature measurement, characterized in that, Includes the following steps: Visible light images, infrared radiation data, and environmental parameters are acquired through a synchronous acquisition unit. Based on the environmental parameters, a joint optimization objective is constructed that includes the heat conduction equation and convection constraints, wherein the parameters of the heat conduction equation are determined by the medium properties in the environmental parameters; The visible light image and infrared radiation data are input into a dual-stream encoder to extract visible light texture features and infrared radiation features, and cross-modal joint feature representation is generated by frequency domain coding fusion. Based on the joint optimization objective, the cross-modal joint feature representation is solved by block-alternating optimization to obtain the preliminary temperature field distribution; Dynamic confidence weights are calculated based on the gradient information of the visible light image and the gradient distribution of the preliminary temperature field, and the final temperature field is generated and the visualization results are output.

2. The composite temperature measurement method using infrared and visible light images according to claim 1, characterized in that, The steps of acquiring visible light images, infrared radiation data, and environmental parameters through the synchronous acquisition unit include: RGB image sequences were acquired using a visible light imaging module. ;in, Image height; Image width; For timestamps; Indicates the location place, time Visible light image intensity values; Infrared radiation flux was collected using a microbolometer array. ;in, Indicates the location place, time The infrared radiation flux value; Simultaneous detection of environmental parameters, including: Wind speed vector obtained using a three-dimensional ultrasonic anemometer ;in, For a moment The three-dimensional wind speed vector; for Directional wind speed component; for Directional wind speed component; for Directional wind speed component; Ambient temperature is obtained through a temperature sensor. ; Relative humidity is obtained through a humidity sensor. ; The timing synchronization controller ensures the time synchronization of multi-source data acquisition, with the synchronization error controlled within the range of 1-5ms, ensuring that all data correspond to the scene state at the same moment.

3. The composite temperature measurement method using infrared and visible light images according to claim 1, characterized in that, The steps for constructing a joint optimization objective based on environmental parameters include: Establish heat conduction constraint terms: ； in: This is the heat conduction loss term, used to constrain the physical consistency of the temperature field over time; For the Laplace operator of the temperature field; This is the partial derivative of temperature with respect to time; The density of the medium is obtained from environmental parameters; Specific heat capacity is obtained from environmental parameters; Thermal conductivity, obtained from environmental parameters; The square of the L2 norm; Establish convection constraint terms: ； in, This represents the heat convection loss term, limiting the impact of wind speed on temperature field migration; The measured three-dimensional wind speed vector; This is the first-order gradient vector of temperature, representing the direction and rate of temperature change; The scalar product of the convection terms; Combined to form a joint optimization objective: ； in: This is the total loss function; This represents the temperature prediction value generated from multimodal features; The temperature value observed by the infrared sensor; Weights for the heat conduction term; This represents the weight of the convection term.

4. The composite temperature measurement method using infrared and visible light images according to claim 1, characterized in that, The steps for the dual-stream encoder to extract visible light texture features and infrared radiation feature representations include: Visible light feature extraction: ； in, It is the visible light feature vector; For residual network models; The input is an RGB image; Output feature dimension; A deep residual network is used, with 18-34 layers and an output dimension d=256−512; Infrared feature extraction: ； in, It is the infrared radiation characteristic vector; It is a multilayer perceptron structure; This is an infrared radiation flux image; Output feature dimension; A 3-5 layer fully connected network is used, with 128-256 nodes in each layer, and the activation function is ReLU.

5. The composite temperature measurement method using infrared and visible light images according to claim 4, characterized in that, The step of generating cross-modal joint features through frequency domain coding fusion includes: Frequency domain coding of dual-stream features: ； in: This is the feature representation after frequency domain coding; This is the original feature vector; For frequency domain coding levels; This represents a vector concatenation operation; The index of the current level for frequency domain coding; For the first The frequency coefficient of the level; Pi is a constant. Generate cross-modal joint feature representations: ； in: The final joint feature output is used for temperature prediction; For the first The weighting coefficients of each fusion branch; For the first The weight matrix of each fusion layer; For the first Bias terms for each fusion layer; This represents the concatenation operation of two vectors; The number of fully connected fusion layers; It is a linear rectification activation function.

6. The composite temperature measurement method using infrared and visible light images according to claim 1, characterized in that, The steps of the block-based alternating optimization solution include: Represent the optimization variables as triples ;in, For cross-modal network parameter sets; This represents the current temperature field distribution tensor. For model residuals; fixed and Based on current observation loss For network parameters Perform gradient descent update: ； in, For neural network model parameters; This refers to the current iteration round; The learning rate; For observation loss; To calculate the gradient of the network parameters; With the temperature field variable fixed, the preconditioned conjugate gradient method is used to apply the method. Find the optimal solution: ； in, The updated temperature field; This is the preconditional conjugate gradient method; This is the sparse coefficient matrix constructed by linearizing the heat conduction equation; This is the right-hand vector formed by combining the heat source term with the initial estimate; fixed The residual term is dynamically updated using an explicit time integration method: ； in, For the first The residual tensor after the next iteration; This is the residual term, used to fit unmodeled physical errors; For the first The residual term tensor of the next iteration; For time step; The time derivative of the residual; This is the index for the iteration round.

7. The composite temperature measurement method using infrared and visible light images according to claim 1, characterized in that, The steps for calculating the dynamic credibility weight include: Extracting visible light images gradient magnitude plot The calculation method is as follows: ； in, For the image in Gradient strength at a point; Visible light image; , Visible light images in shaft and Gradient along the axial direction; Extracting the preliminary temperature field distribution gradient magnitude plot The calculation method is as follows: ； in, The intensity of the temperature field gradient; Preliminary temperature field distribution; For the temperature field in Partial derivatives in the direction; For the temperature field in Partial derivatives in the direction; Based on the two gradient magnitudes above, a confidence weight distribution is generated. : ； in: Use the Sigmoid activation function; It is an exponential function; This represents the confidence weight of each pixel; For visible light images at position Gradient magnitude at; For the temperature field at position The gradient magnitude at that point.

8. The composite temperature measurement method using infrared and visible light images according to claim 7, characterized in that, The steps for generating the final temperature field and outputting the visualization results include: Based on dynamic credibility weight For the initial temperature field After weighted adjustment, the final temperature field is obtained. : ； in, To smooth the temperature estimate for a local area; The final output temperature field; To make a preliminary estimate of the temperature field; As a dynamic credibility weighting factor; When satisfied When this occurs, an abnormal region marker is triggered, where: / pixel represents the temperature gradient threshold; The abnormal area will be highlighted on the temperature distribution map with a flashing outline, and an audible and visual alarm will be triggered simultaneously. The image outline flicker frequency is set to 2-5Hz; The sound and light device is activated at the same frequency, generating visual and auditory alarm signals.

9. A composite temperature measurement system for infrared and visible light images, using the composite temperature measurement method for infrared and visible light images as described in any one of claims 1-8, comprising: Multi-source synchronous acquisition module: integrates visible light sensor, infrared array and environmental detection unit; Physical constraint modeling module: Real-time calculation of heat conduction equations and convection constraint terms; Neural feature fusion module: realizes dual-stream coding and frequency domain fusion; Dynamic optimization solution module: Executes the block-alternating optimization algorithm; Temperature visualization module: Generates temperature distribution maps with confidence weights.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the composite temperature measurement method of infrared image and visible light image as described in any one of claims 1-8.

Images