Thermal infrared three-dimensional modeling method and system based on linear Gaussian primitive

Abstract

The invention discloses a thermal infrared three-dimensional modeling method and system based on linear Gaussian primitives, and the method comprises the steps: obtaining an infrared image of power transformation equipment, so as to construct a multi-view image set of the power transformation equipment; generating a sparse point cloud from the multi-view image set to construct an initial Gaussian primitive, and converting the initial Gaussian primitive into a linear Gaussian primitive through a linear attenuation function; constructing a thermal infrared physical model, and inputting the image set and the primitives into the physical model to obtain a thermal infrared synthetic image; constructing a composite loss function to calculate an error value of the infrared image and the composite image, calculating a gradient of the composite image, and carrying out back propagation iteration on the error value and the gradient to update parameters of the physical model; according to the method, the thermal infrared physical model is constructed, the atmospheric transmission effect and the thermal diffusion process are modeled, and the thermal loss is compensated by combining a residual addition mechanism, so that the problem of thermal radiation confusion caused by depth information attenuation is solved, and high-precision three-dimensional rendering under the dynamic illumination condition is realized.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction, and in particular to a thermal infrared three-dimensional modeling method and system based on linear Gaussian basis elements. Background Art

[0002] Porcelain insulation bushings are core insulation components in power systems, and their operational status is directly related to the safety and stability of the power grid. However, over long-term operation, factors such as contaminant deposition, internal electric field distortion, and material aging can gradually lead to hidden defects such as microcracks or localized overheating. These defects can rapidly worsen in harsh environments such as overvoltage and humidity, ultimately causing serious failures such as arcing or equipment failure.

[0003] Traditional inspection methods have significant shortcomings in detecting these early defects. For example, manual inspections are not only inefficient but also lack comprehensive coverage of hidden areas. Furthermore, visible light-based modeling methods struggle to accurately capture subtle surface temperature fluctuations caused by internal equipment faults (such as insulation degradation and poor contact) in the complex and changing environment of substations. Consequently, they are unable to provide drones with accurate information on the equipment's surface thermal status.

[0004] With the widespread adoption of drone technology in substation inspections, the need for refined perception of the surface condition of porcelain bushing equipment has become increasingly urgent. However, while existing drone-mounted infrared detection technology can capture thermal radiation information from the equipment's surface, it is limited by multiple factors, including infrared sensor pixel resolution, thermal conductivity, and the substation's dynamic lighting conditions. This results in infrared images with blurred edges, attenuated depth information, and confusion regarding the thermal radiation of adjacent targets, severely restricting the ability of drone inspections to provide early warning of porcelain bushing equipment failures. Furthermore, these technical bottlenecks make it difficult to meet the practical inspection requirements for quickly and accurately capturing infrared signatures on the surface of porcelain bushing substation equipment and constructing high-precision models. Summary of the Invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a thermal infrared three-dimensional modeling method and system based on linear Gaussian primitives to solve the problems in the prior art such as image edge blurring caused by pixels and heat conduction, confusion and blurring of thermal radiation due to depth information attenuation, and poor adaptability to dynamic lighting in substations.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a thermal infrared three-dimensional modeling method based on linear Gaussian primitives, comprising:

[0008] S1: Acquire infrared images of substation equipment to construct a multi-view image set of substation equipment;

[0009] S2: Based on the structure-from-motion algorithm, a sparse point cloud is generated from the multi-view image set to construct the initial Gaussian primitives, which are then converted into linear Gaussian primitives using a linear decay function.

[0010] S3: constructing a thermal infrared physical model, and inputting the multi-view image set and the linear Gaussian basis element into the physical model to obtain a thermal infrared composite image;

[0011] S4: Construct a composite loss function to calculate the error value between the infrared image and the thermal infrared composite image, and calculate the gradient of the thermal infrared composite image through a dynamic adaptive gradient scaling strategy, and back-propagate the error value and gradient to iteratively update the parameters of the thermal infrared physical model to output an optimized thermal infrared physical model.

[0012] In an optional embodiment, the S1 includes the following specific steps:

[0013] Collecting images of the surrounding environment of the porcelain bushing substation equipment using a drone, and analyzing the images to obtain analysis results; the analysis results include the drone scanning range, the drone scanning path, and the drone's camera pose;

[0014] Based on the analysis results, drone inspection routes are planned to carry out inspections at different times of the day. The drone's camera is used to capture multi-angle images of the porcelain-sleeved substation equipment, acquiring thermal infrared image data of the equipment under different weather conditions.

[0015] Based on the infrared image data of porcelain bushing substation equipment, a multi-view image set of substation equipment is constructed.

[0016] In an optional embodiment, the S2 includes the following specific steps:

[0017] S201: Based on the multi-view image set of the substation equipment, a point cloud of the substation equipment is generated by using a structure-from-motion algorithm;

[0018] S202: constructing and initializing an initial Gaussian primitive based on each point in the point cloud;

[0019] S203: constructing a Mahalanobis distance function, introducing an alignment factor into the Mahalanobis distance function to adjust the distribution range of the Mahalanobis distance function, and obtaining an adjusted Mahalanobis distance function;

[0020] S204: reconstructing the Gaussian kernel of the initial Gaussian basis element based on the modified Mahalanobis distance, and replacing the kernel function of the reconstructed initial Gaussian basis element with a linear attenuation function to convert the initial Gaussian basis element into a linear Gaussian basis element;

[0021] In an optional implementation, the adjusted Mahalanobis distance function in S203 is expressed as follows:

[0022]

[0023]

[0024] Where, is the Mahalanobis distance function of the initial Gaussian basis; X is the coordinate of any point in three-dimensional space; i is the center of the initial Gaussian basis; Σ is the covariance matrix, which is used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is the diagonal matrix; T is the transpose sign;

[0025] The expression of the linear attenuation function in S204 is:

[0026]

[0027] Where, is a linear decay function; is the alignment factor;

[0028] The function expression of the linear Gaussian basis element in S204 is:

[0029]

[0030] Where, is a linear Gaussian basis.

[0031] In an optional embodiment, S3 includes the following specific steps:

[0032] S301: Construct a thermal infrared physics model to simulate the atmospheric transmission effect of thermal radiation attenuation and the thermal conduction effect of heat diffusion;

[0033] The thermal infrared physics model includes an atmospheric transmission effect module based on a multi-layer perceptron network and a heat conduction module based on a convolutional neural network;

[0034] S302: Position-encode the spatial coordinates of the linear Gaussian basis element to obtain a position feature vector, and position-encode the shooting time stamp of the infrared image in the multi-view image set to obtain a time feature vector;

[0035] S303: Inputting the position feature vector and the time feature vector into the atmospheric transmission module, and predicting the attenuation coefficient of the linear Gaussian basis element through the multi-layer perceptron network to update the spherical harmonic coefficient of the linear Gaussian basis element, thereby outputting the linear Gaussian basis element after thermal radiation attenuation;

[0036] S304: Projecting the linear Gaussian primitives after thermal radiation attenuation onto a two-dimensional plane to form a two-dimensional Gaussian, and rendering the two-dimensional Gaussian set through a differentiable rasterizer to output an initial rendered image.

[0037] S305: Extract the second-order gradient feature image of the initial rendered image based on the heat conduction module, perform feature splicing on the initial rendered image and the second-order gradient image, and input the formed multi-channel feature image into the convolutional neural network to output a thermal infrared composite image corrected by the heat conduction effect after residual addition fusion.

[0038] In an optional embodiment, the attenuation coefficient of the linear Gaussian basis element in S303 is expressed as follows:

[0039]

[0040] Where, is the atmospheric absorption coefficient; is the atmospheric scattering coefficient; d is the propagation distance of thermal radiation in the atmosphere; It is the atmospheric transmission effect module; is the position feature vector; is the time feature vector;

[0041] The function expression of the spherical harmonic coefficient of the linear Gaussian basis element in S303 is:

[0042]

[0043] Where, is the attenuated spherical harmonic coefficient; are the initial spherical harmonic coefficients;

[0044] The function expression of the thermal infrared composite image in S305 is:

[0045]

[0046] Where, It is a thermal infrared composite image; is the initial rendered image; ConvBlock is the convolution layer; concat is the concatenation function; is the second-order gradient feature image.

[0047] In an optional embodiment, the step S304 specifically includes the following steps:

[0048] Performing view transformation through sputtering technology and calculating a new covariance matrix to project the linear Gaussian basis element after thermal radiation attenuation onto a two-dimensional image plane to obtain a two-dimensional Gaussian basis element;

[0049] Build a differentiable rasterizer to sort 2D Gaussian primitives;

[0050] Calculate the color and opacity of all two-dimensional Gaussian primitives within the pixel using the Gaussian rendering formula to obtain the final color of the pixel, and output a rendered two-dimensional image;

[0051] The new covariance matrix has the following functional expression:

[0052]

[0053] Where, is the new covariance matrix; Σ is the covariance matrix; J is the Jacobi matrix of the projection transformation; W is the perspective transformation matrix;

[0054] The Gaussian rendering formula is expressed as follows:

[0055]

[0056] Where C is the final pixel color; is the color of the j-th two-dimensional Gaussian basis element; is the transparency of the j-th two-dimensional Gaussian basis element; N is the number of two-dimensional Gaussian basis elements that affect the color of the pixel. The multiplicative transparency factor represents the cumulative transparency product from the 1st to the j-1th 2D Gaussian primitives, and is used to adjust the contribution of each 2D Gaussian primitive to the final pixel color.

[0057] In an optional embodiment, the S4 includes the following specific steps:

[0058] S401: Establish a heat conduction regularization loss function to constrain the heat conduction module parameters in the thermal infrared physics model and construct a composite loss function:

[0059] The composite loss function includes a heat conduction regularization term loss function, an absolute error loss function, and a similarity loss function;

[0060] S402: Calculating error values ​​between the infrared image and the thermal infrared composite image based on the composite loss function;

[0061] S403: Calculating the Mahalanobis distance between each pixel in the thermal infrared synthetic image and the linear Gaussian basis based on a dynamic gradient adjustment strategy, and constructing a scaling factor using the Mahalanobis distance to adjust the gradient of the thermal infrared synthetic image;

[0062] S404: back-propagating the error value and the adjusted thermal infrared composite image gradient to iteratively update the parameters of the thermal infrared physical model;

[0063] S405: Determine whether the number of iterations reaches a preset iteration threshold. If so, obtain an optimized thermal infrared physical model and output an optimal thermal infrared composite image. If not, return to S402.

[0064] In an optional embodiment, the expression of the heat conduction regularization term loss function in S401 is:

[0065]

[0066] Where, is the heat conduction regularization loss function; Pixel The temperature value at is the neighborhood pixel set Medium pixel The temperature value at

[0067] The expression of the composite loss function in S401 is:

[0068]

[0069] Where, is the composite loss function; is the weight coefficient of the heat conduction regularization loss function; is the similarity loss function; is the weight coefficient of the similarity loss function; is the absolute error loss function;

[0070] The functional expression of the Mahalanobis distance in S403 is:

[0071]

[0072] Where, is the Mahalanobis distance; is the coordinate of any point in the thermal infrared composite image; is the center of the linear Gaussian basis element;

[0073] The function expression of the scaling factor in S403 is:

[0074]

[0075] Where, is the scaling factor;

[0076] The function expression of the thermal infrared composite image gradient adjusted in S404 is:

[0077]

[0078] Where, is the adjusted thermal infrared composite image gradient; is the initial thermal infrared synthetic image gradient.

[0079] In a second aspect, the present invention provides a thermal infrared three-dimensional modeling system based on linear Gaussian primitives, comprising:

[0080] An image acquisition module is used to obtain infrared images of the substation equipment to construct a multi-view image set of the substation equipment;

[0081] Gaussian generation module, which is used to generate a sparse point cloud from a multi-view image set based on the motion recovery structure algorithm to construct an initial Gaussian basis element, and convert the initial Gaussian basis element into a linear Gaussian basis element through a linear attenuation function;

[0082] a model construction module for constructing a thermal infrared physical model, inputting the multi-view image set and the linear Gaussian basis element into the physical model to obtain a thermal infrared composite image and a physically corrected thermal infrared linear Gaussian basis element;

[0083] The model optimization module is used to construct a composite loss function to calculate the error value of the infrared image and the thermal infrared composite image, and calculate the gradient of the thermal infrared composite image through a dynamic adaptive gradient scaling strategy. The error value and gradient are back-propagated to iteratively update the parameters of the thermal infrared physical model to obtain an optimized thermal infrared physical model.

[0084] The beneficial effects brought about by the embodiments provided by the present invention include:

[0085] The present invention replaces the exponential decay function in the initial Gaussian basis with a linear decay function, reuses the original framework parameters to maintain the stability of the mathematical structure, and significantly reduces high-frequency artifacts in 3D Gaussian rendering. At the same time, an alignment factor is introduced to dynamically adjust the Mahalanobis distance distribution range, enhancing the ability to characterize large-scale structures. The sharp characteristics of the linear function are used to highlight edge information, effectively resisting the blurring effect caused by heat conduction.

[0086] The present invention constructs a thermal infrared physics model, and models the atmospheric transmission effect and heat diffusion process through an atmospheric transmission effect module constructed based on a multi-layer perceptron network and a heat conduction module based on a convolutional neural network. It combines the residual addition mechanism to compensate for heat loss, solves the thermal radiation confusion problem caused by depth information attenuation, and achieves high-precision three-dimensional rendering under dynamic lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0088] Figure 1 A flow chart of a thermal infrared three-dimensional modeling method based on linear Gaussian basis elements in an embodiment of this specification is shown;

[0089] Figure 2 A flow chart of a thermal infrared three-dimensional modeling system based on linear Gaussian basis elements in an embodiment of this specification is shown. DETAILED DESCRIPTION

[0090] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed in order to provide a comprehensive understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present invention by illustrating examples of the present invention.

[0091] Example 1

[0092] like Figure 1 As shown, this embodiment provides a thermal infrared 3D modeling method based on linear Gaussian primitives, including:

[0093] S1: Acquire infrared images of substation equipment to construct a multi-view image set of substation equipment;

[0094] Exemplarily, S1 includes the following specific steps:

[0095] S101: collecting an image of the surrounding environment of the porcelain bushing substation equipment using a drone, and analyzing the image to obtain an analysis result;

[0096] The analysis results include the drone scanning range, the drone scanning path and the drone’s camera pose;

[0097] S102: Planning the drone inspection task route based on the analysis results to perform the inspection task at different times throughout the day, and using the camera carried by the drone to shoot the porcelain-sleeved substation equipment from multiple angles to obtain thermal infrared image data of the porcelain-sleeved substation equipment under different weather conditions at different times;

[0098] In this embodiment, the analysis results of the surrounding environment of the porcelain-insulated substation equipment are used to plan the route of the drone inspection mission, ensuring the safe operation of the unmanned inspection equipment in complex environments and avoiding equipment damage caused by environmental factors. At the same time, the inspection tasks are scheduled at different times of the day to cover the twelve-hour time point, ensuring comprehensive data acquisition under different lighting and weather conditions.

[0099] S103: Construct a multi-view image set of the substation equipment based on the infrared image data of the porcelain bushing substation equipment.

[0100] S2: Based on the structure-from-motion algorithm, a sparse point cloud is generated from the multi-view image set to construct the initial Gaussian primitives, which are then converted into linear Gaussian primitives using a linear decay function.

[0101] Exemplarily, S2 includes the following specific steps:

[0102] S201: Based on the multi-view image set of the substation equipment, a point cloud of the substation equipment is generated by using a structure-from-motion algorithm;

[0103] S202: constructing and initializing an initial Gaussian primitive based on each point in the point cloud;

[0104] S203: constructing a Mahalanobis distance function, introducing an alignment factor into the Mahalanobis distance function to adjust the distribution range of the Mahalanobis distance function, and obtaining an adjusted Mahalanobis distance function;

[0105] The expression of the adjusted Mahalanobis distance function in S203 is:

[0106]

[0107]

[0108] Where, is the Mahalanobis distance function of the initial Gaussian basis; X is the coordinate of any point in three-dimensional space; i is the center of the initial Gaussian basis; Σ is the covariance matrix, which is used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is the diagonal matrix; T is the transpose sign;

[0109] S204: reconstructing the Gaussian kernel of the initial Gaussian basis element based on the modified Mahalanobis distance, and replacing the kernel function of the reconstructed initial Gaussian basis element with a linear attenuation function to convert the initial Gaussian basis element into a linear Gaussian basis element;

[0110] Among them, the expression of the linear attenuation function in S204 is:

[0111]

[0112] Where, is a linear decay function; is the alignment factor;

[0113] Among them, the function expression of the linear Gaussian basis element in S204 is:

[0114]

[0115] Where, is a linear Gaussian basis.

[0116] In this embodiment, an alignment factor is introduced to adjust the scaling of the Mahalanobis distance to ensure that it can accurately represent large structures and retain small but important details, so as to avoid the loss of details in the infrared high-frequency area, so that the linear Gaussian primitives can accurately render sharp edges and complex textures, and the coverage range can also be reasonable; at the same time, the exponential decay function in the initial Gaussian primitive is replaced by a linear decay function, and the original framework parameters are reused to maintain the stability of the mathematical structure, which significantly reduces the high-frequency artifacts in the three-dimensional Gaussian rendering and improves the quality of infrared modeling.

[0117] S3: constructing a thermal infrared physical model, and inputting the multi-view image set and the linear Gaussian basis element into the physical model to obtain a thermal infrared composite image;

[0118] Exemplarily, S3 includes the following specific steps:

[0119] S301: Construct a thermal infrared physics model to simulate the atmospheric transmission effect of thermal radiation attenuation and the thermal conduction effect of heat diffusion;

[0120] The thermal infrared physics model includes an atmospheric transmission effect module based on a multi-layer perceptron network and a heat conduction module based on a convolutional neural network.

[0121] In this embodiment, the atmospheric transmission effect module is used to simulate the atmospheric transmission effect of thermal radiation attenuation. When thermal radiation propagates in the atmosphere, it is attenuated due to absorption by greenhouse gases such as water vapor and carbon dioxide, and scattering by nitrogen and oxygen molecules and cloud particles. The specific phenomenon can be described by the Bouguer-Lambert-Beer law:

[0122]

[0123]

[0124] In this formula, I is the true radiation intensity after attenuation; is the initial radiation intensity when the propagation distance is 0; is the medium attenuation coefficient; is the atmospheric absorption coefficient; is the atmospheric scattering coefficient; d is the propagation distance of thermal radiation in the atmosphere;

[0125] S302: Position-encode the spatial coordinates of the linear Gaussian basis element to obtain a position feature vector, and position-encode the shooting time stamp of the infrared image in the multi-view image set to obtain a time feature vector;

[0126] Among them, the function expression of position encoding is:

[0127]

[0128] Where, Encode for position; is the input scalar, which is the spatial coordinate of the linear Gaussian basis element or the shooting time stamp of the infrared image: L is the number of dimensions of the position encoding; k is the frequency level index.

[0129] S303: Inputting the position feature vector and the time feature vector into the atmospheric transmission module, and predicting the attenuation coefficient of the linear Gaussian basis element through the multi-layer perceptron network to update the spherical harmonic coefficient of the linear Gaussian basis element, thereby outputting the linear Gaussian basis element after thermal radiation attenuation;

[0130] Specifically, the depth D of the multilayer perceptron network is set to 8, the hidden layer dimension W is set to 256, and the initial parameter atmospheric absorption coefficient is set to and atmospheric scattering coefficient Set to 0, and the propagation distance d to 1, so as to use the multilayer perceptron network to decouple the attenuation and geometric effects, take the position-encoded linear Gaussian unit position γ(x) and the shooting time γ(t) as input to determine the attenuation coefficient of the linear Gaussian unit at that moment, apply the obtained coefficient to the linear Gaussian unit, update the spherical harmonic coefficient of the linear Gaussian unit, and output the linear Gaussian unit after thermal radiation attenuation;

[0131] Among them, the attenuation coefficient of the linear Gaussian basis element in S303 is expressed as follows:

[0132]

[0133] In the formula It is the atmospheric transmission effect module; is the position feature vector; is the time feature vector;

[0134] Among them, the function expression of the spherical harmonic coefficient of the linear Gaussian basis element in S303 is:

[0135]

[0136] Where, is the attenuated spherical harmonic coefficient; are the initial spherical harmonic coefficients;

[0137] S304: Projecting the linear Gaussian primitives after thermal radiation attenuation onto a two-dimensional plane to form a two-dimensional Gaussian, and rendering the two-dimensional Gaussian set through a differentiable rasterizer to output an initial rendered image.

[0138] Specifically, S304 includes the following steps:

[0139] Performing view transformation through sputtering technology and calculating a new covariance matrix to project the linear Gaussian basis element after thermal radiation attenuation onto a two-dimensional image plane to obtain a two-dimensional Gaussian basis element;

[0140] Among them, the new covariance matrix has the following function expression:

[0141]

[0142] Where, is the new covariance matrix; Σ is the covariance matrix; J is the Jacobi matrix of the projection transformation; W is the perspective transformation matrix;

[0143] Build a differentiable rasterizer to sort 2D Gaussian primitives;

[0144] Calculate the color and opacity of all two-dimensional Gaussian primitives within the pixel using the Gaussian rendering formula to obtain the final color of the pixel, and output a rendered two-dimensional image;

[0145] Among them, the expression of Gaussian rendering formula is:

[0146]

[0147] Where C is the final pixel color; is the color of the j-th two-dimensional Gaussian basis element, calculated from the spherical harmonic coefficients SH; is the transparency of the j-th two-dimensional Gaussian basis element; N is the number of two-dimensional Gaussian basis elements that affect the color of the pixel. The multiplicative transparency factor represents the cumulative transparency product from the 1st to the j-1th two-dimensional Gaussian basis element, which is used to adjust the contribution of each two-dimensional Gaussian basis element to the final pixel color;

[0148] S305: Extract the second-order gradient feature image of the initial rendered image based on the heat conduction module, perform feature splicing on the initial rendered image and the second-order gradient image, and input the formed multi-channel feature image into the convolutional neural network to output a thermal infrared composite image corrected by the heat conduction effect after residual addition fusion.

[0149] Among them, the function expression of the second-order gradient feature image in S305 is:

[0150]

[0151]

[0152] Where, is the second-order gradient feature image; is the two-dimensional Laplace operator; is the rate of change of the two-dimensional plane temperature field u with time t; is the thermal diffusion capacity of the material; k is the thermal conductivity; c is the specific heat capacity; s is the density;

[0153] The function expression of the thermal infrared composite image in S305 is:

[0154]

[0155] Where, It is a thermal infrared composite image; is the initial rendered image; ConvBlock is the convolution layer; concat is the concatenation function.

[0156] Specifically, the initial rendered image is a thermal infrared image. Since the thermal infrared image represents a two-dimensional temperature field and is applicable to the two-dimensional temperature field heat conduction equation, this embodiment extracts the second-order gradient feature image of the initial rendered image through the two-dimensional temperature field heat conduction equation. In this embodiment, the heat conduction module extracts the second-order gradient feature image of the initial rendered image, and the convolution layer in the convolutional neural network is set to the convolution layer depth D = 3, the input feature dimension Wi = [2n, n, n], and the output feature dimension Wo = [n, n, n], where n represents the input image feature dimension, so as to fuse the initial rendered image with the second-order gradient feature image to simulate different pixel positions. value, and adopt the residual addition mechanism in this process to effectively solve the heat loss problem caused by heat conduction.

[0157] S4: Construct a composite loss function to calculate the error value between the infrared image and the thermal infrared composite image, and calculate the gradient of the thermal infrared composite image through a dynamic adaptive gradient scaling strategy, and back-propagate the error value and gradient to iteratively update the parameters of the thermal infrared physical model to obtain an optimized thermal infrared physical model.

[0158] Exemplarily, S4 includes the following specific steps:

[0159] S401: Establish a heat conduction regularization loss function to constrain the heat conduction module parameters in the thermal infrared physics model and construct a composite loss function:

[0160] In some embodiments, the composite loss function includes a heat conduction regularization term loss function, an absolute error loss function, and a similarity loss function;

[0161] Among them, the expression of the heat conduction regularization loss function in S401 is:

[0162]

[0163] Where, is the heat conduction regularization loss function; Pixel The temperature value at is the neighborhood pixel set Medium pixel The temperature value at

[0164] Among them, the expression of the composite loss function in S401 is:

[0165]

[0166] Where, is the composite loss function; is the weight coefficient of the heat conduction regularization loss function; is the similarity loss function; is the weight coefficient of the similarity loss function; is the absolute error loss function;

[0167] In this embodiment, a heat conduction regularization loss function is introduced to constrain the model learning process so that the generated image features conform to the physical laws of heat conduction (i.e., the temperature field changes smoothly in space). The heat conduction regularization loss function traverses each pixel in the image, calculates the grayscale gradient difference between it and the neighboring pixels, and constructs a regularization term in the form of a weighted sum of squares to effectively suppress the pseudo edges caused by thermal noise and improve the smoothness of the temperature distribution of the generated image, so as to better fit the actual heat conduction situation.

[0168] S402: Calculating error values ​​between the infrared image and the thermal infrared composite image based on the composite loss function;

[0169] S403: Calculating the Mahalanobis distance between each pixel in the thermal infrared synthetic image and the linear Gaussian basis based on a dynamic gradient adjustment strategy, and constructing a scaling factor using the Mahalanobis distance to adjust the gradient of the thermal infrared synthetic image;

[0170] The functional expression of the Mahalanobis distance in S403 is:

[0171]

[0172] Where, is the Mahalanobis distance; is the coordinate of any point in the thermal infrared composite image; is the center of the linear Gaussian basis element;

[0173] The function expression of the scaling factor in S403 is:

[0174]

[0175] Where, is the scaling factor;

[0176] S404: back-propagating the error value and the adjusted thermal infrared composite image gradient to iteratively update the parameters of the thermal infrared physical model;

[0177] The function expression of the thermal infrared composite image gradient adjusted in S404 is:

[0178]

[0179] Where, is the adjusted thermal infrared composite image gradient; is the initial thermal infrared composite image gradient;

[0180] S405: Determine whether the number of iterations reaches a preset iteration threshold. If so, obtain an optimized thermal infrared physical model and output an optimal thermal infrared composite image. If not, return to S402.

[0181] In this embodiment, when the thermal infrared physical model is optimized, the optimal linear Gaussian basis element and the optimal thermal infrared synthetic image will be obtained. At this time, the optimal linear Gaussian basis element is the constituent unit of the thermal infrared three-dimensional model of the substation equipment, and the optimal thermal infrared synthetic image is the rendering result of the thermal infrared three-dimensional model of the substation equipment.

[0182] Example 2

[0183] like Figure 2 As shown, this embodiment provides a thermal infrared 3D modeling system 100 based on linear Gaussian primitives, including:

[0184] An image acquisition module 101 is used to acquire infrared images of the substation equipment to construct a multi-view image set of the substation equipment;

[0185] A Gaussian generation module 102 is configured to generate a sparse point cloud from a multi-view image set based on a structure-from-motion algorithm to construct an initial Gaussian basis element, and convert the initial Gaussian basis element into a linear Gaussian basis element through a linear attenuation function;

[0186] A model building module 103 is used to build a thermal infrared physical model, input the multi-view image set and the linear Gaussian basis element into the physical model to obtain a thermal infrared composite image and a physically corrected thermal infrared linear Gaussian basis element;

[0187] The model optimization module 104 is used to construct a composite loss function to calculate the error value between the infrared image and the thermal infrared composite image, and calculate the gradient of the thermal infrared composite image through a dynamic adaptive gradient scaling strategy, and back-propagate the error value and gradient to iteratively update the parameters of the thermal infrared physical model to obtain an optimized thermal infrared physical model.

[0188] The foregoing description is merely a preferred embodiment of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive. The scope of the present invention is defined by the appended claims, not the foregoing description, and all variations that come within the meaning and range of equivalents of the claims are intended to be included within the present invention.

Claims

1. A thermal infrared three-dimensional modeling method based on linear Gaussian primitives, characterized in that: include: S1: Acquire infrared images of substation equipment to construct a multi-view image set of substation equipment; S2: Based on the structure-from-motion algorithm, a sparse point cloud is generated from the multi-view image set to construct the initial Gaussian primitives, which are then converted into linear Gaussian primitives using a linear decay function. S3: constructing a thermal infrared physical model, and inputting the multi-view image set and the linear Gaussian basis element into the physical model to obtain a thermal infrared composite image; S4: Construct a composite loss function to calculate the error value between the infrared image and the thermal infrared composite image, and calculate the gradient of the thermal infrared composite image through a dynamic adaptive gradient scaling strategy, and back-propagate the error value and gradient to iteratively update the parameters of the thermal infrared physical model to obtain an optimized thermal infrared physical model.

2. The method according to claim 1, characterized in that The S1 includes the following specific steps: Collecting images of the surrounding environment of the porcelain bushing substation equipment based on a drone, and analyzing the images to obtain analysis results; The analysis results include the drone scanning range, the drone scanning path and the drone's camera pose; Based on the analysis results, drone inspection routes are planned to carry out inspections at different times of the day. The drone's camera is used to capture multi-angle images of the porcelain-sleeved substation equipment, acquiring thermal infrared image data of the equipment under different weather conditions. Based on the infrared image data of porcelain bushing substation equipment, a multi-view image set of substation equipment is constructed.

3. The method according to claim 1, characterized in that The S2 includes the following specific steps: S201: Based on the multi-view image set of the substation equipment, a point cloud of the substation equipment is generated by using a structure-from-motion algorithm; S202: constructing and initializing an initial Gaussian primitive based on each point in the point cloud; S203: constructing a Mahalanobis distance function, introducing an alignment factor into the Mahalanobis distance function to adjust the distribution range of the Mahalanobis distance function, and obtaining an adjusted Mahalanobis distance function; S204: reconstructing the Gaussian kernel of the initial Gaussian basis element based on the modified Mahalanobis distance, and replacing the kernel function of the reconstructed initial Gaussian basis element with a linear attenuation function to convert the initial Gaussian basis element into a linear Gaussian basis element.

4. The method according to claim 2, characterized in that The Mahalanobis distance function adjusted in S203 is expressed as follows: ； ； Where, is the Mahalanobis distance function of the initial Gaussian basis; X is the coordinate of any point in three-dimensional space; i is the center of the initial Gaussian basis; Σ is the covariance matrix, which is used to control the size, shape and direction of the three-dimensional Gaussian; R is the rotation matrix; S is the diagonal matrix; T is the transpose sign; The expression of the linear attenuation function in S204 is: ； Where, is a linear decay function; is the alignment factor; The function expression of the linear Gaussian basis element in S204 is: ； Where, is a linear Gaussian basis.

5. The method according to claim 1, wherein The S3 includes the following specific steps: S301: Construct a thermal infrared physics model to simulate the atmospheric transmission effect of thermal radiation attenuation and the thermal conduction effect of heat diffusion; The thermal infrared physics model includes an atmospheric transmission effect module based on a multi-layer perceptron network and a heat conduction module based on a convolutional neural network; S302: Position-encode the spatial coordinates of the linear Gaussian basis element to obtain a position feature vector, and position-encode the shooting time stamp of the infrared image in the multi-view image set to obtain a time feature vector; S303: Inputting the position feature vector and the time feature vector into the atmospheric transmission module, and predicting the attenuation coefficient of the linear Gaussian basis element through the multi-layer perceptron network to update the spherical harmonic coefficient of the linear Gaussian basis element, thereby outputting the linear Gaussian basis element after thermal radiation attenuation; S304: Projecting the linear Gaussian primitives after thermal radiation attenuation onto a two-dimensional plane to form a two-dimensional Gaussian, and rendering the two-dimensional Gaussian set through a differentiable rasterizer to output an initial rendered image; S305: Extract the second-order gradient feature image of the initial rendered image based on the heat conduction module, perform feature splicing on the initial rendered image and the second-order gradient image, and input the formed multi-channel feature image into the convolutional neural network to output a thermal infrared composite image corrected by the heat conduction effect after residual addition fusion.

6. The method according to claim 5, characterized in that The attenuation coefficient of the linear Gaussian basis element in S303 is expressed as follows: ； Where, is the atmospheric absorption coefficient; is the atmospheric scattering coefficient; d is the propagation distance of thermal radiation in the atmosphere; It is the atmospheric transmission effect module; is the position feature vector; is the time feature vector; The function expression of the spherical harmonic coefficient of the linear Gaussian basis element in S303 is: ； Where, is the attenuated spherical harmonic coefficient; are the initial spherical harmonic coefficients; The function expression of the thermal infrared composite image in S305 is: ； Where, It is a thermal infrared composite image; is the initial rendered image; ConvBlock is the convolution layer; concat is the concatenation function; is the second-order gradient feature image.

7. The method according to claim 5, characterized in that The S304 specifically includes the following steps: Performing view transformation through sputtering technology and calculating a new covariance matrix to project the linear Gaussian basis element after thermal radiation attenuation onto a two-dimensional image plane to obtain a two-dimensional Gaussian basis element; Build a differentiable rasterizer to sort 2D Gaussian primitives; Calculate the color and opacity of all two-dimensional Gaussian primitives within the pixel using the Gaussian rendering formula to obtain the final color of the pixel, and output a rendered two-dimensional image; The new covariance matrix has the following functional expression: ； Where, is the new covariance matrix; Σ is the covariance matrix; J is the Jacobi matrix of the projection transformation; W is the perspective transformation matrix; The Gaussian rendering formula is expressed as follows: ； Where C is the final pixel color; is the color of the j-th two-dimensional Gaussian basis element; is the transparency of the j-th two-dimensional Gaussian basis element; N is the number of two-dimensional Gaussian basis elements that affect the color of the pixel. The multiplicative transparency factor represents the cumulative transparency product from the 1st to the j-1th 2D Gaussian primitives, and is used to adjust the contribution of each 2D Gaussian primitive to the final pixel color.

8. The method according to claim 1, characterized in that The S4 includes the following specific steps: S401: Establish a heat conduction regularization loss function to constrain the heat conduction module parameters in the thermal infrared physics model and construct a composite loss function: The composite loss function includes a heat conduction regularization term loss function, an absolute error loss function, and a similarity loss function; S402: Calculating error values ​​between the infrared image and the thermal infrared composite image based on the composite loss function; S403: Calculating the Mahalanobis distance between each pixel in the thermal infrared synthetic image and the linear Gaussian basis based on a dynamic gradient adjustment strategy, and constructing a scaling factor using the Mahalanobis distance to adjust the gradient of the thermal infrared synthetic image; S404: back-propagating the error value and the adjusted thermal infrared composite image gradient to iteratively update the parameters of the thermal infrared physical model; S405: Determine whether the number of iterations reaches a preset iteration threshold. If so, obtain the optimized thermal infrared physical model and output the optimal thermal infrared composite image; if not, return to S402.

9. The method according to claim 8, characterized in that The expression of the heat conduction regularization term loss function in S401 is: ； Where, is the heat conduction regularization loss function; Pixel The temperature value at is the neighborhood pixel set Medium pixel The temperature value at The expression of the composite loss function in S401 is: ； Where, is the composite loss function; is the weight coefficient of the heat conduction regularization loss function; is the similarity loss function; is the weight coefficient of the similarity loss function; is the absolute error loss function; The functional expression of the Mahalanobis distance in S403 is: ； Where, is the Mahalanobis distance; is the coordinate of any point in the thermal infrared composite image; is the center of the linear Gaussian basis element; The function expression of the scaling factor in S403 is: ； Where, is the scaling factor; The function expression of the thermal infrared composite image gradient adjusted in S404 is: ； Where, is the adjusted thermal infrared composite image gradient; is the initial thermal infrared synthetic image gradient.

10. A thermal infrared three-dimensional modeling system based on linear Gaussian primitives, characterized in that: include: An image acquisition module is used to obtain infrared images of the substation equipment to construct a multi-view image set of the substation equipment; Gaussian generation module, which is used to generate a sparse point cloud from a multi-view image set based on the motion recovery structure algorithm to construct an initial Gaussian basis element, and convert the initial Gaussian basis element into a linear Gaussian basis element through a linear attenuation function; a model construction module for constructing a thermal infrared physical model, inputting the multi-view image set and the linear Gaussian basis element into the physical model to obtain a thermal infrared composite image and a physically corrected thermal infrared linear Gaussian basis element; The model optimization module is used to construct a composite loss function to calculate the error value of the infrared image and the thermal infrared composite image, and calculate the gradient of the thermal infrared composite image through a dynamic adaptive gradient scaling strategy. The error value and gradient are back-propagated to iteratively update the parameters of the thermal infrared physical model to obtain an optimized thermal infrared physical model.

Images