Temperature field reconstruction method based on multi-source information fusion

Abstract

The invention discloses a temperature field reconstruction method based on multi-source information fusion. The method comprises the following steps: acquiring multi-point temperature data measured by a plurality of temperature sensors deployed on an aircraft of which a temperature field is to be reconstructed; inputting the acquired multi-point temperature data into a pre-constructed and trained deep neural network model to obtain a global temperature field output by the deep neural network model, training the deep neural network model by using a pre-acquired training data set, the training data set comprising a plurality of training data, the training data comprises multi-point temperature data measured by a plurality of temperature sensors deployed on the aircraft under the same working condition, an infrared camera temperature field obtained through an aircraft infrared image shot by an infrared camera, and a ground experiment temperature field corresponding to the aircraft obtained through a ground experiment platform. The method can reduce the error between the predicted temperature field and the real temperature field, and improves the accuracy and credibility of the reconstruction result of the aircraft temperature field.

Description

Technical Field

[0001] The present invention relates to the technical field of temperature field reconstruction, and in particular to a temperature field reconstruction method based on multi-source information fusion. Background Art

[0002] With the development of high-precision and complex electronic equipment such as aircraft, miniaturization has become one of the goals of electronic equipment design. However, the narrow space and limited heat dissipation brought about by miniaturization design may cause local overheating of the equipment, affecting the safety and performance of the system. Thermal management is an important means to ensure the reliability of the operation of electronic components and systems. As a key method of thermal management, temperature field reconstruction can effectively realize the monitoring of aircraft operation status and accurate and rapid diagnosis and prediction of faults, timely detect and handle temperature anomalies, and ensure the safe operation of the system. The existing temperature field reconstruction methods include interpolation method, physical simulation method and deep neural network learning method. Among them, interpolation methods including bilinear interpolation method, co-kriging and nonlinear interpolation method usually have large reconstruction errors; although the physical simulation method can give high-precision numerical calculation results, it has the problem of long time consumption and large computing resources; deep neural network can solve nonlinear and high-dimensional problems. By using a large amount of temperature field data as training data for deep neural network model training, it can learn the temperature distribution law of components under different heating powers, build a temperature field proxy model, and achieve good prediction and reconstruction effects in temperature field reconstruction tasks.

[0003] At present, for the task of temperature field reconstruction, the training data of deep neural network models usually uses a temperature dataset from a single source. Common temperature datasets are mainly the following: experimental temperature data obtained by building an experimental platform on the ground to simulate the flight environment; simulated temperature data obtained by high-precision numerical simulation calculations using simulation software; point temperature data collected by temperature sensors deployed on orbiting aircraft; temperature data obtained by taking pictures of on-orbit aircraft with infrared cameras.

[0004] However, the above data types have their own shortcomings, and none of them can accurately reflect the overall temperature of the on-orbit spacecraft. Specifically, the ground experimental platform sets the external environmental conditions of the spacecraft in an idealized way, ignoring the influencing factors such as cosmic rays and solar radiation changes. There is a large error between the experimental temperature data and the actual temperature field; the simulation calculation makes an idealized approximation for both the external environmental conditions and the spacecraft modeling, and the calculated simulated temperature data is difficult to truly reflect the temperature field of the on-orbit spacecraft; the temperature measurement results of the temperature sensor are accurate and fast, but due to the limited number and position of the sensors deployed on the spacecraft, only limited point temperature data can be obtained, and the complete temperature field data cannot be obtained; the infrared camera temperature measurement can obtain complete real-time temperature field information by taking pictures, but the acquisition and transmission of infrared images are prone to noise pollution. Therefore, when the deep neural network model obtained by training the deep neural network model using single-source data is used to reconstruct the temperature field, there is a large deviation between the temperature field reconstruction results obtained and the actual on-orbit spacecraft temperature field data, which cannot well reflect the distribution of the actual temperature field of the on-orbit spacecraft. Summary of the invention

[0005] In order to solve some or all of the technical problems existing in the above-mentioned prior art, the present invention provides a temperature field reconstruction method based on multi-source information fusion.

[0006] The technical solution of the present invention is as follows:

[0007] A temperature field reconstruction method based on multi-source information fusion is provided, the method comprising:

[0008] Acquire multi-point temperature data measured by multiple temperature sensors deployed on the aircraft whose temperature field is to be reconstructed;

[0009] The acquired multi-point temperature data is input into a pre-built and trained deep neural network model to obtain a global temperature field output by the deep neural network model. The deep neural network model is trained using a pre-acquired training data set, and the training data set includes multiple training data, including: multi-point temperature data measured by multiple temperature sensors deployed on an aircraft under the same working condition, an infrared camera temperature field obtained by an infrared image of the aircraft taken by an infrared camera, and a ground experiment temperature field corresponding to the aircraft obtained through a ground experiment platform. The multi-point temperature data in the training data is used as the input of the deep neural network model, and the fused temperature field corresponding to the training data is used as the output of the deep neural network model to train the deep neural network model. The fused temperature field corresponding to the training data is obtained by fusing the infrared camera temperature field and the ground experiment temperature field in the training data.

[0010] In some optional embodiments, the fusion temperature field is obtained by:

[0011] Fourier transform is used to transform the infrared camera temperature field and the ground experiment temperature field into frequency domain respectively;

[0012] A Gaussian low-pass filter is used to extract the low-frequency information of the frequency domain transformation result of the infrared camera temperature field;

[0013] The high-frequency information of the frequency domain transformation results of the ground experimental temperature field is extracted using a Gaussian high-pass filter;

[0014] The extracted low-frequency information and high-frequency information are fused to obtain fused information;

[0015] The fused information is converted from frequency domain to time domain using inverse Fourier transform to obtain the fused temperature field.

[0016] In some optional implementations, the frequency domain transformation of the temperature field is performed using the following transformation formula:

[0017]

[0018] Among them, F(u,v) represents the frequency domain transformation result of the temperature field, H and W represent the height and width of the temperature field respectively, (x,y) represents the coordinates of the time domain temperature field, (u,v) represents the coordinates of the frequency domain temperature field, I(x,y) represents the temperature value at the position (x,y) in the time domain temperature field, e represents the natural constant, and j represents the imaginary unit.

[0019] In some optional implementations, the low-frequency information of the frequency domain transformation result of the infrared camera temperature field is extracted using the following formula:

[0020]

[0021] Among them, P L represents low-frequency information, D(u,v) represents the distance from the point (u,v) in the frequency domain temperature field to the center point of the frequency domain temperature field, D 0 Indicates the preset Gaussian cutoff frequency, F tim_I (u,v) represents the frequency domain transformation result of the infrared camera temperature field.

[0022] In some optional implementations, the high-frequency information of the frequency domain transformation result of the ground experimental temperature field is extracted using the following formula:

[0023]

[0024] Among them, P H Represents high-frequency information, F tim_G (u,v) represents the frequency domain transformation result of the ground experimental temperature field.

[0025] In some optional implementations, the extracted low-frequency information and high-frequency information are fused using the following formula:

[0026] P fre_F (u,v)=P H +P L ;

[0027] Among them, P fre_F (u,v) represents the fusion information.

[0028] In some optional embodiments, the Gaussian cutoff frequency D 0 Determine using:

[0029] Step 211, respectively calculate the Gaussian cutoff frequency D 0 Take the value in the segment endpoint array [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3] and the error between the fused temperature field and its corresponding multi-point temperature data, and determine the value D corresponding to the minimum error i , the value D i As the optimal Gaussian cutoff frequency, the Gaussian cutoff frequency value range is updated to (D i-1 ,D i+1 ), D i represents the i-th value in the segment endpoint array [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3];

[0030] Step 212, divide the Gaussian cutoff frequency value range into 20 segments at equal intervals to obtain a segment endpoint array [D 1 ,D 2 ,...,D 21 ], calculate the Gaussian cutoff frequency D respectively 0 Get segment endpoint array [D 1 ,D 2 ,...,D 21 ] is the error between the fused temperature field obtained when the value is within ] and its corresponding multi-point temperature data, and the value D corresponding to the minimum error is determined j , the value D j As the optimal Gaussian cutoff frequency, the Gaussian cutoff frequency value range is updated to (D j-1 ,D j+1 ), D j Represents the segment endpoint array [D 1 ,D 2 ,...,D 21 ];

[0031] Step 213, repeat step 212 until the set loop stop condition is reached, and the current optimal Gaussian cutoff frequency is used as the Gaussian cutoff frequency D 0 .

[0032] In some optional embodiments, the error between the fused temperature field and its corresponding multi-point temperature data is calculated using the following formula:

[0033]

[0034] Among them, Loss PI It represents the error between the fused temperature field and its corresponding multi-point temperature data, I tim_S (x i ,y i ) represents multi-point temperature data (x i ,y i ) temperature value at the position, P tim_F (x i ,y i ) represents the temperature field corresponding to the multi-point temperature data (x i ,y i ) position, and m represents the number of points of multi-point temperature data.

[0035] In some optional implementations, the cycle stop condition is:

[0036] Loss PI (D j+1 )-Loss PI (D j-1 )<1×10 -3 ;

[0037] Among them, Loss PI (D j+1 ) represents the Gaussian cutoff frequency D 0 Take D j+1 The error between the fused temperature field obtained and its corresponding multi-point temperature data, Loss PI (D j-1 ) represents the Gaussian cutoff frequency D 0 Take D j-1 The error between the fused temperature field obtained and its corresponding multi-point temperature data.

[0038] In some optional embodiments, using the multi-point temperature data in the training data as the input of the deep neural network model, using the fused temperature field corresponding to the training data as the output of the deep neural network model, training the deep neural network model includes the following steps:

[0039] Step 221, inputting the multi-point temperature data in the training data into the deep neural network model to obtain the predicted global temperature field output by the deep neural network model;

[0040] Step 222, calculating a preset loss function according to the fused temperature field corresponding to the training data and the predicted global temperature field output by the deep neural network model;

[0041] Step 223, determine whether the preset training stop condition is reached, if so, use the current deep neural network model as the deep neural network model that has completed training, if not, use the loss function to update the parameters of the deep neural network model, and return to step 221.

[0042] The main advantages of the technical solution of the present invention are as follows:

[0043] The temperature field reconstruction method based on multi-source information fusion of the present invention reconstructs the temperature field of the aircraft by fusing ground experimental temperature field data, infrared camera temperature field data and multi-point temperature data collected by sensors. It can reduce the influence of noise and errors existing in single data, reduce the error between the predicted temperature field and the actual temperature field, and improve the accuracy and credibility of the aircraft temperature field reconstruction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flow chart of a temperature field reconstruction method based on multi-source information fusion provided by an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of the principle of a temperature field reconstruction method based on multi-source information fusion provided by an embodiment of the present invention;

[0047] Figure 3 A schematic diagram of a fusion temperature field acquisition process provided by an embodiment of the present invention;

[0048] Figure 4 A schematic diagram of the structure of an aircraft heating plate provided as an example of the present invention;

[0049] Figure 5a-5c Schematic diagrams of a real temperature field, a noisy infrared temperature field and a ground experimental temperature field given as examples of the present invention;

[0050] Figure 6a To use Figure 5a-5c The schematic diagram of the fusion temperature field obtained by the temperature field shown;

[0051] Figure 6b for Figure 6a The fusion temperature field shown is Figure 5a Schematic diagram of the error between the actual temperature fields shown;

[0052] Figure 6c For Figure 5b Schematic diagram of the temperature field obtained after filtering and denoising the temperature field shown;

[0053] Figure 6d for Figure 6c The temperature field shown is Figure 5a Schematic diagram of the error between the actual temperature fields shown;

[0054] Figure 7a A schematic diagram of the mean absolute error between the temperature field reconstruction result obtained by the model and the real temperature field after model training using ground experimental temperature field data, noisy infrared camera temperature field data, filtered noise-reduced temperature field data, and fused temperature field data as outputs of a deep neural network model when Gaussian noise is superimposed on the infrared camera temperature field data given as an example of the present invention;

[0055] Figure 7b It is a schematic diagram of the improvement of the signal-to-noise ratio of the filtered denoised temperature field data and the fused temperature field data when different Gaussian noises are superimposed, compared with the signal-to-noise ratio of the infrared camera temperature field data before processing, given as an example of the present invention;

[0056] Figure 7c A schematic diagram of the mean absolute error between the temperature field reconstruction result obtained by the model and the real temperature field after model training using ground experimental temperature field data, noisy infrared camera temperature field data, filtered noise-reduced temperature field data, and fused temperature field data as the output of a deep neural network model when the infrared camera temperature field data is superimposed with salt and pepper noise, given as an example of the present invention;

[0057] Figure 7d It is a schematic diagram of the improvement of the signal-to-noise ratio of the filtered denoised temperature field data and the fused temperature field data when different salt and pepper noises are superimposed, compared with the signal-to-noise ratio of the infrared camera temperature field data before processing, given as an example of the present invention;

[0058] Figure 8a-8c Schematic diagrams of another real temperature field, another noisy infrared temperature field and another ground experimental temperature field respectively given as examples of the present invention;

[0059] Figure 9a To use Figure 8a-8c The schematic diagram of the fusion temperature field obtained by the temperature field shown;

[0060] Figure 9b for Figure 9a The fusion temperature field shown is Figure 8a Schematic diagram of the error between the actual temperature fields shown;

[0061] Figure 10a-10c They are schematic diagrams of the mean absolute error between the temperature field reconstruction result obtained by the model and the real temperature field after the ground experimental temperature field data and the fused temperature field data are used as the output of the deep neural network model for model training when three different temperature biases are superimposed on the ground experimental temperature field data according to the examples of the present invention;

[0062] Figure 10d-10f They are schematic diagrams of the mean absolute error between the temperature field reconstruction result obtained by the model and the real temperature field after the model training using the noisy infrared camera temperature field data and the fused temperature field data as the output of the deep neural network model when the ground experimental temperature field data is superimposed with three different temperature biases according to the examples of the present invention;

[0063] Figure 10g-10i They are schematic diagrams of improvement in the signal-to-noise ratio of the corresponding fused temperature field data when three different temperature biases are superimposed on the ground experimental temperature field data compared with the signal-to-noise ratio of the temperature field data before processing, respectively, given as examples of the present invention. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0065] The technical solution provided by the embodiments of the present invention is described in detail below with reference to the accompanying drawings.

[0066] refer to Figure 1-2 The embodiment of the present invention provides a temperature field reconstruction method based on multi-source information fusion, the method comprising the following steps:

[0067] Step 1, obtaining multi-point temperature data measured by multiple temperature sensors deployed on the aircraft whose temperature field is to be reconstructed;

[0068] Step 2: Input the acquired multi-point temperature data into a pre-built and trained deep neural network model to obtain the global temperature field output by the deep neural network model.

[0069] In an embodiment of the present invention, the deep neural network model is trained using a pre-acquired training data set, and the training data set includes multiple training data, including: multi-point temperature data measured by multiple temperature sensors deployed on the aircraft under the same working conditions, an infrared camera temperature field obtained by an infrared image of the aircraft taken by an infrared camera, and a ground experimental temperature field corresponding to the aircraft obtained through a ground experimental platform.

[0070] In an embodiment of the present invention, when a training data set is used to train a deep neural network model, the multi-point temperature data in the training data is used as the input of the deep neural network model, and the fused temperature field corresponding to the training data is used as the output of the deep neural network model to train the deep neural network model.

[0071] In an embodiment of the present invention, the fused temperature field corresponding to the training data is obtained by fusing the infrared camera temperature field and the ground experiment temperature field in the training data.

[0072] In the embodiment of the present invention, the training data may adopt existing historical data, or may be obtained by re-measurement, shooting and experimentation.

[0073] It should be noted that, in the embodiment of the present invention, different training data correspond to different operating conditions of the aircraft, and the infrared camera temperature field represents the infrared camera temperature field after noise reduction.

[0074] The temperature field reconstruction method based on multi-source information fusion provided in an embodiment of the present invention reconstructs the temperature field of an aircraft by fusing ground experimental temperature field data, infrared camera temperature field data and multi-point temperature data collected by sensors. It can reduce the influence of noise and errors existing in single data, reduce the error between the predicted temperature field and the actual temperature field, and improve the accuracy and credibility of the aircraft temperature field reconstruction results.

[0075] refer to Figure 3 , further, in an embodiment of the present invention, the fusion temperature field corresponding to the training data is obtained by:

[0076] Step 201, using Fourier transform to perform frequency domain transformation on the temperature field of the infrared camera and the temperature field of the ground experiment respectively;

[0077] Step 202, using a Gaussian low-pass filter to extract low-frequency information of the frequency domain transformation result of the infrared camera temperature field;

[0078] Step 203, using a Gaussian high-pass filter to extract high-frequency information of the frequency domain transformation result of the ground experimental temperature field;

[0079] Step 204, fusing the extracted low-frequency information and high-frequency information to obtain fused information;

[0080] Step 205, using inverse Fourier transform to convert the fused information from frequency domain to time domain to obtain a fused temperature field.

[0081] In the embodiment of the present invention, a two-dimensional discrete Fourier transform (DFT transform) is used to perform frequency domain transform on the temperature field of the infrared camera and the ground experiment temperature field to complete the conversion from the time domain temperature field to the frequency domain temperature field, which is specifically expressed as follows:

[0082]

[0083] Among them, F(u,v) represents the frequency domain transformation result of the temperature field, and the frequency domain transformation result of the temperature field is the frequency domain temperature field. H and W represent the height and width of the temperature field respectively. (x,y) represents the coordinates of the time domain temperature field, and the time domain temperature field is the temperature field before the frequency domain transformation. (u,v) represents the coordinates of the frequency domain temperature field. I(x,y) represents the temperature value at the position (x,y) in the time domain temperature field. e represents a natural constant, and j represents an imaginary unit.

[0084] Specifically, set: I tim_I (x, y) is the temperature field of the infrared camera, F tim_I (u, v) is the frequency domain transformation result of the infrared camera temperature field, that is, the frequency domain temperature field corresponding to the infrared camera temperature field, then F tim_I (u,v) can be expressed as:

[0085]

[0086] Setting: I tim_G (x, y) is the ground experimental temperature field, F tim_G (u, v) is the frequency domain transformation result of the ground experimental temperature field, that is, the frequency domain temperature field corresponding to the ground experimental temperature field, then F tim_G (u,v) can be expressed as:

[0087]

[0088] It should be noted that, in the embodiment of the present invention, in order to achieve the fusion of the infrared camera temperature field and the ground experiment temperature field, the infrared camera temperature field and the ground experiment temperature field have the same size, that is, the height and width of the two temperature fields are the same.

[0089] Furthermore, since the frequency domain image after Fourier transformation is shifted, the area closer to the center point represents the low-frequency information of the image. Therefore, in the embodiment of the present invention, a Gaussian low-pass filter and a Gaussian high-pass filter are used to extract the low-frequency information and high-frequency information of the temperature field respectively.

[0090] Specifically, when the height and width of the frequency domain image are H and W respectively, the coordinates of the center point C of the frequency domain image can be expressed as:

[0091] C(u,v)=((W-1) / 2,(H-1) / 2);

[0092] Then the distance D between each point M(u,v) and the center point C(u,v) in the frequency domain image can be expressed as:

[0093] D(u,v)=(u-(W-1) / 2) 2 +(v-(H-1) / 2) 2 .

[0094] For frequency domain images, low-frequency information is extracted by attenuating high frequencies. When the Gaussian cutoff frequency is D 0 When , the transfer function of the Gaussian low-pass filter (GLPF) can be expressed as:

[0095]

[0096] Among them, H LP (u,v) represents the transfer function of the Gaussian low-pass filter.

[0097] After passing through the Gaussian low-pass filter, the low-frequency information extracted from the frequency domain image can be expressed as:

[0098] P L (u,v)=H LP (u,v)F(u,v);

[0099] Among them, P L (u,v) represents low-frequency information.

[0100] Similarly, using Gaussian filter, we can get the high-frequency information of the frequency domain image. When the Gaussian cutoff frequency is D 0 When , the transfer function of the Gaussian high-pass filter (GHPF) can be expressed as:

[0101]

[0102] Among them, H HP (u,v) represents the transfer function of the Gaussian high-pass filter.

[0103] After passing through the Gaussian high-pass filter, the high-frequency information extracted from the frequency domain image can be expressed as:

[0104] P H (u,v)=H HP (u,v)F(u,v);

[0105] Among them, P H(u,v) represents high-frequency information.

[0106] Based on the above analysis, in an embodiment of the present invention, the following formula is used to extract the low-frequency information of the frequency domain transformation result of the infrared camera temperature field:

[0107]

[0108] Among them, P L represents low-frequency information, D(u,v) represents the distance from the point (u,v) in the frequency domain temperature field to the center point of the frequency domain temperature field, D 0 Indicates the preset Gaussian cutoff frequency, F tim_I (u, v) represents the frequency domain transformation result of the infrared camera temperature field, that is, the frequency domain temperature field corresponding to the infrared camera temperature field.

[0109] The high-frequency information of the frequency domain transformation results of the ground experimental temperature field is extracted using the following formula:

[0110]

[0111] Among them, P H Represents high-frequency information, F tim_G (u, v) represents the frequency domain transformation result of the ground experimental temperature field, that is, the frequency domain temperature field corresponding to the ground experimental temperature field.

[0112] Furthermore, in the embodiment of the present invention, when the low-frequency information and the high-frequency information are fused, the two are directly superimposed, and the following formula is specifically used to fuse the extracted low-frequency information and high-frequency information:

[0113]

[0114] Among them, P fre_F (u,v) represents the fusion information.

[0115] Furthermore, in the embodiment of the present invention, the fusion information is converted from the frequency domain to the time domain using the inverse Fourier transform (DFT inverse transform) to obtain the fusion temperature field, which is specifically expressed as:

[0116]

[0117] Among them, P tim_F (x,y) represents the fusion temperature field, P fre_F (u,v) represents the fusion information.

[0118] Since the ground experiment temperature field data is less affected by mutation noise, and the infrared camera temperature field data is noisy but can reflect the overall change trend of the temperature field, therefore, in an embodiment of the present invention, by adopting the frequency domain information fusion method, Fourier transform and Gaussian filter are used to respectively extract the high-frequency information of the ground experiment temperature field and the low-frequency information of the infrared camera temperature field for fusion, the noise in the infrared camera temperature field can be removed, the distribution characteristics of the ground experiment temperature field can be extracted, the quality of the temperature field data can be improved, and then the prediction accuracy of the trained deep neural network model can be improved.

[0119] Furthermore, since the fused temperature fields obtained are different when different fusion thresholds (i.e., Gaussian cutoff frequencies) are selected for filtering and temperature field fusion, in order to obtain the best fusion threshold and improve the temperature field data fusion effect, a segmented adaptive threshold selection method is used in the embodiment of the present invention to determine the fusion threshold.

[0120] Specifically, in the embodiment of the present invention, the Gaussian cutoff frequency D 0 Determine using:

[0121] Step 211, respectively calculate the Gaussian cutoff frequency D 0 Take the value in the segment endpoint array [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3] and the error between the fused temperature field and its corresponding multi-point temperature data, and determine the value D corresponding to the minimum error i , the value D i As the optimal Gaussian cutoff frequency, the Gaussian cutoff frequency value range is updated to (D i-1 ,D i+1 ), D i represents the i-th value in the segment endpoint array [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3];

[0122] In the embodiment of the present invention, considering the Gaussian cutoff frequency D 0 >0, and when D 0 When D is too large (greater than 1e3) or too small (1e-3), different sizes of D 0 The impact on the error is already less than 1e-5, and the change in error can be ignored. Therefore, the Gaussian cutoff frequency D 0 The initial value range of is set to (1e3, 1e-3), and the initial segment endpoint array is set to [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3].

[0123] In an embodiment of the present invention, considering that the temperature data measured by the sensor can most realistically reflect the current temperature field, the multi-point temperature data measured by the sensor is used as a standard, and the error between the fused temperature field and its corresponding multi-point temperature data is calculated to determine whether the selected cutoff frequency is the optimal cutoff frequency.

[0124] In the embodiment of the present invention, the error between the fused temperature field and the corresponding multi-point temperature data is calculated using the following formula:

[0125]

[0126] Among them, Loss PI It represents the error between the fused temperature field and its corresponding multi-point temperature data, I tim_S (x i ,y i ) represents multi-point temperature data (x i ,y i ) temperature value at the position, P tim_F (x i ,y i ) represents the temperature field corresponding to the multi-point temperature data (x i ,y i ) position, and m represents the number of points of multi-point temperature data.

[0127] According to the above error formula, it can be seen that the smaller the error between the fused temperature field and its corresponding multi-point temperature data, the better the temperature field fusion effect is, and the closer the fused temperature field is to the real temperature field.

[0128] Step 212, divide the Gaussian cutoff frequency value range into 20 segments at equal intervals to obtain a segment endpoint array [D 1 ,D 2 ,...,D 21 ], calculate the Gaussian cutoff frequency D respectively 0 Get segment endpoint array [D 1 ,D 2 ,...,D 21 ] is the error between the fused temperature field obtained when the value is within ] and its corresponding multi-point temperature data, and the value D corresponding to the minimum error is determined j , the value D j As the optimal Gaussian cutoff frequency, the Gaussian cutoff frequency value range is updated to (D j-1 ,D j+1 ), D j Represents the segment endpoint array [D 1 ,D 2 ,...,D 21 ];

[0129] Step 213, repeat step 212 until the set loop stop condition is reached, and the current optimal Gaussian cutoff frequency is used as the Gaussian cutoff frequency D 0 .

[0130] In the embodiment of the present invention, the cycle stop condition is specifically set according to actual needs.

[0131] In an optional implementation of the embodiment of the present invention, the loop stop condition is set to:

[0132] Loss PI (D j+1 )-Loss PI (D j-1 )<1×10 -3 ;

[0133] Among them, Loss PI (D j+1 ) represents the Gaussian cutoff frequency D 0 Take D j+1 The error between the fused temperature field obtained and its corresponding multi-point temperature data, Loss PI (D j-1 ) represents the Gaussian cutoff frequency D 0 Take D j-1 The error between the fused temperature field obtained and its corresponding multi-point temperature data.

[0134] When the above-mentioned cycle stop condition is met, it means that the Gaussian cutoff frequency D 0 In the interval (D j-1 ,D j+1 The difference in fusion effect between different values ​​of ) is extremely small and can be ignored.

[0135] In the embodiment of the present invention, by adopting the above method to select the value of the Gaussian cutoff frequency, the optimal Gaussian cutoff frequency can be determined and the optimal fusion temperature field can be obtained.

[0136] Furthermore, in an embodiment of the present invention, the multi-point temperature data in the training data is used as the input of the deep neural network model, and the fused temperature field corresponding to the training data is used as the output of the deep neural network model, and the deep neural network model is trained, further comprising the following steps:

[0137] Step 221, inputting the multi-point temperature data in the training data into the deep neural network model to obtain the predicted global temperature field output by the deep neural network model;

[0138] In an embodiment of the present invention, the parameters of the deep neural network model are initialization parameters. During the training process of the deep neural network model, the parameters of the deep neural network model can be continuously updated and learned.

[0139] Step 222, calculating a preset loss function according to the fused temperature field corresponding to the training data and the predicted global temperature field output by the deep neural network model;

[0140] In the embodiment of the present invention, the loss function is specifically set according to actual needs. For example, the loss function uses the mean absolute error (MAE) or mean square error (MSE) of the fused temperature field and the predicted global temperature field.

[0141] Step 223, determine whether the preset training stop condition is reached, if so, use the current deep neural network model as the deep neural network model that has completed the training, if not, use the loss function to update the parameters of the deep neural network model, and return to step 221.

[0142] In an embodiment of the present invention, the training stop condition is specifically set according to the actual situation, for example, it is set to: the loss function obtained by verifying the deep neural network model using a preset verification data set reaches the minimum value or the decrease of the loss function is less than the set value.

[0143] Furthermore, in one embodiment of the present invention, the stochastic gradient descent method is used to train and update the parameters of the deep neural network model.

[0144] Specifically, the following formula is used to update the parameters of the deep neural network model:

[0145]

[0146] Among them, Θ t+1 represents the parameters of the deep neural network model at the t+1th iteration, Θ t represents the parameters of the deep neural network model at the tth iteration, Δ[·] represents the optimizer, η represents the learning rate, L(Θ) represents the loss function, and Θ represents the parameters of the deep neural network model. The optimizer is, for example, Adam, SGD, etc. The learning rate is pre-set to control the speed of parameter update.

[0147] The effect of the temperature field reconstruction method based on multi-source information fusion provided by an embodiment of the present invention is described below with reference to specific examples:

[0148] refer to Figure 4 In this example, a simulation test is performed by deploying heating elements on a heating plate. The heating plate is a rectangular shape with a side length of L. The heating plate is surrounded by an insulating wall made of insulating material. The heating plate can only pass through a portion of the insulating wall with a length of The heat dissipation holes exchange heat energy with the outside world. There are multiple heating elements φk (k = 1, 2, ..., 11) distributed inside the heating plate, i.e., heat sources. By setting the heat dissipation conditions and simulating the actual heating situation of the aircraft, the numerical calculation method is used to obtain high-precision temperature field data.

[0149] Based on the high-precision temperature field data obtained by numerical calculation methods, high-resolution temperature field images and low-resolution temperature field images are divided. The resolution of the high-resolution temperature field image is 200*200, and the resolution of the low-resolution temperature field image is 50*50. The high-resolution temperature field image is used as the real temperature field data. The real temperature field data is superimposed with common image noise of different intensities and types as the temperature field data of the infrared camera. The low-resolution temperature field image is interpolated into a high-resolution temperature field image and then superimposed with temperature biases of different sizes as the ground experiment temperature field data. 20 fixed-point temperature values ​​are equally spaced in the real temperature field data as the multi-point temperature data collected by the sensor. The three types of data are fused to obtain the corresponding fused temperature field.

[0150] Furthermore, based on the acquisition method of the three types of data set above, for the infrared camera temperature field data, two types of noise, point Gaussian noise and salt and pepper noise, are selected by superimposing the high-resolution temperature field image to simulate the internal noise and external noise of the infrared camera temperature measurement. Since the internal noise comes from factors such as vibration inside the camera, the superimposed point Gaussian noise is a Gaussian noise that randomly superimposes a certain distribution at each point in the image, and the noise size is determined by different means and variances; since external noise may be an impact on the global process such as transmission, the salt and pepper noise is to change a certain point value to black or white in a certain proportion on the global image. Specifically, the high-resolution temperature field image is randomly selected in proportion within the global range, and the temperature value of the point is randomly changed to the maximum and minimum values ​​of the temperature field.

[0151] The temperature field data of different infrared cameras obtained above were used for fusion experiments with the temperature field data of the ground experiment and the multi-point temperature data. During the fusion, the temperature field data of the ground experiment and the multi-point temperature data remained unchanged. The temperature field data of the ground experiment was generated by bilinear interpolation of the low-resolution temperature field image to high resolution, and the temperature bias was set to 0. The multi-point temperature data was generated by selecting 20 points from the high-resolution temperature field image obtained by the high-precision temperature field data. A Gaussian filter with a filter kernel size of 15 was selected to filter out Gaussian noise, and a median filter with a filter kernel size of 3 was selected to filter out salt and pepper noise.

[0152] Figure 5a-5c The schematic diagrams of a real temperature field, a noisy infrared temperature field and a ground experimental temperature field are respectively given as examples of the present invention. Figure 6a To use Figure 5a-5cThe schematic diagram of the fusion temperature field obtained by the temperature field shown, Figure 6b for Figure 6a The fusion temperature field shown is Figure 5a Schematic diagram of the error between the actual temperature field shown, Figure 6c For Figure 5b The schematic diagram of the temperature field obtained after filtering and denoising the temperature field shown in FIG. Figure 6d for Figure 6c The temperature field shown is Figure 5a Schematic diagram of the error between the actual temperature field shown.

[0153] According to the above figures, it can be seen that compared with the temperature field after filtering and denoising, the mean absolute error (MAE) between the fused temperature field and the true temperature field is smaller, and the fused temperature field is closer to the true temperature field.

[0154] Gaussian noise of different sizes and types (noise distribution obeys mean = 0, variance = 0.5, 1.0, 1.5, 2.0, ..., 5.0) and salt and pepper noise (noise threshold = 0.08, 0.10, 0.12, ..., 0.20) are superimposed on the temperature field data of the infrared camera, and experiments are carried out using the temperature field reconstruction method based on multi-source information fusion provided in the embodiments of the present invention. Figure 7a A schematic diagram of the mean absolute error (MAE) between the temperature field reconstruction result obtained by the model and the real temperature field after model training using ground experimental temperature field data, noisy infrared camera temperature field data, filtered noise-reduced temperature field data, and fused temperature field data as the output of a deep neural network model when Gaussian noise is superimposed on the infrared camera temperature field data given as an example of the present invention; Figure 7b It is a schematic diagram of the improvement of the signal-to-noise ratio of the filtered denoised temperature field data and the fused temperature field data when different Gaussian noises are superimposed, compared with the signal-to-noise ratio of the infrared camera temperature field data before processing, given as an example of the present invention; Figure 7c A schematic diagram of the mean absolute error (MAE) between the temperature field reconstruction result obtained by the model and the real temperature field after model training using ground experimental temperature field data, noisy infrared camera temperature field data, filtered noise-reduced temperature field data, and fused temperature field data as the output of a deep neural network model when the infrared camera temperature field data is superimposed with salt and pepper noise given as an example of the present invention; Figure 7d The figure is a schematic diagram showing the improvement of the signal-to-noise ratio of the temperature field data of the infrared camera before processing when the temperature field data of the infrared camera is subjected to the superposition of different salt and pepper noises. Figures 7a-7dIn the figure, experiment corresponds to the ground experiment temperature field data, infrared corresponds to the noisy infrared camera temperature field data, filter corresponds to the filtered and denoised temperature field data, and fusion corresponds to the fused temperature field data.

[0155] Among them, when training the deep neural network model, 1000 sets of training data are used, the PyTorch framework is used, the MLP network structure is used, the optimizer is Adam, the batch size is 8, the training cycle is 200, the learning rate is cos change, the change cycle is 200, and the minimum value is 1e-6.

[0156] According to the above figures, it can be seen that although the direct denoising effect of filtering and denoising by filters is better when the noise is relatively small, when the noise is large and affects the overall data distribution (for example, when the superimposed Gaussian noise is greater than 2.0 and the salt and pepper noise is greater than 0.14), the filter denoising effect is limited. The fused temperature field can integrate the comprehensive information of the two types of data, which can effectively reduce the error, improve the signal-to-noise ratio of the temperature field, and improve the prediction accuracy of the temperature field.

[0157] Furthermore, based on the above-mentioned acquisition method of the three types of data, for the ground experimental temperature field data, the low-resolution temperature field image is bilinearly interpolated into a high-resolution (200*200) image to generate the ground experimental temperature field data, and the temperature bias is set to 0.1, 0.5, 2, and 10 respectively.

[0158] The fusion experiment was conducted using the temperature field data of different ground experiments obtained above, the temperature field data of infrared cameras, and the temperature data of multiple points. During the fusion, the temperature field data of infrared cameras and the temperature field data of multiple points remained unchanged. The temperature field data of infrared cameras were generated by superimposing noise using high-resolution temperature field images. The superimposed noise selected was point Gaussian noise (obeying a Gaussian distribution with a mean of 1 and a variance of 1) and salt and pepper noise (signal-to-noise ratio of 0.5). The temperature data of multiple points was generated by selecting 20 points from the high-resolution temperature field image obtained from the high-precision temperature field data.

[0159] Figure 8a-8c Schematic diagrams of another real temperature field, another noisy infrared temperature field and another ground experimental temperature field given as examples of the present invention, Figure 9a To use Figure 8a-8c The schematic diagram of the fusion temperature field obtained by the temperature field shown, Figure 9b for Figure 9a The fusion temperature field shown is Figure 8a Schematic diagram of the error between the actual temperature field shown.

[0160] It can be seen from the above figures that the mean absolute error (MAE) between the fused temperature field and the true temperature field is smaller, and the fused temperature field is closer to the true temperature field.

[0161] Different sizes of temperature bias (bias = 1, 2, 3, ..., 10) are superimposed on the ground experimental temperature field data, and the same size of Gaussian noise (noise distribution obeys mean = 0, variance = 1) or salt and pepper noise (noise threshold = 0.5) is superimposed on the infrared camera temperature field data, and experiments are carried out using the temperature field reconstruction method based on multi-source information fusion provided by the embodiments of the present invention. Figure 10a-10c Schematic diagrams of the mean absolute error (MAE) between the temperature field reconstruction result obtained by the model and the real temperature field after model training using the ground experimental temperature field data and the fused temperature field data as the output of the deep neural network model when the ground experimental temperature field data is superimposed with three different temperature biases according to the examples of the present invention; Figure 10d-10f Schematic diagrams of the mean absolute error (MAE) between the temperature field reconstruction result obtained by the model and the real temperature field after model training using noisy infrared camera temperature field data and fused temperature field data as the output of the deep neural network model when three different temperature biases are superimposed on the ground experimental temperature field data given in the examples of the present invention; Figure 10g-10i The schematic diagrams are respectively given in the examples of the present invention, showing the improvement of the signal-to-noise ratio of the fused temperature field data when the ground experimental temperature field data is superimposed with three different temperature biases compared with the signal-to-noise ratio of the temperature field data before processing. Figures 10a-10i In the figure, experiment corresponds to the ground experimental temperature field data, infrared corresponds to the noisy infrared camera temperature field data, and D_best and fusion correspond to the fused temperature field data.

[0162] Among them, when training the deep neural network model, 1000 sets of training data are used, the PyTorch framework is used, the MLP network structure is used, the optimizer is Adam, the batch size is 8, the training cycle is 200, the learning rate is cos change, the change cycle is 200, and the minimum value is 1e-6.

[0163] It can be seen from the above figures that for temperature deviations of different sizes, the temperature field reconstruction method based on multi-source information fusion provided by the embodiment of the present invention has good robustness, can significantly reduce the prediction error, improve the temperature field signal-to-noise ratio, and improve the temperature field prediction accuracy.

[0164] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In addition, "front", "back", "left", "right", "upper" and "lower" in this article are all referenced to the placement state shown in the accompanying drawings.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A temperature field reconstruction method based on multi-source information fusion, characterized in that: include: Acquire multi-point temperature data measured by multiple temperature sensors deployed on the aircraft whose temperature field is to be reconstructed; The acquired multi-point temperature data is input into a pre-built and trained deep neural network model to obtain a global temperature field output by the deep neural network model. The deep neural network model is trained using a pre-acquired training data set, and the training data set includes multiple training data, including: multi-point temperature data measured by multiple temperature sensors deployed on an aircraft under the same working condition, an infrared camera temperature field obtained by an infrared image of the aircraft taken by an infrared camera, and a ground experiment temperature field corresponding to the aircraft obtained through a ground experiment platform. The multi-point temperature data in the training data is used as the input of the deep neural network model, and the fused temperature field corresponding to the training data is used as the output of the deep neural network model to train the deep neural network model. The fused temperature field corresponding to the training data is obtained by fusing the infrared camera temperature field and the ground experiment temperature field in the training data.

2. The temperature field reconstruction method based on multi-source information fusion according to claim 1 is characterized in that: The fusion temperature field is obtained by: Fourier transform is used to transform the infrared camera temperature field and the ground experiment temperature field into frequency domain respectively; A Gaussian low-pass filter is used to extract the low-frequency information of the frequency domain transformation result of the infrared camera temperature field; The high-frequency information of the frequency domain transformation results of the ground experimental temperature field is extracted using a Gaussian high-pass filter; The extracted low-frequency information and high-frequency information are fused to obtain fused information; The fused information is converted from frequency domain to time domain using inverse Fourier transform to obtain the fused temperature field.

3. The temperature field reconstruction method based on multi-source information fusion according to claim 2 is characterized in that: The frequency domain transformation of the temperature field is performed using the following transformation formula: Among them, F(u,v) represents the frequency domain transformation result of the temperature field, H and W represent the height and width of the temperature field respectively, (x,y) represents the coordinates of the time domain temperature field, (u,v) represents the coordinates of the frequency domain temperature field, I(x,y) represents the temperature value at the position (x,y) in the time domain temperature field, e represents the natural constant, and j represents the imaginary unit.

4. The temperature field reconstruction method based on multi-source information fusion according to claim 3 is characterized in that: The low-frequency information of the frequency domain transformation result of the infrared camera temperature field is extracted using the following formula: Among them, P L represents low-frequency information, D(u,v) represents the distance from the point (u,v) in the frequency domain temperature field to the center point of the frequency domain temperature field, D0 represents the preset Gaussian cutoff frequency, F tim_I (u,v) represents the frequency domain transformation result of the infrared camera temperature field.

5. The temperature field reconstruction method based on multi-source information fusion according to claim 4 is characterized in that: The high-frequency information of the frequency domain transformation results of the ground experimental temperature field is extracted using the following formula: Among them, P H Represents high-frequency information, F tim_G (u,v) represents the frequency domain transformation result of the ground experimental temperature field.

6. The temperature field reconstruction method based on multi-source information fusion according to claim 5 is characterized in that: The extracted low-frequency information and high-frequency information are fused using the following formula: P fre_F (u,v)=P H +P L ; Among them, P fre_F (u,v) represents the fusion information.

7. The temperature field reconstruction method based on multi-source information fusion according to claim 4 or 5, characterized in that: The Gaussian cutoff frequency D0 is determined in the following way: Step 211, respectively calculate the error between the fused temperature field and the corresponding multi-point temperature data when the Gaussian cutoff frequency D0 takes the value in the segmented endpoint array [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3], and determine the value D corresponding to the minimum error i , the value D i As the optimal Gaussian cutoff frequency, the Gaussian cutoff frequency value range is updated to (D i-1 ,D i+1 ), D i represents the i-th value in the segment endpoint array [1e-3, 5e-3, 1e-2, 5e-2, 1e-2, 5e-1, 1, 5, 1e1, 5e1, 1e2, 5e2, 1e3]; Step 212, divide the Gaussian cutoff frequency value range into 20 segments at equal intervals to obtain a segment endpoint array [D1, D2, ..., D 21 ], calculate the Gaussian cutoff frequency D0 and take the segment endpoint array [D1,D2,...,D 21 ] is the error between the fused temperature field obtained when the value is within ] and its corresponding multi-point temperature data, and the value D corresponding to the minimum error is determined j , the value D j As the optimal Gaussian cutoff frequency, the Gaussian cutoff frequency value range is updated to (D j-1 ,D j+1 ), D j Represents the segment endpoint array [D1,D2,...,D 21 ]; Step 213, repeating step 212 until the set loop stop condition is reached, and taking the current optimal Gaussian cutoff frequency as the Gaussian cutoff frequency D0.

8. The temperature field reconstruction method based on multi-source information fusion according to claim 7 is characterized in that: The error between the fused temperature field and its corresponding multi-point temperature data is calculated using the following formula: Among them, Loss PI It represents the error between the fused temperature field and its corresponding multi-point temperature data, I tim_S (x i ,y i ) represents multi-point temperature data (x i ,y i ) temperature value at the position, P tim_F (x i ,y i ) represents the temperature field corresponding to the multi-point temperature data (x i ,y i ) position, and m represents the number of points of multi-point temperature data.

9. The temperature field reconstruction method based on multi-source information fusion according to claim 8 is characterized in that: The cycle stop condition is: Loss PI (D j+1 )-Loss PI (D j-1 )<1×10 -3 ; Among them, Loss PI (D j+1 ) indicates that the Gaussian cutoff frequency D0 is D j+1 The error between the fused temperature field obtained and its corresponding multi-point temperature data, Loss PI (D j-1 ) indicates that the Gaussian cutoff frequency D0 is D j-1 The error between the fused temperature field obtained and its corresponding multi-point temperature data.

10. The temperature field reconstruction method based on multi-source information fusion according to claim 1 is characterized in that: Taking the multi-point temperature data in the training data as the input of the deep neural network model and taking the fused temperature field corresponding to the training data as the output of the deep neural network model, training the deep neural network model includes the following steps: Step 221, inputting the multi-point temperature data in the training data into the deep neural network model to obtain the predicted global temperature field output by the deep neural network model; Step 222, calculating a preset loss function according to the fused temperature field corresponding to the training data and the predicted global temperature field output by the deep neural network model; Step 223, determine whether the preset training stop condition is reached, if so, use the current deep neural network model as the deep neural network model that has completed training, if not, use the loss function to update the parameters of the deep neural network model, and return to step 221.

Images