A method of sample expansion of a thermographic image

Abstract

The application discloses a sample expansion method of thermal imaging images, and belongs to the technical field of infrared thermal imaging and hyperspectral image processing. The application firstly constructs a layered migration simulation framework of spectral-spatial dual-domain coupling on the basis of a long-wave infrared full-link radiation transmission model, breaks through the limitation that the prior art can only realize pixel-level spectral linear migration, and completes full-dimension simulation from pixel-level spectral eigenvalues, target-level spatial textures to scene-level multi-interference composition; secondly, an adversarial iterative optimization closed loop based on physical consistency constraint is constructed, the infrared radiation physical model is combined with adversarial learning, and adaptive iterative optimization of simulation parameters is realized. The application can realize non-cooperative target thermal imaging sample expansion with large quantity, high physical consistency and high domain adaptability by only a small amount of cooperative target samples, and effectively solves the pain points of poor sample authenticity, insufficient diversity and significant domain offset in the prior art.

Description

Technical Field

[0001] This invention relates to the field of infrared thermal imaging and hyperspectral image processing technology, and more specifically, to a method for sample augmentation of thermal imaging images. Background Technology

[0002] Long-wave infrared hyperspectral non-cooperative target detection systems can detect and identify high-value targets in complex scenarios based on the infrared spectral radiation characteristics of objects, and are widely used in military reconnaissance, security monitoring, and field search and rescue. The development of target detection and recognition algorithms for these systems requires massive amounts of labeled samples from various scenarios for model training, testing, and accuracy verification. However, non-cooperative targets often employ counter-reconnaissance methods such as camouflage nets, vegetation cover, smoke and dust interference, and decoy targets, making data acquisition extremely difficult and costly. This inability to obtain sufficient samples to support algorithm development has become a core bottleneck restricting the development of infrared non-cooperative target detection technology.

[0003] Among existing infrared thermal imaging sample simulation technologies, the patent "A Fully Digital Simulation System and Simulation Method for Infrared Imaging Systems" (CN102184288B) discloses a distributed and collaborative infrared imaging full-link simulation scheme, but it does not consider the coupling effect between the target and typical non-cooperative scenarios, lacks effective transfer of non-cooperative target image data, and the simulation samples are not realistic enough. The patent "Infrared Camouflage Effect Simulation System and Evaluation Method Based on Full-Link" (CN103440389B) realizes full-link modeling of infrared camera imaging, but the sample simulation depends on the real infrared images of non-cooperative targets, which greatly limits the diversity and generation efficiency of the samples.

[0004] In addition, existing technologies have two major shortcomings: First, existing sample simulations are mostly open-loop spectral parameter adjustments, which can only achieve pixel-level linear shifts in spectral values. They do not consider the coupling relationship between spectral characteristics and spatial texture and structural features in non-cooperative camouflage scenarios. The generated samples suffer from spatial detail distortion and poor physical consistency, and cannot fit the non-uniform radiation distortion caused by camouflage in real scenarios. Second, existing methods lack quantitative realism verification and adaptive optimization mechanisms, and cannot achieve closed-loop iteration of simulation parameters. The generated samples have significant feature domain offsets from real non-cooperative samples, resulting in limited improvement in generalization performance when used for training downstream detection algorithms. At the same time, they have extremely poor adaptability to complex non-cooperative scenarios with multiple types of camouflage. Summary of the Invention

[0005] The purpose of this invention is to provide a method for sample augmentation of thermal imaging images to solve the above-mentioned problems.

[0006] To achieve the above objectives, the present invention provides a method for sample augmentation of thermal imaging images, comprising the following steps:

[0007] S1: Construct a general model for the full-link radiative transfer of long-wave infrared thermal imaging, obtain basic spectral samples of the cooperative target through spectral acquisition equipment, and obtain basic thermal imaging scene images of the cooperative target through infrared thermal imaging cameras;

[0008] S2: Establish a hierarchical radiometric inversion model. Based on the basic spectral samples and basic thermal imaging scene images, complete the radiometric inversion, radiometric energy inversion and camera DN value inversion, and construct the full-link mapping relationship of cooperative target spectrum-radiance-DN value.

[0009] S3: Construct a hierarchical migration simulation framework with spectral-spatial dual-domain coupling. For non-cooperative target scenarios, pixel-level spectral intrinsic migration, target-level spatial non-uniform camouflage modeling, and scene-level multi-interference composite coupling are completed sequentially to generate initial simulation samples.

[0010] S4: Construct an adversarial iterative optimization closed loop based on physical consistency constraints, perform authenticity verification and iterative optimization of simulation parameters on the initial simulation sample, and generate the optimized target simulation sample;

[0011] S5: Based on the optimized hierarchical migration simulation framework, batch sample generation with multiple scenarios and multiple camouflage parameters is performed to complete the sample expansion of thermal imaging images.

[0012] As a further improvement of the present invention, the construction process of the hierarchical radiative inversion model in step S2 is specifically as follows:

[0013] S21: Construct an emissivity inversion model, based on the law of conservation of energy and Kirchhoff's laws, to establish the conversion relationship between target reflectivity and emissivity; according to the law of conservation of energy, the total incident energy E on the target surface satisfies: E a +E r +E t =E, where E a To absorb radiant energy, E r To reflect radiated energy, E t It is the energy of transmitted radiation;

[0014] The relationship between the target emissivity ε(λ), reflectivity ρ(λ), transmittance τ(λ) and the radiant energy of each component is as follows:

[0015]

[0016] According to Kirchhoff's laws, under thermal equilibrium conditions, we have: ε(λ) + ρ(λ) + τ(λ) = 1;

[0017] For non-transparent targets in long-wave infrared thermal imaging scenarios, the transmittance τ(λ)≈0, thus the conversion relationship between emissivity and reflectivity is obtained: ε(λ)+ρ(λ)=1;

[0018] S22: Construct a radiation energy inversion model, based on Planck's blackbody radiation law and combined with atmospheric radiation transfer effect, to establish the mapping relationship between target emissivity and camera entrance pupil radiation energy;

[0019] The total radiant energy R received at the entrance pupil of the camera λ (T target )for:

[0020] R λ (T target )=τ atom (λ)·Lg(λ)+L up (λ)

[0021] Where, τ atom (λ) represents atmospheric transmittance, L up (λ) represents the upward atmospheric radiance, L g (λ) represents the total radiance of the ground target, and its expression is:

[0022] L g (λ)=ε(λ)·B(λ,T target )+[1-ε(λ)]·L down( λ)

[0023] In the formula, B(λ,T) target (T) represents temperature T target The blackbody spectral radiant exitance is calculated based on Planck's formula:

[0024]

[0025] Where c1 is the first radiation constant, c2 is the second radiation constant, and L down (λ) represents the downward atmospheric radiance;

[0026] Further considering the transmittance τ of the camera optical system camera (λ) is the effective radiant energy actually received by the camera:

[0027] R λ =τ camera (λ)·R λ (T target )

[0028] S23: Construct a camera DN value inversion model, establish the mapping relationship between the radiant energy received by the camera and the DN value of the image pixels, and complete the construction of the full-link mapping.

[0029] As a further improvement of the present invention, the specific process of pixel-level spectral intrinsic migration in step S3 is as follows:

[0030] For the camouflage type of the target non-cooperative scenario, the corresponding spectral data of the camouflage material is obtained, and a nonlinear weighted mixing model in the spectral domain is constructed to complete the intrinsic transfer of the spectrum of the cooperative target.

[0031] The expression for the spectral domain nonlinear weighted mixing model is:

[0032] ε mix (λ)=k(λ)·ε target (λ)+(1-k(λ))·ε camou (λ)

[0033] Where, ε mix (λ) represents the mixed spectral radiance after migration, ε target (λ) represents the spectral radiance of the cooperative target, ε camou (λ) represents the spectral emissivity of the camouflage material, and k(λ) is a wavelength-dependent weighting coefficient, expressed as follows:

[0034] k(λ) = η·τ camou (λ)·exp(-α·ΔT)

[0035] In the formula, η is the coverage duty cycle of the camouflage material, and τ camou (λ) represents the spectral transmittance of the camouflage material, ΔT represents the temperature difference between the target and the camouflage layer, and α represents the temperature influence coefficient, which was obtained through pre-experiment calibration.

[0036] As a further improvement of the present invention, the specific process of target-level spatial non-uniform camouflage modeling in step S3 is as follows:

[0037] Based on the fundamental DN value image obtained from pixel-level spectral intrinsic transfer, a spatial non-uniform modulation model is constructed to fit the spatial radiation distortion caused by the camouflage scene. The expression of the spatial non-uniform modulation model is as follows:

[0038] DN(x,y)=DN0(x,y)·M(x,y)+N(x,y)

[0039] Wherein, DN(x,y) is the modulated pixel DN value, DN0(x,y) is the base DN value after pixel-level migration, M(x,y) is the spatial modulation matrix, which is determined by the spatial texture features of the camouflage type, and N(x,y) is the spatial correlation noise term, which is used to fit the random radiation distortion of the real scene.

[0040] As a further improvement of the present invention, the specific process of scene-level multi-interference composite coupling in step S3 is as follows:

[0041] For complex non-cooperative scenarios involving multiple types of camouflage interference, a multi-interference coupled radiative transfer model is constructed to complete the space-spectral joint coupling simulation of multiple types of camouflage interference. The expression of the multi-interference coupled radiative transfer model is as follows:

[0042] R total (λ,x,y)=R target (λ,x,y)·τ atm (λ,x,y)·τ camou (λ,x,y)+R interfere (λ,x,y)·τ atm (λ,x,y)+L up (λ)

[0043] Among them, R total (λ,x,y) represents the total radiant energy at spatial location (x,y), R target (λ,x,y) represents the target's own radiant energy, τ atm (λ,x,y) represents the spatially dependent atmospheric transmittance, τ camou (λ,x,y) represents the spatially correlated transmittance of the camouflage layer, R interfere (λ,x,y) represents the spatial distribution of radiated energy from the interference source.

[0044] As a further improvement of the present invention, in step S4, the construction process of the adversarial iterative optimization closed loop based on physical consistency constraints first constructs a physical consistency verification module, specifically as follows:

[0045] Construct the physical constraint loss function L phys This is used to verify the physical consistency of infrared radiation in the initial simulation samples. The expression for the physical constraint loss function is:

[0046] L phys =ω1·L spec +ω2·L temp

[0047] Among them, L spec The loss represents the spectral consistency, which is the mean square error between the simulated sample spectral curve and the real non-cooperative sample spectral curve, while simultaneously constraining Kirchhoff's law ε(λ) + ρ(λ) = 1; L temp The temperature radiation uniformity loss is used to constrain the temperature value corresponding to the DN value of the simulation sample to conform to Planck's blackbody radiation law; ω1 and ω2 are preset weighting coefficients.

[0048] As a further improvement of the present invention, the construction process of the adversarial iterative optimization closed loop based on physical consistency constraints further includes constructing a feature domain adversarial adaptation module, specifically:

[0049] An adversarial network structure is constructed, which includes a generator and a discriminator. The generator is the hierarchical transfer simulation framework used to generate simulation samples. The discriminator is a binary classification network used to distinguish whether the input sample is a real non-cooperative sample or a simulation sample.

[0050] Constructing the adversarial loss function L adv Its expression is:

[0051] L adv =E[log(D(real))]+E[log(1-D(sim))]

[0052] Where D is the output probability of the discriminator, real is the real non-cooperative sample, sim is the simulated sample, and E is the mathematical expectation.

[0053] As a further improvement of the present invention, the construction process of the adversarial iterative optimization closed loop based on physical consistency constraints also includes constructing a downstream task feedback optimization module, specifically:

[0054] Using non-cooperative object detection algorithms as the downstream task, a task feedback loss function L is constructed. task The task feedback loss function is the detection accuracy loss of the detection algorithm after training on simulated samples on the real non-cooperative sample test set;

[0055] Construct the total optimization loss function L total Its expression is:

[0056] L total =L phys +λ1·L adv +λ2·Ltask

[0057] Wherein, λ1 and λ2 are preset balance weight coefficients;

[0058] The simulation parameters of the hierarchical migration simulation framework are updated by iteratively optimizing and minimizing the total optimization loss function until the loss function converges, thus completing the optimization of the simulation samples.

[0059] As a further improvement of the present invention, in step S1, the spectral acquisition device is a Fourier transform infrared spectrometer, and the basic spectral sample acquired is the target reflectance spectral data in the 7.5μm-12μm long-wave infrared band; the infrared thermal imaging camera is a long-wave infrared hyperspectral thermal imaging camera, and the basic thermal imaging scene image acquired is a long-wave infrared hyperspectral thermal imaging image of the cooperative target.

[0060] As a further improvement of the present invention, in step S5, the batch sample generation specifically involves: based on the optimized hierarchical migration simulation framework, traversing the preset value ranges of camouflage type, camouflage parameters, environmental parameters, and interference parameters to generate a large batch of multi-scene thermal imaging simulation samples, thereby completing sample expansion; the camouflage type includes at least one of occlusion type, coverage type, and interference type, and the camouflage parameters include at least one of coverage duty cycle, atmospheric transmittance, target temperature, and interference source intensity.

[0061] Compared with the prior art, the advantages of this invention are:

[0062] (1) This scheme breaks through the limitation of existing technologies that can only achieve pixel-level linear spectral migration by constructing a hierarchical migration simulation framework with spectral-spatial dual-domain coupling. It realizes full-dimensional simulation from pixel-level spectral intrinsics, target-level spatial texture to scene-level multi-interference composite, effectively fits the spectral-spatial coupling characteristics of real non-cooperative scenes, greatly improves the physical consistency and detail realism of simulation samples, and realizes the generation of samples of multi-type camouflage composite scenes, significantly improving the diversity of samples.

[0063] (2) This scheme constructs an adversarial iterative optimization closed loop based on physical consistency constraints, which combines the infrared radiation physical model with adversarial learning. This avoids the problem of generating pseudo samples that do not conform to physical laws by pure data-driven methods, and eliminates the feature domain offset between simulation samples and real samples through adversarial adaptation. At the same time, it introduces downstream task feedback to achieve adaptive optimization of samples. When the generated samples are used for training non-cooperative target detection algorithms, the generalization performance and robustness of the algorithm can be significantly improved.

[0064] (3) This scheme improves upon the traditional full-link radiation transfer model, retains the core advantage of cooperative sample-driven model, and significantly reduces the dependence on real non-cooperative samples through hierarchical migration and closed-loop optimization. Only a small number of cooperative samples and a small number of real non-cooperative samples are needed for prior features to achieve a large number of highly realistic sample expansions, effectively solving the industry pain points of high difficulty and high cost in obtaining non-cooperative target thermal imaging samples. Attached Figure Description

[0065] Figure 1 This is a schematic diagram of the process of the present invention;

[0066] Figure 2 Figure (a) shows the non-cooperative sample images generated under the occlusion class conditions in this invention, and Figure (b) shows the basic sample captured by the long-wave infrared camera and the non-cooperative sample generated by simulation. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0068] like Figure 1 As shown in the figure, this embodiment discloses a sample augmentation method for thermal imaging images, and the specific implementation process is as follows:

[0069] Step 1: Basic Data Acquisition and Construction of the End-to-End Radiative Transmission Model

[0070] 1.1 Basic Data Acquisition: Using an ALPHA II Fourier transform infrared spectrometer, reflectance spectral data of cooperative targets (such as uncamouflaged vehicles and equipment) and typical camouflage materials (camouflage nets, vegetation, hay, etc.) in the 7.5μm-12μm long-wave infrared band were acquired as basic spectral samples. Using a FLIR A655sc long-wave infrared hyperspectral thermal imaging camera, long-wave infrared hyperspectral thermal imaging images of cooperative targets in typical land scenes were acquired as basic thermal imaging scene images. At the same time, a small number of real non-cooperative target samples were acquired as prior data for subsequent optimization.

[0071] 1.2 Construction of a general model for end-to-end radiative transfer: Based on infrared radiative transfer theory, a general model for the entire link from target self-radiation, atmospheric transfer, camera optical system to detector response is constructed to clarify the propagation relationship of radiative energy in the entire imaging process, providing a theoretical basis for subsequent inversion and simulation.

[0072] Step 2: Construction of a hierarchical radiative inversion model

[0073] 2.1 Construction of the emissivity inversion model:

[0074] According to the law of conservation of energy, the total incident energy E on the target surface satisfies: E a +E r +E t =E, where E a To absorb radiant energy, E r To reflect radiated energy, E t It is the energy of transmitted radiation;

[0075] The relationship between the target emissivity ε(λ), reflectivity ρ(λ), transmittance τ(λ) and the radiant energy of each component is as follows:

[0076]

[0077] According to Kirchhoff's laws, under thermal equilibrium conditions, we have: ε(λ) + ρ(λ) + τ(λ) = 1;

[0078] For non-transparent targets in long-wave infrared thermal imaging scenarios, the transmittance τ(λ)≈0, thus obtaining the conversion relationship between emissivity and reflectivity: ε(λ)+ρ(λ)=1, which converts the collected reflectivity spectrum into an emissivity spectrum that characterizes the intrinsic properties of the target.

[0079] 2.2 Construction of the radiation energy inversion model:

[0080] Based on Planck's blackbody radiation law and considering the atmospheric radiative transfer effect, the total radiative energy at the camera's entrance pupil is:

[0081] R λ (Ttarget)=τ atom (λ)·Lg(λ)+L up (λ)

[0082] Among them, the total radiance L of the ground target g (λ)=ε(λ)·B(λ,T target )+[1-ε(λ)]·L down( λ);

[0083] Blackbody spectral radiative exitance The first radiation constant is c1 = 3.7418 × 10⁻⁶. 8 W·μm 4 / m 2 The second radiation constant c2 = 1.4388 × 10 4 μm·K;

[0084] Atmospheric transmittance τ atom (λ), atmospheric upward radiation L up (λ), Atmospheric downward radiation L down (λ) The inversion results of the NASA JPLHyTES public dataset (7.5-12μm, 256 spectral bands) were used as the simulation values;

[0085] Considering the transmittance τ of the camera optical system camera (λ), the final effective radiated energy received by the camera is:

[0086] R λ =τ camera (λ)·R λ (T target )

[0087] 2.3 Construction of the camera DN value inversion model:

[0088] Based on the camera's calibrated response coefficient G and bias coefficient B, a mapping relationship between radiant energy and pixel DN value is established:

[0089] DN=G·R λ ′+B

[0090] Complete the construction of the full-link mapping relationship between the spectrum, radiation and DN values ​​of the cooperative target.

[0091] Step 3: Hierarchical migration simulation of spectral-spatial dual-domain coupling

[0092] 3.1 Pixel-level spectral intrinsic transfer:

[0093] For three types of non-cooperative scenarios—masking, coverage, and interference—corresponding nonlinear spectral transfer models are constructed:

[0094] Shading scenarios: When shaded by trees, adjust the target temperature parameter T. target Spectral migration is completed based on end-to-end mapping; when obscured by smoke, dust, rain, or fog, the atmospheric transmittance τ is set. atom (λ) fluctuates randomly in the range of 0.4 to 0.9, completing the spectral migration;

[0095] Coverage scenarios: Based on a spectral domain nonlinear weighted mixture model, spectral transfer of camouflage nets / vegetation cover is achieved. The model expression is:

[0096] ε mix (λ)=k(λ)·ε target (λ)+(1-k(λ))·ε camou (λ)

[0097] Where the weighting coefficient k(λ) = η·τ camou (λ)·exp(-α·ΔT), where η is the coverage duty cycle (0~1), ΔT is the temperature difference between the target and the camouflage layer, and α is calibrated to 0.008 K through preliminary experiments. -1 ;

[0098] Interference scenarios: Firelight interference introduces the spectrum of hay burning, and false interference introduces the spectrum of plastic false targets. Based on the above hybrid model, spectral transfer is completed to generate spectral data of interference sources.

[0099] 3.2 Target-level Spatial Non-uniform Camouflage Modeling:

[0100] Based on the base DN value image obtained through pixel-level migration, a spatially non-uniform modulation model is constructed:

[0101] DN(x,y)=DN0(x,y)·M(x,y)+N(x,y)

[0102] Camouflage net coverage scenario: M(x,y) adopts a periodic grid function to simulate the mesh cutout structure of the camouflage net, and the grid period is set to 5~20mm according to the specifications of the camouflage net;

[0103] Shaded scene: M(x,y) uses a two-dimensional Gaussian function superposition to simulate the distribution of light spots and shadows in the shade of trees, and the standard deviation of the Gaussian kernel is set to 10~50 pixels;

[0104] Smoke and dust obscuring scene: M(x,y) adopts a random smoothing matrix, and a Gaussian blur kernel is introduced to simulate the scattering effect of smoke and dust. The size of the blur kernel is set to 3~15 pixels.

[0105] N(x, y) uses spatially correlated Gaussian noise, with the standard deviation set to the measured noise level of the camera, to fit the random distortion of the real scene.

[0106] 3.3 Scene-level multi-interference composite coupling modeling:

[0107] For complex scenarios involving multiple camouflage types (such as smoke and dust concealment + camouflage net coverage + firelight interference), a multi-interference coupled radiative transfer model is constructed:

[0108] R total (λ,x,y)=R target (λ,x,y)·τ atm (λ,x,y)·τ camou (λ,x,y)+R interfere (λ,x,y)·τ atm (λ,x,y)+L up (λ)

[0109] A spatial-spectral joint coupling simulation of multiple types of camouflage interference was completed. The generated DN value image was embedded into the basic thermal imaging scene image to obtain the initial simulation sample.

[0110] Step 4: Adversarial Iterative Optimization Closed Loop Based on Physical Consistency Constraints

[0111] 4.1 Physical consistency verification:

[0112] Construct the physical constraint loss function:

[0113] L phys =ω1·L spec +ω2·L temp

[0114] Where L spec The mean square error of the spectra of the simulated samples and the real non-cooperative samples is calculated, while constraining ε(λ) + ρ(λ) = 1; L temp The mean square error of the temperature inversion temperature based on the DN value of the simulation sample and Planck's law is calculated to ensure that the sample conforms to physical laws.

[0115] 4.2 Feature Domain Adversarial Adaptation:

[0116] An adversarial network is constructed, with the generator based on the aforementioned hierarchical transfer simulation framework and the discriminator employing a 5-layer convolutional neural network. The input consists of simulated samples and real non-cooperative samples, and the output is the probability that a sample is a real sample. An adversarial loss function is constructed as follows:

[0117] L adv =E[log(D(real))]+E[log(1-D(sim))]

[0118] Adversarial training minimizes the difference in feature domains between simulated and real samples.

[0119] 4.3 Downstream task feedback optimization:

[0120] Using the YOLOv5 non-cooperative object detection algorithm as the downstream task, a task feedback loss function L is constructed. task , where mAP is the detection algorithm trained on simulated samples and tested on real non-cooperative sample test sets;

[0121] Construct the total loss function:

[0122] L total =L phys +0.5·L adv +0.3·L task

[0123] The gradient descent method is used to iteratively optimize the simulation parameters (coverage duty cycle, atmospheric transmittance, temperature difference, etc.) until the total loss function converges, generating the optimized target simulation sample.

[0124] Step 5: Batch Sample Expansion

[0125] like Figure 2 As shown, based on the optimized hierarchical migration simulation framework, the preset value ranges of parameters such as camouflage type, coverage duty cycle (0.1-0.9), atmospheric transmittance (0.4-0.9), target temperature (260K~320K), and interference source intensity are traversed to generate a large number of multi-scene, multi-parameter thermal imaging simulation samples, thus completing the sample expansion.

[0126] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0127] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style of the specification is merely for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for sample augmentation of thermal imaging images, characterized in that, Includes the following steps: S1: Construct a general model for the full-link radiative transfer of long-wave infrared thermal imaging, obtain basic spectral samples of the cooperative target through spectral acquisition equipment, and obtain basic thermal imaging scene images of the cooperative target through infrared thermal imaging cameras; S2: Establish a hierarchical radiometric inversion model. Based on the basic spectral samples and basic thermal imaging scene images, complete the radiometric inversion, radiometric energy inversion and camera DN value inversion, and construct the full-link mapping relationship of cooperative target spectrum-radiance-DN value. S3: Construct a hierarchical migration simulation framework with spectral-spatial dual-domain coupling. For non-cooperative target scenarios, pixel-level spectral intrinsic migration, target-level spatial non-uniform camouflage modeling, and scene-level multi-interference composite coupling are completed sequentially to generate initial simulation samples. S4: Construct an adversarial iterative optimization closed loop based on physical consistency constraints, perform authenticity verification and iterative optimization of simulation parameters on the initial simulation sample, and generate the optimized target simulation sample; S5: Based on the optimized hierarchical migration simulation framework, batch sample generation with multiple scenarios and multiple camouflage parameters is performed to complete the sample expansion of thermal imaging images.

2. The method for sample augmentation of thermal imaging images according to claim 1, characterized in that, In step S2, the construction process of the hierarchical radiative inversion model is as follows: S21: Construct an emissivity inversion model, based on the law of conservation of energy and Kirchhoff's laws, to establish the conversion relationship between target reflectivity and emissivity; according to the law of conservation of energy, the total incident energy E on the target surface satisfies: E a +E r +E t =E, where E a To absorb radiant energy, E r To reflect radiated energy, E t It is the energy of transmitted radiation; The relationship between the target emissivity ε(λ), reflectivity ρ(λ), transmittance τ(λ) and the radiant energy of each component is as follows: According to Kirchhoff's laws, under thermal equilibrium conditions, we have: ε(λ) + ρ(λ) + τ(λ) = 1; For non-transparent targets in long-wave infrared thermal imaging scenarios, the transmittance τ(λ)≈0, thus the conversion relationship between emissivity and reflectivity is obtained: ε(λ)+ρ(λ)=1; S22: Construct a radiation energy inversion model, based on Planck's blackbody radiation law and combined with atmospheric radiation transfer effect, to establish the mapping relationship between target emissivity and camera entrance pupil radiation energy; The total radiant energy R received at the entrance pupil of the camera λ (T target )for: R λ (T target )=τ atom (λ)·Lg(λ)+L up (l) Where, τ atom (λ) represents atmospheric transmittance, L up (λ) represents the upward atmospheric radiance, L g (λ) represents the total radiance of the ground target, and its expression is: L g (λ)=ε(λ)·B(λ,T target )+[1-e(λ)]·L down (l) In the formula, B(λ,T) target (T) represents temperature T target The blackbody spectral radiant exitance is calculated based on Planck's formula: Where c1 is the first radiation constant, c2 is the second radiation constant, and L down (λ) represents the downward atmospheric radiance; Further considering the transmittance τ of the camera optical system camera (λ) is the effective radiant energy actually received by the camera: R λ ′=τ camera (λ)·R λ (T target ) S23: Construct a camera DN value inversion model, establish the mapping relationship between the radiant energy received by the camera and the DN value of the image pixels, and complete the construction of the full-link mapping.

3. The method for sample augmentation of thermal imaging images according to claim 1, characterized in that, In step S3, the specific process of pixel-level spectral intrinsic transfer is as follows: For the camouflage type of the target non-cooperative scenario, the corresponding spectral data of the camouflage material is obtained, and a nonlinear weighted mixing model in the spectral domain is constructed to complete the intrinsic transfer of the spectrum of the cooperative target. The expression for the spectral domain nonlinear weighted mixing model is: e mix (λ)=k(λ)·e target (λ)+(1-k(λ))·e camou (l) Where, ε mix (λ) represents the mixed spectral radiance after migration, ε target (λ) represents the spectral radiance of the cooperative target, ε camou (λ) represents the spectral emissivity of the camouflage material, and k(λ) is a wavelength-dependent weighting coefficient, expressed as follows: k(λ)=η·τ camou (λ)·exp(-α·ΔT) In the formula, η is the coverage duty cycle of the camouflage material, and τ camou (λ) represents the spectral transmittance of the camouflage material, ΔT represents the temperature difference between the target and the camouflage layer, and α represents the temperature influence coefficient, which was obtained through pre-experiment calibration.

4. The method for sample augmentation of thermal imaging images according to claim 3, characterized in that, In step S3, the specific process of target-level spatial non-uniform camouflage modeling is as follows: Based on the fundamental DN value image obtained from pixel-level spectral intrinsic transfer, a spatial non-uniform modulation model is constructed to fit the spatial radiation distortion caused by the camouflage scene. The expression of the spatial non-uniform modulation model is as follows: DN(x,y)=DN0(x,y)·M(x,y)+N(x,y) Wherein, DN(x,y) is the modulated pixel DN value, DN0(x,y) is the base DN value after pixel-level migration, M(x,y) is the spatial modulation matrix, which is determined by the spatial texture features of the camouflage type, and N(x,y) is the spatial correlation noise term, which is used to fit the random radiation distortion of the real scene.

5. The method for sample augmentation of thermal imaging images according to claim 4, characterized in that, In step S3, the specific process of scene-level multi-interference composite coupling is as follows: For complex non-cooperative scenarios involving multiple types of camouflage interference, a multi-interference coupled radiative transfer model is constructed to complete the space-spectral joint coupling simulation of multiple types of camouflage interference. The expression of the multi-interference coupled radiative transfer model is as follows: R total (λ,x,y)=R target (λ,x,y)·τ atm (λ,x,y)·τ camou (λ,x,y)+R interfere (λ,x,y)·τ atm (λ,x,y)+L up (l) Among them, R total (λ,x,y) represents the total radiant energy at spatial location (x,y), R target (λ,x,y) represents the target's own radiant energy, τ atm (λ,x,y) represents the spatially dependent atmospheric transmittance, τ camou (λ,x,y) represents the spatially correlated transmittance of the camouflage layer, R interfere (λ,x,y) represents the spatial distribution of radiated energy from the interference source.

6. The method for sample augmentation of thermal imaging images according to claim 1, characterized in that, In step S4, the construction process of the adversarial iterative optimization closed loop based on physical consistency constraints first involves constructing a physical consistency verification module, specifically: Construct the physical constraint loss function L phys This is used to verify the physical consistency of infrared radiation in the initial simulation samples. The expression for the physical constraint loss function is: L phys =ω1·L spec +ω2·L temp Among them, L spec The loss represents the spectral consistency, which is the mean square error between the simulated sample spectral curve and the real non-cooperative sample spectral curve, while simultaneously constraining Kirchhoff's law ε(λ) + ρ(λ) = 1; L temp The temperature radiation uniformity loss is used to constrain the temperature value corresponding to the DN value of the simulation sample to conform to Planck's blackbody radiation law; ω1 and ω2 are preset weighting coefficients.

7. The method for sample augmentation of thermal imaging images according to claim 6, characterized in that, The construction process of the adversarial iterative optimization closed loop based on physical consistency constraints also includes constructing a feature domain adversarial adaptation module, specifically: An adversarial network structure is constructed, which includes a generator and a discriminator. The generator is the hierarchical transfer simulation framework used to generate simulation samples. The discriminator is a binary classification network used to distinguish whether the input sample is a real non-cooperative sample or a simulation sample. Constructing the adversarial loss function L adv Its expression is: L adv =E[log(D(real))]+E[log(1-D(sim))] Where D is the output probability of the discriminator, real is the real non-cooperative sample, sim is the simulated sample, and E is the mathematical expectation.

8. The method for sample augmentation of thermal imaging images according to claim 7, characterized in that, The construction process of the adversarial iterative optimization closed loop based on physical consistency constraints also includes constructing a downstream task feedback optimization module, specifically: Using non-cooperative object detection algorithms as the downstream task, a task feedback loss function L is constructed. task The task feedback loss function is the detection accuracy loss of the detection algorithm after training on simulated samples on the real non-cooperative sample test set; Construct the total optimization loss function L total Its expression is: L total =L phys +λ1·L adv +λ2·L task Wherein, λ1 and λ2 are preset balance weight coefficients; The simulation parameters of the hierarchical migration simulation framework are updated by iteratively optimizing and minimizing the total optimization loss function until the loss function converges, thus completing the optimization of the simulation samples.

9. The method for sample augmentation of thermal imaging images according to claim 1, characterized in that, In step S1, the spectral acquisition device is a Fourier transform infrared spectrometer, and the basic spectral sample acquired is the target reflectance spectral data in the 7.5μm-12μm long-wave infrared band; the infrared thermal imaging camera is a long-wave infrared hyperspectral thermal imaging camera, and the basic thermal imaging scene image acquired is the long-wave infrared hyperspectral thermal imaging image of the cooperative target.

10. The method for sample augmentation of thermal imaging images according to claim 1, characterized in that, In step S5, the batch sample generation specifically involves: based on the optimized hierarchical migration simulation framework, traversing the preset value ranges of camouflage type, camouflage parameters, environmental parameters, and interference parameters to generate a large batch of multi-scene thermal imaging simulation samples, thereby completing sample expansion; the camouflage type includes at least one of occlusion type, coverage type, and interference type, and the camouflage parameters include at least one of coverage duty cycle, atmospheric transmittance, target temperature, and interference source intensity.

Images