Method, system, medium and product for occluded target three-dimensional reconstruction of infrared images

Abstract

The application discloses an occluded target three-dimensional reconstruction method, system, medium and product of an infrared image, and the method comprises the following steps: obtaining a multi-view aligned pseudo-ground-truth depth map by using a depth estimation network after pre-processing of thermal infrared images of an occluded target under multi-view; initializing a three-dimensional Gaussian scene by using the thermal infrared images and the pseudo-ground-truth depth map; and optimizing each three-dimensional Gaussian cell in the current three-dimensional Gaussian scene according to a preset loss function to realize detail enhancement, atmospheric artifact elimination and scene robust enhancement on the current three-dimensional Gaussian scene, so that a reconstructed three-dimensional Gaussian scene is obtained. The application aims to comprehensively utilize the significant advantages of thermal infrared imaging on an occluded target, three-dimensional Gaussian scene representation, atmospheric transmission modeling of physical constraints and robust view constraints, and realize high-fidelity and high-precision three-dimensional reconstruction of the occluded target.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology in the field of image processing, specifically to a method, system, medium, and product for three-dimensional reconstruction of occluded targets in infrared images. Background Technology

[0002] Thermal infrared imaging detects the thermal radiation of objects in the mid-to-far infrared band to image targets in a scene. It does not rely on visible light illumination and offers advantages such as all-weather operation (day and night) and the ability to penetrate smoke, vegetation gaps, and camouflage to a certain extent. Compared to visible light images, thermal infrared images better reflect the temperature distribution differences between targets and the background, making them useful for discovering critical targets that are partially obscured, poorly lit, or camouflaged. Therefore, they have received widespread attention in fields such as security monitoring, border patrol, and emergency rescue. However, limited by sensor technology, optical system aperture, and operating wavelength, existing thermal infrared imaging equipment generally suffers from low spatial resolution, lack of detailed texture, and low signal-to-noise ratio, making it difficult to directly meet the needs for high-precision 3D reconstruction of obscured targets.

[0003] For 3D reconstruction in complex scenes, existing technologies are mainly based on visible light images. In recent years, methods such as neural radiation fields and 3D Gaussian sputtering have emerged for scene modeling and novel perspective synthesis. These methods typically assume high-resolution input images with minimal occlusion and rich texture. However, when targets are severely occluded by trees, railings, camouflage nets, or are only clearly visible in the thermal infrared domain, traditional visible light-based 3D reconstruction methods often struggle to obtain reliable structural and textural information. Furthermore, research on thermal infrared images largely focuses on 2D target detection, segmentation, and enhancement. A few works attempt to improve infrared image clarity using super-resolution networks, but these are mostly limited to image-domain enhancement and not tightly coupled with 3D scene representation, failing to optimize the quality of occluded targets holistically from the perspectives of 3D geometry and multi-view consistency. On the other hand, thermal infrared imaging is susceptible to atmospheric turbulence, aerosol scattering, and background thermal radiation, often resulting in floating artifacts, contour distortion, and decreased contrast in long-distance imaging. Existing dehazing, deturbulence removal, or atmospheric correction methods are mostly based on visible light imaging assumptions, failing to fully consider the radiative transfer characteristics of the thermal infrared band. When directly applied to infrared occlusion scenarios, they easily mistake atmospheric artifacts for real structures or over-smoothing leads to further loss of target details. Furthermore, in occluded target 3D reconstruction tasks, because the target is partially or completely occluded in most views, the effective supervisory information available for constraining reconstruction is very limited. Traditional reconstruction frameworks relying on dense multi-view supervision are prone to problems such as incomplete structure, insufficient detail, and sensitivity to noise and artifacts, lacking robust enhancement mechanisms for occlusion and insufficient supervision scenarios. Moreover, existing occluded target processing methods based on thermal infrared imaging mostly focus only on 2D image enhancement or simple target detection, failing to systematically consider the overall chain of "low-resolution infrared imaging—atmospheric artifact interference—occlusion leading to insufficient supervision." They lack a unified framework that balances detail enhancement, atmospheric artifact elimination, and 3D reconstruction robustness, resulting in low 3D reconstruction accuracy and insufficient detail in existing technologies that rely solely on visible light imaging, under strong occlusion and complex atmospheric conditions, and with low-quality thermal infrared image input. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, system, medium and product for three-dimensional reconstruction of occluded targets from infrared images, in view of the above-mentioned problems in the prior art. The present invention aims to comprehensively utilize the significant advantages of thermal infrared imaging for occluded targets, three-dimensional Gaussian scene representation, and atmospheric transmission modeling and robust viewpoint constraints with physical constraints to achieve high-fidelity and high-precision three-dimensional reconstruction of occluded targets.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A method for 3D reconstruction of an occluded target from an infrared image, comprising the following steps: S101, after preprocessing the thermal infrared images of the occluded target from multiple perspectives, a depth estimation network is used to obtain a pseudo-true depth map aligned from multiple perspectives. S102, Initialize the 3D Gaussian scene using thermal infrared images and pseudo-true depth maps; S103, optimize each 3D Gaussian primitive in the current 3D Gaussian scene according to the preset loss function to achieve detail enhancement, atmospheric artifact elimination and scene robustness enhancement of the current 3D Gaussian scene, thereby obtaining the reconstructed 3D Gaussian scene.

[0006] Optionally, the initialization of the 3D Gaussian scene using thermal infrared images and pseudo-true depth maps in step S102 includes: S201, for each coordinate (u,v) in the pseudo-true depth map, back-projection is performed using the camera intrinsic parameters of the thermal infrared camera to obtain the three-dimensional point X in the camera coordinate system. i,u,v ; S202, each three-dimensional point X is respectively... i,u,v As a corresponding 3D Gaussian primitive in a 3D Gaussian scene mean Three-dimensional Gaussian elements are initialized and calculated based on thermal infrared images and pseudo-true depth maps. covariance matrix Opacity and color feature vector Thus, an initial three-dimensional Gaussian scene composed of various three-dimensional Gaussian primitives is obtained: ; in, This is the initial 3D Gaussian scene. Let k be the k-th three-dimensional Gaussian element.

[0007] Optionally, the covariance matrix Opacity and color feature vector The expression for the computation function is: ; ; ; in, It is an orthogonal basis matrix. for transpose, The standard deviation of the gradient of the thermal infrared image. The standard deviation of the gradient of the pseudo-true depth map. Clamp( is the truncation function) The expression ,0,1) truncates the input to [0,1]. For minimum opacity, This is the depth penalty coefficient. and These are the gradients of the thermal infrared image and the pseudo-true depth map, respectively. Coordinates in thermal infrared image The radiation value at that location coordinates The weight, coordinates The neighborhood of; where: ; ; ; in, For spatial weighting scale, and These represent the confidence weights for the thermal infrared image and the pseudo-true depth map, respectively. Coordinates in thermal infrared image gradient at, Coordinates in the pseudo-true depth map The gradient at that point.

[0008] Optionally, in step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement of the current 3D Gaussian scene by enhancing details, eliminating atmospheric artifacts, and constraining pose, the function expression of the preset loss function is: ; in, The default loss function is... To enhance the loss in detail, To eliminate losses from atmospheric artifacts, For pose constraint loss, To enhance the balance of losses for details, This is the balance coefficient for pose constraint loss.

[0009] Optionally, in step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement for detail enhancement, atmospheric artifact removal, and pose constraint of the current 3D Gaussian scene, detail enhancement of the current 3D Gaussian scene includes: S301, Perform super-resolution processing on the pre-trained two-dimensional super-resolution model of the pre-processed thermal infrared images from various perspectives to obtain the reference thermal infrared image; S302, for the current 3D Gaussian scene, high-resolution rendered images and depth maps at various viewpoints are obtained using the volume rendering operator according to the following formula: ; in, and These are the high-resolution rendered image and depth map of viewpoint i obtained from the current 3D Gaussian scene using the volume rendering operator. For volume rendering operators, The intrinsic and extrinsic parameters of the thermal infrared camera at viewpoint i. For camera internal parameters, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. S303 downsamples the high-resolution rendered images from various viewpoints to obtain low-resolution rendered results: ; in, This is the low-resolution rendering result after reconstruction. For downsampling operators; S304, construct the detail enhancement loss as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ; ; ; ; in, and All are balance parameters. For low resolution loss, For high resolution loss, For deep consistency loss, For the number of viewpoints, The image is a preprocessed thermal infrared image at viewpoint i. The low-resolution rendering result is obtained by downsampling at viewpoint i. The image is a high-resolution rendered image obtained from the reconstructed 3D Gaussian scene at viewpoint i. The reference thermal infrared image obtained through super-resolution processing. For the pseudo-ground depth map at viewpoint i, This is the depth map of view i obtained through the volume rendering operator.

[0010] Optionally, in step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement for detail enhancement, atmospheric artifact removal, and pose constraint of the current 3D Gaussian scene, atmospheric artifact removal of the current 3D Gaussian scene includes: S401, for each line of sight in the rendering process of a 3D Gaussian primitive, samples several 3D positions based on the interaction area with the current 3D Gaussian scene. They are then normalized to a fixed range; all thermal infrared images are arranged in chronological order to obtain a relative time index. , three-dimensional position and relative time index Concatenate into input vector ; S402, [determines] the various three-dimensional positions corresponding input vector The corresponding atmospheric transmittance is predicted using a pre-trained sinusoidal activated transmission network. ; ; in, For sinusoidal activation of the transmission network, The prediction results are for a sinusoidal activated transmission network. For three-dimensional position Located in relative time index The atmospheric transmittance, wherein the sinusoidal activated transmission network is a multilayer perceptron, and the functional expression of each hidden layer in the multilayer perceptron is: ; in, and The first and The output features of the hidden layer and The first The weight matrix and bias vector of the hidden layer. It is a sinusoidal activation function; S403 embeds atmospheric transmittance multiplicatively into each three-dimensional Gaussian unit to obtain color feature increments. : ; in, For three-dimensional Gaussian elements The cumulative transmittance, For three-dimensional Gaussian elements Opacity For three-dimensional Gaussian elements Atmospheric transmittance at the center of the , For three-dimensional Gaussian elements The mean, For three-dimensional Gaussian elements Relative time index, For three-dimensional Gaussian elements The color feature vector makes the final pixel color for: ; S404, atmospheric artifact elimination loss is constructed based on atmospheric transmittance as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ; in, This is the spatial gradient operator.

[0011] Optionally, in step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement of the current 3D Gaussian scene by enhancing details, eliminating atmospheric artifacts, and constraining pose, the robust scene enhancement of the current 3D Gaussian scene by constraining pose includes: S501, camera pose of the thermal infrared camera from various viewpoints i Using it as a baseline, rotational and translational perturbations are applied in its vicinity to generate a set of perturbation viewpoints. ,in, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. and Let be the rotation matrix and translation vector of the j-th perturbation viewpoint under viewpoint i, respectively. The number of perturbation viewpoints; S502, based on the current 3D Gaussian scene, uses volume rendering operators to perturb the view set. The following volume rendering produces a pseudo-view: ; in, Let i be the pseudo-view of the j-th perturbed viewpoint under viewpoint i. For volume rendering operators, For the current 3D Gaussian scene, For three-dimensional position Located in relative time index Atmospheric transmittance. For camera internal parameters; S503 uses a pre-trained geometric network to predict camera poses from each pseudo-view: ; in, Let i be the camera pose at the j-th perturbed viewpoint under viewpoint i. and Let be the predicted rotation matrix and the predicted translation vector for the j-th perturbation viewpoint under viewpoint i, respectively. It is a geometric network; S504, construct the pose constraint loss as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ; in, For the number of viewpoints, for The inverse matrix.

[0012] The present invention also provides a three-dimensional reconstruction system for an occluded target from an infrared image, comprising a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the three-dimensional reconstruction method for the occluded target from the infrared image.

[0013] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute, via a processor, a method for three-dimensional reconstruction of an occluded target from an infrared image.

[0014] The present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute, via a processor, a method for three-dimensional reconstruction of an occluded target from the infrared image.

[0015] Compared with existing technologies, this invention mainly achieves the following beneficial effects: 1) This invention combines low-resolution thermal infrared occluded target images with pseudo-ground depth for 3D Gaussian scene initialization, and introduces high-resolution supervision provided by a pre-trained super-resolution model to explicitly constrain geometric and detail consistency in 3D space, thereby reconstructing the high-resolution geometric structure and detailed texture of occluded targets from low-resolution thermal infrared data. 2) This invention proposes an atmospheric artifact elimination module based on a sinusoidal activated transmission network to address the atmospheric propagation characteristics of thermal infrared imaging. It models atmospheric attenuation in the form of a logarithmic transmittance field and modulates the 3D Gaussian radiation contribution in a multiplicative manner in the rendering equation, effectively eliminating atmospheric floating artifacts and long-distance thermal turbulence interference, and improving the clarity and physical credibility of target boundaries and scene structures near the occluded area. 3) This invention utilizes a perturbation perspective and a pre-trained visual geometry network to construct a scene robust enhancement module. In the absence of sufficient real-view supervision, especially in severely occluded areas, it introduces additional geometric priors to the 3D scene through pose constraints, significantly alleviating the ambiguity caused by low-texture infrared images and improving the ability to extrapolate new perspectives. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0017] Figure 2 This is a visual evaluation chart of the method in this embodiment and five existing 3D reconstruction methods on the TI-NSD dataset, where... Figure 2 (a) is the visual evaluation chart of 3DGS; Figure 2 (b) is the visual evaluation chart of Thermal3D-GS; Figure 2 (c) is the visual evaluation chart of Scaffold-GS; Figure 2 (d) is the visual evaluation chart of Mip-Splatting; Figure 2 (e) is the visual evaluation chart of SRGS; Figure 2 (f) is the visual evaluation diagram of this embodiment; Figure 2 (g) is the ground truth label image (GT). Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0019] like Figure 1 As shown, the three-dimensional reconstruction method for occluded targets from infrared images in this embodiment includes the following steps: S101, after preprocessing the thermal infrared images of the occluded target from multiple perspectives, a depth estimation network is used to obtain a pseudo-true depth map aligned from multiple perspectives. S102, Initialize the 3D Gaussian scene using thermal infrared images and pseudo-true depth maps; S103, optimize each 3D Gaussian primitive in the current 3D Gaussian scene according to the preset loss function to achieve detail enhancement, atmospheric artifact elimination and scene robustness enhancement of the current 3D Gaussian scene, thereby obtaining the reconstructed 3D Gaussian scene.

[0020] Using the reconstructed 3D Gaussian scene, high-resolution rendered images of the occluded target can be obtained from various specified viewpoints. The high-resolution rendered image obtained from the reconstructed 3D Gaussian scene at any viewpoint i can be denoted as... It should be noted that in this embodiment, "low resolution" refers to the resolution of the original multi-view thermal infrared image, and "high resolution" refers to the resolution of the reference thermal infrared image obtained through super-resolution processing, or the high-resolution rendered image obtained from the 3D Gaussian scene rendering. and the reference thermal infrared image obtained through super-resolution processing They have the same resolution.

[0021] In step S101, when the preprocessed thermal infrared images of the occluded target from multiple perspectives are used to obtain a pseudo-ground depth map aligned from multiple perspectives using a depth estimation network, the number of perspectives N can be selected as needed. Generally, thermal infrared images from no less than 10 perspectives can be obtained, which can be expressed as follows: ,in The image is a thermal infrared image at viewpoint i. Thermal infrared image preprocessing includes obtaining the intrinsic and extrinsic parameters for each viewpoint through known calibration procedures or based on the multi-view motion geometry recovery (SfM) method. ,in For camera internal parameters, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively.

[0022] When using a depth estimation network to obtain pseudo-ground truth depth maps aligned to multiple viewpoints, a preprocessed thermal infrared image is input into the depth estimation network to obtain pseudo-ground truth depth maps aligned to each viewpoint. Let the depth estimation network be denoted as... Then the perspective pseudo-true depth map Represented as: ; In the specific implementation, The depth anything2 model, employing an existing encoder-decoder structure, takes a single thermal infrared image as input and outputs a depth map at the same resolution as the input. By introducing edge-preserving loss during training, the depth map exhibits better gradient characteristics at the boundaries of targets and occlusions, which is beneficial for subsequent fine-grained 3D Gaussian modeling.

[0023] Step S102, which initializes the 3D Gaussian scene using thermal infrared images and pseudo-true depth maps, includes: S201, for each coordinate in the pseudo-true depth map Three-dimensional points in the camera coordinate system are obtained by back-projection using the camera intrinsic parameters of a thermal infrared camera. , can be represented as: ; in, For back projection operators, From the perspective pseudo-true depth map In position Depth value at that location, For camera internal parameters, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. S202, each three-dimensional point is respectively... As a corresponding 3D Gaussian primitive in a 3D Gaussian scene mean Three-dimensional Gaussian elements are initialized and calculated based on thermal infrared images and pseudo-true depth maps. covariance matrix Opacity and color feature vector Thus, an initial three-dimensional Gaussian scene composed of various three-dimensional Gaussian primitives is obtained: ; in, This is the initial 3D Gaussian scene. Let k be the k-th three-dimensional Gaussian element.

[0024] Wherein, the covariance matrix Opacity and color feature vector The expression for the computation function is: ; ; ; in, It is an orthogonal basis matrix. for transpose, The standard deviation of the gradient of the thermal infrared image. The standard deviation of the gradient of the pseudo-true depth map. Clamp( is the truncation function) The expression ,0,1) truncates the input to [0,1]. For minimum opacity, This is the depth penalty coefficient. and These are the gradients of the thermal infrared image and the pseudo-true depth map, respectively. Coordinates in thermal infrared image The radiation value at that location coordinates The weight, coordinates The neighborhood of; where: ; ; ; in, For spatial weighting scale, and These are the confidence weights for the thermal infrared image and the pseudo-true depth map, respectively (each can be set to 0.5). Coordinates in thermal infrared image gradient at, Coordinates in the pseudo-true depth map The gradient at that point.

[0025] In step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement of the current 3D Gaussian scene by enhancing details, eliminating atmospheric artifacts, and constraining pose, the function expression of the preset loss function is as follows: ; in, The default loss function is... To enhance the loss in detail, To eliminate losses from atmospheric artifacts, For pose constraint loss, To enhance the balance of losses for details, This is the balance coefficient for pose constraint loss. Through... By performing backpropagation and iterative optimization, the parameters of the 3D Gaussian scene and the transmission network can be updated simultaneously, enabling the 3D Gaussian scene to be realized, enhancing details, eliminating atmospheric artifacts, and exhibiting higher geometric stability and reconstruction robustness in unseen viewpoints and occluded areas.

[0026] In this embodiment, detail enhancement of the current 3D Gaussian scene is achieved through a detail enhancement module. This module takes multi-view low-resolution infrared images of occluded targets acquired by a thermal infrared imaging device as input, utilizes pseudo-ground depth map generation and 3D Gaussian scene initialization to construct a 3D Gaussian scene with relatively coarse structure and detail, and combines this with high-resolution supervision signals obtained from a pre-trained super-resolution model for detail enhancement loss. Specifically, in step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement of the current 3D Gaussian scene by enhancing details, eliminating atmospheric artifacts, and constraining pose, the enhancement of details in the current 3D Gaussian scene includes: S301, the pre-trained two-dimensional super-resolution model of the pre-processed thermal infrared images from various viewpoints is subjected to super-resolution processing to obtain a reference thermal infrared image; to improve the detail resolution of the thermal infrared image, a pre-trained two-dimensional super-resolution model is introduced to generate a high-resolution supervision signal. Let the two-dimensional super-resolution model be denoted as... ( Then, for each preprocessed thermal infrared image, we have: ; in, In order to be with the first High-resolution reference thermal infrared images corresponding to each viewpoint are used as high-resolution pseudo-truth value supervision for the 3D rendering results. This refers to the number of viewpoints. It should be noted that the 2D super-resolution model is a well-known existing model; the appropriate network model can be selected as needed. S302, in the current 3D Gaussian scene Based on this, the current 3D Gaussian scene High-resolution rendered images and depth maps for each viewpoint are obtained using the volume rendering operator according to the following formula: ; in, and These are the high-resolution rendered image and depth map of viewpoint i obtained from the current 3D Gaussian scene using the volume rendering operator. For volume rendering operators, The intrinsic and extrinsic parameters of the thermal infrared camera at viewpoint i. For camera internal parameters, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. S303 downsamples the high-resolution rendered images from various viewpoints to obtain low-resolution rendered results: ; in, This is the low-resolution rendering result after reconstruction. For downsampling operators; S304, construct the detail enhancement loss as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ; ; ; ; in, and All are balance parameters. For low resolution loss, For high resolution loss, For deep consistency loss, For the number of viewpoints, The image is a preprocessed thermal infrared image at viewpoint i. The low-resolution rendering result is obtained by downsampling at viewpoint i. The image is a high-resolution rendered image obtained from the reconstructed 3D Gaussian scene at viewpoint i. The reference thermal infrared image obtained through super-resolution processing. For the pseudo-ground depth map at viewpoint i, This is the depth map at viewpoint i obtained through the volume rendering operator. The detail enhancement loss is minimized. For the current 3D Gaussian scene In By jointly optimizing parameters such as scene details and structural refinement, we can achieve scene detail enhancement and structural refinement.

[0027] In this embodiment, atmospheric artifact removal is achieved through an atmospheric artifact removal module. This module is used to explicitly model the atmospheric transmission characteristics during thermal infrared imaging. For atmospheric floating artifacts caused by atmospheric turbulence, aerosol scattering, and other factors during infrared imaging, a sinusoidal activated transmission network is constructed to eliminate floating artifacts and long-distance blur introduced by atmospheric propagation. Specifically, in step S103, the atmospheric artifact removal module optimizes each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve detail enhancement, atmospheric artifact removal, and scene robustness enhancement with pose constraints for the current 3D Gaussian scene. Atmospheric artifact removal for the current 3D Gaussian scene includes: S401, for each line of sight in the rendering process of a 3D Gaussian primitive, samples several 3D positions based on the interaction area with the current 3D Gaussian scene. They are then normalized to a fixed range; all thermal infrared images are arranged in chronological order to obtain a relative time index. , three-dimensional position and relative time index Concatenate into input vector ; S402, [determines] the various three-dimensional positions corresponding input vector The corresponding atmospheric transmittance is predicted using a pre-trained sinusoidal activated transmission network. ; ; in, To activate the transmission network with sinusoidal activation, The prediction results are from a sinusoidal activated transmission network. For three-dimensional position Located in relative time index Atmospheric transmittance. The sinusoidal activation transmission network is a multilayer perceptron (MLP), and the function expression for each hidden layer in the MLP is: ; in, and The first and The output features of the hidden layer and The first The weight matrix and bias vector of the hidden layer. The sinusoidal activation function can improve the ability to express high-frequency changes in the network and is more suitable for fitting local rapid changes caused by thermal turbulence and floating particles. S403 embeds atmospheric transmittance multiplicatively into each three-dimensional Gaussian unit to obtain color feature increments. : ; in, For three-dimensional Gaussian elements The cumulative transmittance, For three-dimensional Gaussian elements Opacity For three-dimensional Gaussian elements Atmospheric transmittance at the center of the , For three-dimensional Gaussian elements The mean, For three-dimensional Gaussian elements Relative time index, For three-dimensional Gaussian elements The color feature vector makes the final pixel color for: ; S404, atmospheric artifact elimination loss is constructed based on atmospheric transmittance as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ; in, This is the spatial gradient operator. The enhancement of detail loss is minimized. and atmospheric artifact elimination loss It can simultaneously optimize 3D Gaussian scenes and transmission networks, resulting in renderings with clearer boundaries and fewer atmospheric artifacts around occluded targets.

[0028] In this embodiment, scene robustness enhancement is achieved through a scene robustness enhancement module, which is mainly used to enhance the stability of 3D structures under conditions of severe occlusion and insufficient supervision information. Addressing the issues of limited visible viewpoints and insufficient supervision for occluded targets, the scene robustness enhancement module generates perturbation viewpoints around known training viewpoints, renders pseudo-views using the current 3D scene, and introduces a pre-trained visual geometry network to estimate the pose of the pseudo-views. Pose constraints enhance the geometric robustness of the 3D scene in unseen viewpoints and occluded regions. Specifically, in step S103, when the scene robustness enhancement module optimizes each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve scene robustness enhancement of the current 3D Gaussian scene through detail enhancement, atmospheric artifact removal, and pose constraints, the scene robustness enhancement of the current 3D Gaussian scene through pose constraints includes: S501, camera pose of the thermal infrared camera from various viewpoints i Using it as a baseline, rotational and translational perturbations are applied in its vicinity to generate a set of perturbation viewpoints. ,in, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. and Let be the rotation matrix and translation vector of the j-th perturbation viewpoint under viewpoint i, respectively. The number of perturbation viewpoints; the perturbation range can be adaptively set according to the scene scale and camera field of view, ensuring that the perturbation viewpoints remain within the local visual neighborhood of the target; the number of perturbation viewpoints. Values ​​can be selected as needed; S502, based on the current 3D Gaussian scene, uses volume rendering operators to perturb the view set. The following volume rendering produces a pseudo-view: ; in, Let i be the pseudo-view of the j-th perturbed viewpoint under viewpoint i. For volume rendering operators, For the current 3D Gaussian scene, For three-dimensional position Located in relative time index Atmospheric transmittance. This is the camera's intrinsic parameter; optionally, the corresponding rendering depth or normal vector map can be output simultaneously to enhance geometric constraints. S503 uses a pre-trained geometric network to predict camera poses from each pseudo-view: ; in, Let i be the camera pose at the j-th perturbed viewpoint under viewpoint i. and Let be the predicted rotation matrix and the predicted translation vector for the j-th perturbation viewpoint under viewpoint i, respectively. It is a geometric network; S504, To measure the difference between the predicted pose and the perturbation ground truth pose, this embodiment employs dual constraints of rotation and translation to construct the pose constraint loss as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ; in, For the number of viewpoints, for The inverse matrix, log( ) denotes a logarithmic mapping that maps a rotation matrix to a Lie algebra space.

[0029] To verify the effectiveness of the 3D reconstruction method for occluded targets using infrared images in this embodiment, a validation experiment was conducted using the TI-NSD dataset. The TI-NSD dataset contains 20 different scenes, each with over 300 infrared images and a spatial resolution of 120*180. COLMAP was used as the motion geometry recovery model to obtain the intrinsic and extrinsic parameters of the camera for each scene, and 12.5% ​​of the data was used as test data. To accurately evaluate the performance of the proposed method and more intuitively compare the advantages and disadvantages of different methods, three widely used objective evaluation metrics were selected: Structural Similarity Index (SSIM, higher is better), Peak Signal-to-Noise Ratio (PSNR, higher is better), and Learned Perceptual Patch Similarity (LPIPS, lower is better). This embodiment is implemented using a deep learning framework, based on the Adam optimizer to optimize the Gaussian sphere parameters. The initial learning rate was set to 0.0001, and a cosine annealing strategy was used to decay the learning rate. To reconstruct a more detailed target, the output rendering resolution was set to four times the original input resolution. In the experiment, it was compared with five existing 3D reconstruction methods, namely 3DGS, Thermal3D-GS, Scaffold-GS, Mip-Splatting and SRGS. The final results are shown in Table 1, with the best results highlighted in bold.

[0030] Table 1: Objective performance metrics of the method in this embodiment and five existing 3D reconstruction methods on the TI-NSD dataset.

[0031] As shown in Table 1, the method in this embodiment achieves better results than the five existing 3D reconstruction methods in terms of structural similarity index, peak signal-to-noise ratio, and learning-aware image patch similarity. This indicates that the method in this embodiment can reconstruct occluded targets more effectively.

[0032] In addition to objective evaluation indicators, in order to more intuitively understand the effects of different spectral super-resolution imaging methods, this embodiment also performs visual evaluation, which displays the occluded target. Figure 2 This is a visual evaluation image of the method in this embodiment and five existing 3D reconstruction methods on the TI-NSD dataset, where the rectangular boxes represent the locations of occluded targets. Figure 2 (a) is the visual evaluation chart of 3DGS; Figure 2 (b) is the visual evaluation chart of Thermal3D-GS; Figure 2 (c) is the visual evaluation chart of Scaffold-GS; Figure 2 (d) is the visual evaluation chart of Mip-Splatting; Figure 2 (e) is the visual evaluation chart of SRGS; Figure 2 (f) is the visual evaluation diagram of this embodiment; Figure 2 (g) is the ground truth (GT) image. From Figure 2 As can be seen, the method in this embodiment achieves the best reconstruction visual effect on distant, occluded targets, indicating that the method can reconstruct occluded targets very well.

[0033] In summary, the 3D reconstruction method for occluded targets based on infrared images in this embodiment utilizes detail enhancement to construct and optimize a 3D Gaussian scene under low-resolution thermal infrared images and pseudo-ground value depth constraints, thereby obtaining fine geometric structures and texture details of the occluded targets. By explicitly modeling atmospheric transmission during thermal infrared imaging through a sinusoidal activated transmission network in atmospheric artifact elimination, floating artifacts and thermal turbulence interference are adaptively suppressed during the 3D volume rendering stage. Furthermore, the scene robustness enhancement module utilizes perturbation perspectives and pose constraints provided by a pre-trained geometric network to improve the stability and generalization ability of 3D reconstruction under conditions of severe occlusion and insufficient supervised perspectives. This embodiment is the first to organically combine 3D Gaussian scene representation of thermal infrared occluded targets, physically consistent atmospheric transmission modeling, and robust geometric constraints based on perturbation perspectives. Under the premise of relying only on low-quality thermal infrared occlusion input, it achieves 3D reconstruction results with rich detail information, complete structure, and good robustness to atmosphere and occlusion, demonstrating high engineering application value and promising prospects for widespread application.

[0034] Those skilled in the art will understand that the technical solutions provided by this invention can take the form of methods, systems, or computer program products. For example, this invention can provide a three-dimensional reconstruction system for occluded targets in infrared images, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the three-dimensional reconstruction method for occluded targets in infrared images; this invention can provide a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the three-dimensional reconstruction method for occluded targets in infrared images via a processor; this invention can provide a computer program product including a computer program or instructions programmed or configured to execute the three-dimensional reconstruction method for occluded targets in infrared images via a processor. Furthermore, this invention can also take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this invention can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0035] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for three-dimensional reconstruction of an occluded target from an infrared image, characterized in that, Includes the following steps: S101, after preprocessing the thermal infrared images of the occluded target from multiple perspectives, a depth estimation network is used to obtain a pseudo-true depth map aligned from multiple perspectives. S102, Initialize the 3D Gaussian scene using thermal infrared images and pseudo-true depth maps; S103, optimize each 3D Gaussian primitive in the current 3D Gaussian scene according to the preset loss function to achieve detail enhancement, atmospheric artifact elimination and scene robustness enhancement of the current 3D Gaussian scene, thereby obtaining the reconstructed 3D Gaussian scene. In step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement for detail enhancement, atmospheric artifact removal, and pose constraint of the current 3D Gaussian scene, detail enhancement of the current 3D Gaussian scene includes: S301, Perform super-resolution processing on the pre-trained two-dimensional super-resolution model of the pre-processed thermal infrared images from various perspectives to obtain the reference thermal infrared image; S302, for the current 3D Gaussian scene, high-resolution rendered images and depth maps at various viewpoints are obtained using the volume rendering operator according to the following formula: ； in, and These are the high-resolution rendered image and depth map of viewpoint i obtained from the current 3D Gaussian scene using the volume rendering operator. For volume rendering operators, For the current 3D Gaussian scene, For three-dimensional position Located in relative time index Atmospheric transmittance. The intrinsic and extrinsic parameters of the thermal infrared camera at viewpoint i. For camera internal parameters, These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. S303 downsamples the high-resolution rendered images from various viewpoints to obtain low-resolution rendered results: ； in, This is the low-resolution rendering result after reconstruction. For downsampling operators; S304, construct the detail enhancement loss as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ； ； ； ； in, and All are balance parameters. For low resolution loss, For high resolution loss, For deep consistency loss, For the number of viewpoints, The image is a preprocessed thermal infrared image at viewpoint i. The low-resolution rendering result is obtained by downsampling at viewpoint i. The image is a high-resolution rendered image obtained from the reconstructed 3D Gaussian scene at viewpoint i. The reference thermal infrared image obtained through super-resolution processing. For the pseudo-ground depth map at viewpoint i, This is the depth map of view i obtained through the volume rendering operator; In step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement of the current 3D Gaussian scene by enhancing details, eliminating atmospheric artifacts, and constraining pose, the atmospheric artifact elimination of the current 3D Gaussian scene includes: S401, for each line of sight in the rendering process of a 3D Gaussian primitive, samples several 3D positions based on the interaction area with the current 3D Gaussian scene. They are then normalized to a fixed range; all thermal infrared images are arranged in chronological order to obtain a relative time index. , three-dimensional position and relative time index Concatenate into input vector ; S402, [determines] the various three-dimensional positions corresponding input vector The corresponding atmospheric transmittance is predicted using a pre-trained sinusoidal activated transmission network. ； ； in, To activate the transmission network with sinusoidal activation, The prediction result is for a sinusoidal activation transmission network, which is a multilayer perceptron, and the function expression for each hidden layer in the multilayer perceptron is: ； in, and The first and The output features of the hidden layer and The first The weight matrix and bias vector of the hidden layer. It is a sinusoidal activation function; S403 embeds atmospheric transmittance multiplicatively into each three-dimensional Gaussian unit to obtain color feature increments. : ； in, For three-dimensional Gaussian elements The cumulative transmittance, For three-dimensional Gaussian elements Opacity For three-dimensional Gaussian elements Atmospheric transmittance at the center of the , For three-dimensional Gaussian elements The mean, For three-dimensional Gaussian elements Relative time index, For three-dimensional Gaussian elements The color feature vector makes the final pixel color for: ； S404, atmospheric artifact elimination loss is constructed based on atmospheric transmittance as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ； in, For spatial gradient operators; In step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement for detail enhancement, atmospheric artifact removal, and pose constraint of the current 3D Gaussian scene, the robust scene enhancement for pose constraint of the current 3D Gaussian scene includes: S501, camera pose of the thermal infrared camera from various viewpoints i Using it as a baseline, rotational and translational perturbations are applied in its vicinity to generate a set of perturbation viewpoints. ,in These are the rotation matrix and translation vector in the camera's extrinsic parameters, respectively. and Let be the rotation matrix and translation vector of the j-th perturbation viewpoint under viewpoint i, respectively. The number of perturbation viewpoints; S502, based on the current 3D Gaussian scene, uses volume rendering operators to perturb the view set. The following volume rendering produces a pseudo-view: ； in, Let i be the pseudo-view of the j-th perturbed viewpoint under viewpoint i. For volume rendering operators, For camera internal parameters; S503 uses a pre-trained geometric network to predict camera poses from each pseudo-view: ； in, Let i be the camera pose at the j-th perturbed viewpoint under viewpoint i. and Let be the predicted rotation matrix and the predicted translation vector for the j-th perturbation viewpoint under viewpoint i, respectively. It is a geometric network; S504, construct the pose constraint loss as shown in the following formula. To be used for optimization of individual 3D Gaussian primitives in the current 3D Gaussian scene: ； in, For the number of viewpoints, for The inverse matrix.

2. The method for three-dimensional reconstruction of occluded targets from infrared images according to claim 1, characterized in that, Step S102, which initializes the 3D Gaussian scene using thermal infrared images and pseudo-true depth maps, includes: S201, for each coordinate (u,v) in the pseudo-true depth map, back-projection is performed using the camera intrinsic parameters of the thermal infrared camera to obtain the three-dimensional point X in the camera coordinate system. i,u,v ; S202, each three-dimensional point X is respectively... i,u,v As a corresponding 3D Gaussian primitive in a 3D Gaussian scene mean Three-dimensional Gaussian elements are initialized and calculated based on thermal infrared images and pseudo-true depth maps. covariance matrix Opacity and color feature vector Thus, an initial three-dimensional Gaussian scene composed of various three-dimensional Gaussian primitives is obtained: ； in, This is the initial 3D Gaussian scene. Let k be the k-th three-dimensional Gaussian element.

3. The method for three-dimensional reconstruction of occluded targets from infrared images according to claim 2, characterized in that, The covariance matrix Opacity and color feature vector The expression for the computation function is: ； ； ； in, It is an orthogonal basis matrix. for transpose, The standard deviation of the gradient of the thermal infrared image. The standard deviation of the gradient of the pseudo-true depth map. Clamp( is the truncation function) The expression ,0,1) truncates the input to [0,1]. For minimum opacity, This is the depth penalty coefficient. and These are the gradients of the thermal infrared image and the pseudo-true depth map, respectively. Coordinates in thermal infrared image The radiation value at that location coordinates The weight, coordinates The neighborhood of; where: ； ； ； in, For spatial weighting scale, and These represent the confidence weights for the thermal infrared image and the pseudo-true depth map, respectively. Coordinates in thermal infrared image gradient at, Coordinates in the pseudo-true depth map The gradient at that point.

4. The method for three-dimensional reconstruction of occluded targets from infrared images according to claim 1, characterized in that, In step S103, when optimizing each 3D Gaussian primitive in the current 3D Gaussian scene according to a preset loss function to achieve robust scene enhancement of the current 3D Gaussian scene by enhancing details, eliminating atmospheric artifacts, and constraining pose, the function expression of the preset loss function is as follows: ； in, The default loss function is... To enhance the loss in detail, To eliminate losses from atmospheric artifacts, For pose constraint loss, To enhance the balance of losses for details, This is the balance coefficient for pose constraint loss.

5. A three-dimensional reconstruction system for an occluded target from an infrared image, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to perform the method for three-dimensional reconstruction of an occluded target from an infrared image as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the method for three-dimensional reconstruction of an occluded target from an infrared image as described in any one of claims 1 to 4.

7. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the method for three-dimensional reconstruction of an occluded target from an infrared image as described in any one of claims 1 to 4.

Images