Multi-view military image fusion three-dimensional reconstruction and precision calibration method and platform

Abstract

The application relates to the technical field of data processing, and particularly discloses a multi-view military image fusion three-dimensional reconstruction and precision calibration method and platform, which comprises the following steps: acquiring multi-view remote sensing images of a target area, generating an initial three-dimensional model based on the multi-view remote sensing images; acquiring a digital twin benchmark model of the target area which is constructed in advance, and simulating to generate predicted images under each shooting angle based on the digital twin benchmark model; comparing the remote sensing images and the predicted images step by step, calculating imaging deviations, and decomposing the imaging deviations into scene change components and observation interference components; determining the intersection distribution of multi-view observation rays in three-dimensional space based on the spatial poses of each shooting angle, and jointly screening the target area based on the scene change components and the intersection distribution to obtain an effective change area; and determining a final three-dimensional model based on the deviation features of the effective change area, the initial three-dimensional model and the digital twin benchmark model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and platform for multi-view military image fusion, 3D reconstruction, and accuracy calibration. Background Technology

[0002] Currently, using multi-view remote sensing images for 3D reconstruction of target areas has become an important tool in fields such as military reconnaissance, battlefield environmental awareness, and terrain mapping. By acquiring remote sensing images from different angles, the 3D geometric information of the target area can be recovered, providing data support for operational planning and situational awareness.

[0003] In existing technologies, 3D reconstruction of multi-view remote sensing images typically involves extracting image feature points and performing cross-view matching. Based on the matching results and imaging geometric parameters, a 3D point cloud is reconstructed, ultimately generating a 3D model. However, the accuracy of these methods is limited by image quality and imaging conditions. Remote sensing images are inevitably affected by atmospheric disturbances, sensor noise, and illumination changes during acquisition, leading to geometric deviations in the 3D model. When real-world changes occur in the target area, existing methods struggle to distinguish between these real-world changes and observational interference, easily misjudging observational noise as scene changes or overlooking real-world changes. When the predicted 3D model deviates from the baseline model in local areas, existing methods typically employ a globally uniform strategy, lacking a mechanism for differentiated decision-making based on deviation characteristics.

[0004] Therefore, this invention proposes a method and platform for multi-view military image fusion 3D reconstruction and accuracy calibration. Summary of the Invention

[0005] This invention provides a method and platform for multi-view military image fusion, 3D reconstruction, and accuracy calibration to solve the aforementioned problems in the prior art.

[0006] This invention provides a method for multi-view military image fusion, 3D reconstruction, and accuracy calibration, comprising: Acquire multi-view remote sensing images of the target area and generate an initial 3D model based on the multi-view remote sensing images; Obtain a pre-constructed digital twin benchmark model of the target area, and generate predicted images from various shooting angles based on the digital twin benchmark model. The remote sensing images and the predicted images are compared step by step to calculate the imaging deviation, and the imaging deviation is decomposed into scene change components and observation interference components. Based on the spatial pose of each shooting angle, the intersection distribution of multi-view observation rays in three-dimensional space is determined, and the target area is jointly screened based on the scene change components and intersection distribution to obtain the effective change area. Based on the deviation characteristics of the effective change region, the initial 3D model, and the digital twin baseline model, the final 3D model is determined; the deviation characteristics are used to indicate the magnitude and spatial distribution range of the deviation in the effective change region, including the amount and range of deviations in real scene changes.

[0007] Based on the above technical means, this application has at least the following beneficial effects: by decomposing imaging deviation into scene change components and observation interference components, it is possible to distinguish between real scene changes and observation noise, and avoid misjudging observation interference as scene changes; by introducing the intersection distribution information of multi-view observation rays, and using the multi-view cross-confirmation mechanism to screen effective change areas, the reliability of change detection is improved; by fusing the initial three-dimensional model and the digital twin benchmark model based on deviation characteristics, refined reconstruction is achieved, and the accuracy of the final three-dimensional model is improved.

[0008] In one possible implementation, based on the spatial pose of each shooting angle, the intersection distribution of multi-view observation rays in three-dimensional space is determined, and the target area is jointly filtered based on the scene change components and the intersection distribution to obtain the effective change area, including: Obtain the spatial pose parameters corresponding to each shooting angle; Based on spatial pose parameters, the propagation path of imaging rays in three-dimensional space for each frame of remote sensing image is tracked, and the intersection area of ​​each imaging ray in three-dimensional space and the intersection density of imaging rays in each intersection area are determined. The intersection density is used to characterize how many imaging rays from different shooting angles cover the same spatial location. Regions in the scene change components that exceed a preset change threshold and whose intersection density exceeds a preset density threshold are identified as valid change regions.

[0009] Based on the above technical means, by tracking the propagation path of imaging light and calculating the intersection density, the degree of cross-confirmation of multi-view observations is quantified. Combined with the threshold screening of scene change components, false detections and low-confidence change areas from a single viewpoint are effectively eliminated.

[0010] In one possible implementation, the final 3D model is determined based on the deviation characteristics of the effective variation region, the initial 3D model, and the digital twin baseline model, including: Based on the deviation characteristics, a fusion strategy is determined between the initial 3D model and the digital twin baseline model in the effective change region; the fusion strategy is used to indicate whether to perform corrective fusion or tolerance fusion on the effective change region. Based on the fusion strategy, the geometric data of the initial 3D model in the effective change region is fused with the geometric data of the digital twin baseline model to obtain the fused geometric data of the effective change region. The final 3D model is generated based on the fused geometric data.

[0011] Based on the above technical means, the fusion strategy is adaptively selected by deviation characteristics, and different fusion methods are used in different regions to avoid the accuracy loss caused by one-size-fits-all processing.

[0012] In one possible implementation, based on deviation characteristics, a fusion strategy is determined between the initial 3D model and the digital twin baseline model in the effective variation region, including: Based on the deviation characteristics, the first credibility score of the initial 3D model and the second credibility score of the digital twin benchmark model are calculated respectively. If the first confidence score is greater than or equal to the second confidence score, then the fusion strategy is determined to be tolerance fusion; tolerance fusion means preserving the original geometric data of the initial 3D model in the effective change region; If the second confidence score is greater than the first confidence score, then the fusion strategy is determined to be corrective fusion; corrective fusion refers to using the digital twin benchmark model as a reference to correct the geometric deviation of the effective change area in the initial three-dimensional model before fusion.

[0013] Based on the above technical means, by quantitatively comparing the credibility scores of the two models, the automatic decision-making of the fusion strategy is realized, ensuring that the optimal fusion method is adopted for each effective change area.

[0014] In one possible implementation, the deviation features also include the spatial distribution density of scene change components within the effective change region; based on the deviation features, a first credibility score for the initial 3D model and a second credibility score for the digital twin benchmark model are calculated, including: Based on the deviation amount and deviation range of the effective variation region, the first confidence score of the initial 3D model is determined; the first confidence score is negatively correlated with the deviation amount and deviation range, and is used to characterize the reliability of the initial 3D model in the effective variation region; Based on the spatial distribution density of the effective variation region, a second credibility score is determined for the digital twin benchmark model. The second credibility score is positively correlated with the spatial distribution density and is used to characterize the reliability of the digital twin benchmark model in the effective variation region.

[0015] Based on the above technical means, the credibility is quantified through three dimensions: deviation amount, deviation range, and spatial distribution density. The smaller the deviation, the more credible the observation model; the denser the deviation, the more credible the benchmark model. This achieves multi-dimensional objective quantification of credibility scoring.

[0016] In one possible implementation, a first confidence score for the initial 3D model is determined based on the amount and range of deviation within the effective variation region, including: Based on the deviation amount in the effective variation area, the first deviation influence coefficient is determined; the first deviation influence coefficient is negatively correlated with the deviation amount. The second deviation influence coefficient is determined based on the deviation range of the effective change area; the second deviation influence coefficient is negatively correlated with the deviation range. The first bias influence coefficient and the second bias influence coefficient are weighted and summed to determine the first credibility score of the initial 3D model.

[0017] In one possible implementation, based on a fusion strategy, the geometric data of the initial 3D model in the effective change region is fused with the geometric data of the digital twin baseline model to obtain the fused geometric data of the effective change region, including: When the fusion strategy is tolerance fusion, the original geometric data of the initial 3D model in the effective change region is used as the fused geometric data; When the fusion strategy is corrective fusion, the first set of geometric coordinates of the effective change region in the initial 3D model is extracted, and the second set of geometric coordinates of the corresponding position of the effective change region in the digital twin benchmark model is extracted. The spatial transformation relationship is determined based on the first set of geometric coordinates and the second set of geometric coordinates, and the geometric coordinates of the effective transformation region are transformed using the spatial transformation relationship; The transformed geometric data is used as the fused geometric data.

[0018] Based on the above technical means, the precise alignment of geometric coordinates is achieved through spatial transformation relationships. The geometric deviation of the observation model is corrected with reference to the benchmark model, ensuring the geometric consistency of the correction fusion.

[0019] In one possible implementation, the final 3D model is generated based on the fused geometric data, including: The original geometric data of the corresponding region in the initial 3D model is replaced by the fused geometric data of the effective change region to obtain the updated 3D model. The updated 3D model is output as the final 3D model.

[0020] In one possible implementation, before outputting the updated 3D model as the final 3D model, the following steps are also included: The updated 3D model is subjected to boundary smoothing to eliminate the geometric discontinuities between the effective changed region and the unchanged region in the initial 3D model.

[0021] Based on the above technical means, geometric seams between fused and non-fused regions are eliminated through boundary smoothing, thereby improving the visual quality and geometric consistency of the final 3D model.

[0022] This application embodiment also provides a multi-view military image fusion 3D reconstruction and accuracy calibration platform, which includes: The system comprises the following modules: an acquisition module for acquiring multi-view remote sensing images of the target area and generating an initial 3D model based on these images; a simulation module for acquiring a pre-constructed digital twin reference model of the target area and generating predicted images from various shooting angles based on this model; a calculation module for comparing the remote sensing images and predicted images step-by-step, calculating imaging deviations, and decomposing these deviations into scene change components and observation interference components; a filtering module for determining the intersection distribution of multi-view observation rays in 3D space based on the spatial pose of each shooting angle, and jointly filtering the target area based on the scene change components and the intersection distribution to obtain effective change areas; and a fusion module for determining the final 3D model based on the deviation characteristics of the effective change areas, the initial 3D model, and the digital twin reference model. The deviation characteristics indicate the magnitude and spatial distribution range of the deviations within the effective change areas, including the amount and range of deviations from real scene changes.

[0023] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written documents of this application.

[0024] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a method for multi-view military image fusion, 3D reconstruction, and accuracy calibration in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another method for multi-view military image fusion, 3D reconstruction, and accuracy calibration in an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a multi-view military image fusion 3D reconstruction and accuracy calibration method platform in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0027] It should be noted that in the embodiments of this application, terms such as "first" and "second" are used only for distinguishing descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Example 1

[0028] This embodiment provides a method for multi-view military image fusion, 3D reconstruction, and accuracy calibration. Figure 1 This is a flowchart illustrating the method of this embodiment, which specifically includes the following steps: S101. Acquire multi-view remote sensing images of the target area and generate an initial 3D model based on the multi-view remote sensing images.

[0029] In one possible implementation, multi-view remote sensing images of the target military area are acquired. After acquiring the multi-view remote sensing images, the system preprocesses each frame of the remote sensing images. After preprocessing, based on the spatial pose parameters and imaging payload parameters carried by each frame of the remote sensing images, three-dimensional reconstruction is performed using multi-view geometric principles (such as structure for motion recovery (SFM) or multi-view stereo (MVS) algorithm) to generate an initial three-dimensional model of the target area.

[0030] Multi-view remote sensing images refer to a sequence of remote sensing images obtained by imaging the same target area from different spatial locations and shooting angles. Each frame of the image carries corresponding spatial pose parameters and imaging payload parameters.

[0031] Spatial pose parameters are used to describe the position coordinates and orientation of the imaging platform (such as satellites, drones, etc.) in three-dimensional space at the moment of shooting, while imaging payload parameters are used to describe the imaging characteristics of the sensor, such as focal length, field of view, and resolution.

[0032] The above preprocessing includes at least one of radiation correction processing, atmospheric correction processing, and geometric correction processing.

[0033] Radiometric correction is used to eliminate radiometric distortion caused by differences in sensor response, while atmospheric correction is used to eliminate the effects of atmospheric scattering and absorption on image grayscale values.

[0034] Geometric correction is used to register each frame of remote sensing imagery to a unified geographic coordinate system.

[0035] The initial 3D model is represented in the form of a dense 3D point cloud or triangular mesh, recording the geometric coordinates and texture information of the target area surface.

[0036] The initial 3D model described above is represented in the form of a dense 3D point cloud or a triangular mesh, recording the geometric coordinates and texture information of the target area's surface. Specifically, the dense 3D point cloud consists of a large number of points carrying 3D spatial coordinates (X, Y, Z) and color information (R, G, B), which can finely depict the surface morphology of features and terrain in the target area; the triangular mesh is a continuous surface model generated on the basis of the point cloud through a triangulation algorithm, consisting of a large number of interconnected triangular patches, each containing vertex coordinates and texture mapping coordinates, which can more intuitively present the geometric structure and texture details of the target area.

[0037] Both of the above representations can completely record the spatial geometric information and surface texture information of the target area, providing a data foundation for subsequent step-by-step comparison and deviation calculation with the digital twin benchmark model.

[0038] S102. Obtain a pre-constructed digital twin benchmark model of the target area, and generate predicted images from various shooting angles based on the digital twin benchmark model.

[0039] The aforementioned digital twin benchmark model is a pre-constructed high-precision 3D reference model of the target area, recording the scene geometry and surface texture information of the target area at historical or benchmark times. This benchmark model is constructed by fusing multi-source historical data.

[0040] The aforementioned multi-source historical data includes, but is not limited to, one or more of the following: historical remote sensing images, lidar point cloud data, synthetic aperture radar data, manual mapping data, and geographic information system data.

[0041] The baseline model exists in the form of a 3D mesh model or a high-density point cloud, with an accuracy higher than or equal to the expected accuracy of the initial 3D model, ensuring its effectiveness as a reference benchmark for geometric deviation correction.

[0042] Each 3D point or mesh vertex in the baseline model carries precise spatial coordinate information, and the surface triangular patches or mesh units carry texture mapping relationships, which can completely record the 3D spatial morphology and surface appearance attributes of various features and terrains in the target area.

[0043] In one possible implementation, the specific process of generating predicted images at various shooting angles based on a digital twin benchmark model is as follows: First, for each shooting angle, the spatial pose parameters and imaging payload parameters of the imaging platform corresponding to that angle are obtained. The spatial pose parameters include the shooting position coordinates and shooting attitude angle, while the imaging payload parameters include sensor type, imaging resolution, spectral response range, number of bands, and pixel size. These parameters are consistent with those used when acquiring the remote sensing image in S101 to ensure that the simulated predicted image and the measured remote sensing image are comparable in imaging geometry and image attributes.

[0044] Secondly, a projection transformation relationship from 3D world coordinates to 2D image coordinates is established based on spatial pose parameters. This projection transformation relationship is determined based on the imaging geometry model of the imaging platform. For different types of remote sensing imaging platforms, corresponding imaging geometry models can be adopted: for frame-type cameras, the projection transformation relationship is established based on the collinearity equation; for pushbroom sensors, the projection transformation relationship is established based on the row center projection model; for synthetic aperture radar, the projection transformation relationship is established based on the range-Doppler model. Through the above projection transformation relationship, an accurate mapping from any 3D spatial position in the reference model to pixel coordinates on the predicted image can be achieved.

[0045] Then, using the aforementioned projection transformation relationship, each 3D point or mesh vertex in the digital twin baseline model is projected onto the 2D image plane one by one, determining the corresponding pixel position of each 3D point on the predicted image. For 3D points in the baseline model that are not covered by any viewpoint projection (such as surfaces facing away from the imaging platform or empty areas in the model), the corresponding positions in the predicted image are marked as no-data areas.

[0046] After determining the projection position, for each pixel location, the surface attribute information of the reference model at that location is obtained, including surface material reflectance characteristics, texture grayscale values, or spectral reflectance. Based on the imaging payload parameters, the response characteristics of the imaging sensor are simulated, and the surface attribute information is converted into the grayscale value or spectral value of the predicted image at that pixel location. For regions in the reference model that contain color texture information, the texture color is mapped to the grayscale value of the corresponding band based on the spectral response range of the imaging payload. For regions in the reference model that contain only geometric information but no texture information, grayscale values ​​can be assigned based on geometric features (such as surface normals, slope, etc.), or texture information can be filled using neighborhood interpolation methods.

[0047] Optionally, the radiative transfer effect and atmospheric influence of the imaging scene are also considered during the generation of the predicted image. Specifically, based on atmospheric parameters at the time of imaging (visibility, water vapor content, aerosol optical thickness, etc.), the propagation process of solar radiation along the "atmosphere-surface-sensor" path is simulated using an atmospheric radiative transfer model, including atmospheric absorption, atmospheric scattering, surface reflection, and sensor response. The radiative transfer effect is superimposed into the grayscale value calculation of the predicted image, making the simulated predicted image closer to the measured remote sensing image in terms of radiative characteristics, thereby reducing imaging deviations caused by radiative differences.

[0048] Through the above steps, a simulated predicted image is generated for each shooting angle. This predicted image reflects the scene appearance that should be observed at that shooting angle based on the digital twin benchmark model. The predicted image and the measured remote sensing image at the corresponding angle have the same resolution, field of view, spectral bands, and imaging geometry, thus ensuring that subsequent step-by-step comparisons can be performed under consistent image attributes.

[0049] In one possible implementation, a pre-constructed digital twin reference model of the target area is obtained. After obtaining the digital twin reference model, the scene geometry and imaging load parameters of each imaging platform are simulated and deduced through the imaging physical model based on the digital twin reference model to generate simulated predicted images at each shooting angle.

[0050] The aforementioned imaging payload parameters are a set of data describing the technical characteristics and operating status of the imaging platform's sensors during image acquisition. These parameters include sensor type, imaging resolution, spectral response range, number of bands, pixel size, focal length, and radiometric calibration coefficients. Different types of imaging platforms have different imaging payload parameters: optical remote sensing platforms correspond to the payload parameters of visible light / multispectral / hyperspectral sensors, while synthetic aperture radar platforms correspond to parameters such as wavelength, polarization, and incident angle of radar sensors. The imaging payload parameters determine the resolution, band composition, and radiometric characteristics of the simulated and predicted image, and are crucial inputs for generating predicted images consistent with the attributes of the measured remote sensing images.

[0051] The aforementioned imaging physics model is a comprehensive mathematical model describing the entire process of electromagnetic waves reflected or emitted from a target surface, transmitted through the atmosphere, and imaged by sensors to form a remote sensing image. This model encompasses the radiative transmission process of electromagnetic waves interacting with the Earth's surface, the propagation and attenuation of electromagnetic waves in the atmospheric medium, and the imaging response process of the sensor converting the received radiation signal into image grayscale values. Specifically, the imaging physics model includes the following three components: First, the radiative transfer stage. This stage describes the process by which solar radiation reaches the Earth's surface and is reflected or emitted by the target surface. For optical remote sensing, the model calculates the downward radiance of sunlight reaching the surface after atmospheric scattering and absorption. Combining this with the target surface's reflectivity (such as the bidirectional reflectance distribution function), the model calculates the upward radiance reflected by the target and re-entering the sensor through the atmosphere. For radar remote sensing, the model calculates the echo signal strength of the electromagnetic waves emitted by the radar, after being scattered by the target surface and returning to the receiving antenna. This stage involves parameters such as atmospheric aerosol optical thickness, water vapor content, and atmospheric profile, which can be set based on the atmospheric conditions at the time of imaging.

[0052] Secondly, the geometric projection stage. This stage describes how ground features in three-dimensional space are projected onto a two-dimensional image plane through the geometric and optical relationships of the imaging system. For frame-type optical cameras, the collinearity equation describes the geometric collinearity relationship between spatial points, the camera projection center, and the image point; for pushbroom sensors, the projection relationship is established row by row based on the row center projection model; for synthetic aperture radar, the slant range projection relationship is described based on the range-Doppler model. This stage calculates the precise projection position of each three-dimensional point in the digital twin reference model onto the predicted image based on the spatial pose parameters of the imaging platform (shooting position coordinates and shooting attitude angle).

[0053] Third, the sensor response stage. This stage describes how the radiation signal entering the sensor's entrance pupil undergoes photoelectric conversion, analog-to-digital conversion, and other processes to ultimately output as a digital image grayscale value. Based on parameters such as the sensor's radiometric calibration coefficient, quantum efficiency, dark current noise, and quantization bit depth, the model converts the received radiance value at the entrance pupil into the grayscale value or digital quantization value of the corresponding pixel in the predicted image, ensuring that the simulated predicted image matches the measured remote sensing image in terms of radiometric characteristics.

[0054] Through the three stages of the aforementioned imaging physical model, the scene geometry and material properties of the digital twin baseline model are transformed into two-dimensional simulation prediction images at various shooting angles. The predicted images and the measured remote sensing images at the corresponding angles have consistent resolution, bands, and geometric structures, ensuring that subsequent step-by-step comparisons can be performed on the basis of the same imaging properties.

[0055] S103. Compare the remote sensing image with the predicted image step by step, calculate the imaging deviation, and decompose the imaging deviation into scene change component and observation interference component.

[0056] In one possible implementation, the acquired actual remote sensing image is compared step by step with the simulated prediction image generated in S102, and the imaging deviation between the two is calculated.

[0057] Imaging bias reflects the difference between actual observations and baseline model predictions. This difference may stem from changes in the actual scene of the target area itself, or from interference factors introduced during the observation process, such as changes in atmospheric conditions, sensor noise, and differences in imaging angles.

[0058] The aforementioned scene change component represents the portion of the imaging bias contributed by the actual physical changes occurring within the target area itself. It characterizes the actual geometric or physical changes in the target area's features or terrain at the measured time relative to the historical time corresponding to the baseline model. This component exhibits a continuous spatial distribution; real scene changes typically affect a certain spatial area rather than being isolated, scattered points. The deviation directions of adjacent pixels are consistent, and their amplitudes are similar. The scene change component originates from actual scene changes occurring within the target area, including but not limited to: natural terrain evolution (such as river course changes, landslides, and ground subsidence), the addition, removal, or morphological changes of man-made facilities (such as new construction, demolition, and expansion of buildings, and road construction), changes in vegetation cover (such as deforestation, vegetation growth, and withering), and temporary changes in the battlefield environment (such as fortification construction, craters, and obstacle placement). This component is the core focus of subsequent fusion processing.

[0059] The aforementioned observation interference component represents the portion of imaging deviation contributed by changes in observation conditions or non-ideal characteristics of the imaging system. It characterizes non-realistic changes caused by factors such as atmospheric conditions, sensor noise, and imaging geometric differences. This component exhibits a discrete distribution or low amplitude in space, typically manifesting as isolated scattered noise or randomly distributed weak deviations, lacking consistency and continuity between adjacent pixels. The observation interference component originates from various interference factors during the observation process, including but not limited to: changes in atmospheric conditions (such as differences in radiative transfer caused by atmospheric scattering, absorption, and turbulence), sensor noise (such as photoelectric conversion noise, dark current noise, quantization errors, etc.), differences in imaging angles (such as ground object occlusion, shadow changes, and differences in specular reflection directionality caused by different shooting angles), radiometric calibration errors, and image registration errors. This component does not represent a real change in the target area and is not addressed in post-processing; it is separated and ignored as background noise.

[0060] This step obtains the imaging deviation through step-by-step comparison, and then further decomposes it into scene change components and observation interference components, providing a data foundation for subsequent targeted processing only on the real change areas.

[0061] Specifically, the process of comparison at each level is as follows: The remote sensing image and the simulated prediction image are compared pixel by pixel at multiple resolution levels to obtain the initial deviation data. The initial deviation data is then processed based on spatial distribution characteristics: deviation components with continuous spatial distribution and amplitude exceeding the preset interference threshold are extracted as scene change components, and deviation components with discrete spatial distribution or amplitude below the preset interference threshold are extracted as observation interference components.

[0062] The aforementioned preset interference threshold is an amplitude limit used to distinguish between real scene changes and observation interference. Its physical meaning is as follows: when the imaging deviation amplitude at a certain pixel location is lower than the threshold, the deviation is considered to be mainly caused by random factors in the observation process (such as sensor noise, atmospheric jitter, radiometric calibration error, etc.) and is insufficient to constitute a reliable indication of real scene changes; when the deviation amplitude exceeds the threshold, the deviation is considered to possibly reflect the real geometric or physical changes of the target area.

[0063] In one possible implementation, the preset interference threshold is determined based on the noise characteristics of the imaging payload. Specifically, the signal-to-noise ratio parameters of the imaging payload under different illumination conditions and different gain settings are obtained. Combined with the radiometric resolution of the imaging payload (i.e., the minimum resolvable radiometric difference corresponding to the quantization bits), the equivalent noise amplitude of the imaging payload under typical operating conditions is calculated. This equivalent noise amplitude is multiplied by a preset safety factor (usually 2 to 3) to obtain the preset interference threshold.

[0064] Optionally, the preset interference threshold is also dynamically adjusted based on the imaging conditions of the target area. For remote sensing images acquired under high signal-to-noise ratio and high-quality imaging conditions (such as clear weather and high solar altitude angle), the preset interference threshold can be set to a lower value to fully capture subtle scene changes due to the high quality of the observation data and low noise level. For remote sensing images acquired under low signal-to-noise ratio and adverse imaging conditions (such as cloudy weather, low solar altitude angle, and strong atmospheric turbulence), the preset interference threshold is set to a higher value to avoid misjudging a large amount of noise as scene changes due to the high noise level of the observation data.

[0065] Optionally, the preset interference threshold is also adaptively set based on the scene type of the target area. For areas with simple textures and uniform grayscale (such as deserts and water surfaces), since background changes are sensitive to noise, the preset interference threshold is set to a lower value to maintain the sensitivity to detect small changes. For areas with complex textures and drastic grayscale changes (such as urban building areas and densely vegetated areas), since the background itself has large grayscale changes, the preset interference threshold is set to a higher value to avoid misjudging texture details as scene changes. In this way, the preset interference threshold can adaptively match different imaging conditions and scene characteristics, ensuring that the bias decomposition results have good robustness and accuracy in various scenes.

[0066] The aforementioned scene change components correspond to real scene changes occurring in the target area (such as new buildings, terrain changes, vegetation changes, etc.), and are characterized by continuous spatial distribution and large amplitude.

[0067] The aforementioned observation interference components correspond to pseudo-variables caused by factors such as illumination changes, atmospheric disturbances, sensor noise, and registration errors. They are characterized by discrete spatial distribution, small amplitude, and randomness.

[0068] S104. Based on the spatial pose of each shooting angle, determine the intersection distribution of the multi-view observation rays in three-dimensional space, and jointly screen the target area based on the scene change components and intersection distribution to obtain the effective change area.

[0069] In one possible implementation, based on the spatial pose parameters corresponding to each frame of remote sensing images, the propagation path of the imaging light rays of each pixel in each frame image in three-dimensional space is tracked, and the number of imaging light rays passing through each spatial location from different perspectives is counted as the intersection density of that location. Areas where the scene change component exceeds a preset change threshold and the intersection density exceeds a preset density threshold are identified as valid change areas. That is, only areas that simultaneously meet the two conditions of "existence of real scene change" and "sufficient cross-confirmation of multi-view observations" are included in the subsequent fusion processing, thereby eliminating false changes caused by observation interference and unreliable areas caused by insufficient observation coverage.

[0070] S105. Based on the deviation characteristics of the effective change area, the initial three-dimensional model and the digital twin benchmark model, determine the final three-dimensional model.

[0071] In one possible implementation, based on the deviation amount, deviation range, and spatial distribution density of the effective variation region, a first confidence score of the initial 3D model and a second confidence score of the digital twin benchmark model are calculated respectively. If the first confidence score is greater than or equal to the second confidence score, the original geometric data of the initial 3D model in that region is retained (tolerance fusion). If the second confidence score is greater than the first confidence score, the geometric deviation of the initial 3D model in that region is corrected and then fused with the benchmark model as a reference (correction fusion). The fused geometric data replaces the original geometric data of the corresponding region in the initial 3D model, and the final 3D model is obtained after boundary smoothing.

[0072] The above S104 specifically includes the following steps: S104-1. Obtain the spatial pose parameters corresponding to each shooting angle.

[0073] Specifically, for each acquired remote sensing image frame, the spatial pose parameters at the time of image capture are read from its corresponding metadata file.

[0074] The spatial pose parameters include position parameters and attitude parameters: position parameters include the coordinates (longitude, latitude, altitude) of the imaging platform in three-dimensional space at the time of shooting, and attitude parameters include the three-axis rotation angles (yaw angle, pitch angle, roll angle) of the imaging platform relative to the geographic coordinate system at the time of shooting.

[0075] The aforementioned spatial pose parameters are synchronously recorded at the moment of imaging by the positioning system (such as a global navigation satellite system receiver) and attitude measurement system (such as an inertial measurement unit) mounted on the remote sensing platform, and are transmitted to the ground receiving station or stored in the image metadata along with the remote sensing image data.

[0076] It should be understood that for remote sensing platforms that do not have the ability to record pose in real time, the spatial pose parameters corresponding to each frame of the image can be obtained through post-processing or joint adjustment. This application does not make any specific limitations on this.

[0077] S104-2. Based on spatial pose parameters, track the propagation path of the imaging rays in each frame of remote sensing image in three-dimensional space, and determine the intersection area of ​​each imaging ray in three-dimensional space and the intersection density of imaging rays in each intersection area.

[0078] The aforementioned intersection density is used to characterize how many imaging rays from different shooting angles cover the same spatial location. The higher the intersection density, the more independent viewpoints cross-confirm the spatial location, and the more reliable the change detection results are.

[0079] In one possible implementation, for each pixel in each frame of remote sensing image, based on the spatial pose parameters corresponding to that frame image (including the coordinates of the shooting center position and the shooting attitude angle), combined with the imaging payload parameters (including the sensor pixel size and lens focal length), the direction vector of the imaging ray corresponding to that pixel in three-dimensional space is calculated; taking the shooting center position as the starting point and the direction vector as the propagation direction, the ray equation of the imaging ray of that pixel in three-dimensional space is constructed, and the complete propagation path of the imaging ray from the shooting center through the spatial range of the target area is traced.

[0080] After obtaining the imaging ray propagation paths of each pixel at all shooting angles, the three-dimensional space of the target area is discretized into a regular three-dimensional voxel grid. The number of imaging rays passing through each voxel grid is counted, and this number is determined as the intersection density at that voxel grid location. After traversing all voxel grids, the intersection density distribution data at each spatial location of the target area is obtained.

[0081] For any given spatial location, a higher intersection density indicates that the location is observed by more independent perspectives, and the more reliable the change detection results based on that location are.

[0082] S104-3. Regions in the scene change components that exceed the preset change threshold and whose intersection density exceeds the preset density threshold are identified as valid change regions.

[0083] The aforementioned preset change threshold is used to determine whether the scene change component has reached the minimum amplitude standard required for fusion processing. This threshold is used to distinguish between real scene changes and minor deviations caused by imaging condition fluctuations, image registration errors, etc.

[0084] In one possible implementation, the preset change threshold is determined based on the terrain type of the target area. Different terrain types have different sensitivity requirements for change detection: in flat terrain (such as plains and deserts), the surface structure is simple, and small changes can be effectively identified, so the preset change threshold is set to a low value; in hilly terrain, the undulating surface leads to more shadows and occlusions, so to eliminate misjudgments caused by terrain undulations, the preset change threshold is set to a medium value; in urban built-up areas, buildings are dense and the types of land features are complex, so to avoid misjudging building shadows or partial occlusions as scene changes, the preset change threshold is set to a higher value.

[0085] Optionally, the preset change threshold is also dynamically adjusted based on the imaging payload parameters. For high-resolution remote sensing images, due to their rich detail and sensitivity to minute changes, the preset change threshold is set to a lower value to fully capture change information; for low-resolution remote sensing images, due to their limited detail and relatively significant noise, the preset change threshold is set to a higher value to avoid misjudgment due to noise.

[0086] Optionally, the preset change threshold is also adaptively set based on the historical statistical values ​​of scene changes in the target area.

[0087] Specifically, statistical analysis is performed on the change detection results of historical remote sensing images of the target area over multiple periods. The mean μ and standard deviation σ of the scene change components are calculated, and the preset change threshold is set to μ + k·σ (k is a preset coefficient, usually taken as 2~3). This allows the threshold to be dynamically adjusted based on the regional change characteristics, avoiding missed or false detections of change areas due to a fixed threshold.

[0088] Through joint screening, the effective change area simultaneously meets two conditions: first, there is indeed a significant scene change, and second, the change is confirmed by multiple perspectives.

[0089] S105. Based on the deviation characteristics of the effective change area, the initial three-dimensional model and the digital twin benchmark model, determine the final three-dimensional model.

[0090] In one possible implementation, based on the deviation characteristics (including deviation amount, deviation range, and spatial distribution density) of the effective variation area, the credibility scores of the initial 3D model and the digital twin benchmark model in that area are calculated respectively. If the credibility score of the initial 3D model is higher, the original geometric data of that area is retained (tolerance fusion). If the credibility score of the benchmark model is higher, the initial 3D model is fused after geometric deviation correction with reference to the benchmark model (correction fusion). The fused geometric data replaces the original geometric data of the corresponding area in the initial 3D model, and the final 3D model is obtained after boundary smoothing.

[0091] The above S105 specifically includes the following steps: S105-1. Based on deviation characteristics, determine the fusion strategy between the initial 3D model and the digital twin baseline model in the effective change region.

[0092] The above fusion strategy is used to indicate whether to perform corrective fusion or tolerance fusion on areas of effective change.

[0093] The aforementioned deviation characteristics are used to indicate the magnitude and spatial distribution range of the deviation in the effective change area, including the amount of deviation (i.e., the magnitude of the deviation) and the range (i.e., the spatial distribution range of the deviation) of the real scene change deviation, as well as the spatial distribution density of the scene change components in the effective change area (i.e., the degree of deviation aggregation).

[0094] In one possible implementation, the reliability of the initial 3D model in the effective variation region is evaluated based on the amount and range of deviation. A larger deviation and a wider range indicate a more severe deviation between the initial 3D model and the baseline model in that region, resulting in lower reliability of the initial 3D model. The reliability of the baseline model in the effective variation region is evaluated based on its spatial distribution density. A higher spatial distribution density indicates a more concentrated spatial distribution of deviations and more thorough cross-verification of the region by multi-view observations, thus increasing the reference value of the baseline model in that region. If the reliability of the initial 3D model is higher than that of the baseline model, a tolerance-based fusion strategy is chosen, directly retaining the original geometric data of the initial 3D model in the effective variation region. Conversely, if the reliability of the baseline model is higher than that of the initial 3D model, a correction-based fusion strategy is chosen, correcting the geometric deviations in the effective variation region using the baseline model as a reference before fusion.

[0095] Specifically, the process of determining the fusion strategy is as follows: Based on the deviation characteristics, the first credibility score of the initial 3D model and the second credibility score of the digital twin benchmark model are calculated respectively.

[0096] The first confidence score is used to characterize the reliability of the initial 3D model in the effective variation region.

[0097] The first confidence score is determined based on the deviation amount and range within the effective variation region. The first confidence score is negatively correlated with the deviation amount and range. The smaller the deviation amount and range, the more limited the variation range of the observation model in that region, the more stable the geometric structure, and the more reliable the initial 3D model.

[0098] Specifically, the process of calculating the first confidence score is as follows: Based on the deviation amount of the effective change area, the first deviation influence coefficient is determined, which is negatively correlated with the deviation amount; based on the deviation range of the effective change area, the second deviation influence coefficient is determined, which is negatively correlated with the deviation range; the first deviation influence coefficient and the second deviation influence coefficient are weighted and summed to determine the first confidence score of the initial 3D model.

[0099] Specifically, the deviation reflects the degree of deviation between the initial 3D model and the baseline model within the effective variation area. Multiple deviation level ranges are pre-defined, and each range corresponds to a first deviation influence coefficient value. The larger the deviation, the higher the corresponding deviation level, and the smaller the first deviation influence coefficient value.

[0100] For example, when the deviation is in a lower range, it indicates that the initial 3D model and the benchmark model are in good agreement in this region. In this case, the first deviation influence coefficient is taken as a larger value to preserve the original geometric features of the initial 3D model. When the deviation is in a higher range, it indicates that the initial 3D model and the benchmark model are seriously deviated from each other in this region. In this case, the first deviation influence coefficient is taken as a smaller value to reduce the weight of the initial 3D model in the fusion decision.

[0101] Optionally, if the effective change area covers multiple spatial locations, the statistical value of the deviation at each location can be calculated as the representative value of the deviation in that area, which is used to determine the first deviation influence coefficient.

[0102] For example, when the effective change area includes multiple spatial locations, the average deviation at each location can be taken as the representative value of the deviation for that area. Specifically, when the deviation at each location within the effective change area is relatively uniformly distributed and there are no obvious outliers, the arithmetic mean of the deviations can effectively characterize the overall deviation level of the area. Using this as the representative value of the deviation for subsequent reliability scoring calculations can ensure that the scoring results objectively reflect the overall situation of the area.

[0103] For example, when the distribution of deviations at various locations within the effective variation area is uneven and there are obvious outliers, the median of the deviations is used as the representative value of the deviations for that area. The median is not affected by extreme values ​​and can more robustly reflect the typical level of deviations in that area, avoiding distortion of the representative value due to outliers at individual locations, thereby affecting the accuracy of the credibility score.

[0104] For example, in conservative application scenarios that require focused detection and cannot tolerate the omission of any significant deviations, the maximum value of the deviation at each location within the effective variation area can be used as the representative value of the deviation. This method ensures that the spatial location with the most severe deviation within the area dominates the calculation of the credibility score, thereby triggering a decision that is more inclined to correct and fuse, ensuring reconstruction accuracy in the worst case. Whether the mean, median, or maximum value is used can be flexibly selected according to actual application needs, and this application does not limit this.

[0105] The above method achieves a quantitative mapping of the deviation magnitude to the credibility score of the initial 3D model, providing a basis for subsequent fusion decisions.

[0106] The second credibility score characterizes the reliability of the digital twin baseline model within the effective variation region. The second credibility score is determined based on the spatial distribution density of the effective variation region, and is positively correlated with the spatial distribution density.

[0107] The higher the spatial distribution density, the more concentrated the deviation is in that area, and the more likely it is a change in the real scene rather than random noise. In this case, the baseline model should be trusted for correction.

[0108] Specifically, the process of calculating the second credibility score is as follows: the second credibility score is determined based on the spatial distribution density of the effective change area.

[0109] Spatial distribution density is used to characterize the degree of aggregation of bias within the effective variation area. The higher the density, the more concentrated the bias is in space, the more fully the area is cross-confirmed by multi-view observations, and the higher the reference value of the digital twin benchmark model in the area.

[0110] Multiple density level intervals are pre-defined, and each interval corresponds to a second credibility score value. The higher the spatial distribution density, the higher the corresponding density level and the larger the second credibility score value.

[0111] For example, when the spatial distribution density of the effective change area is in a lower range, it indicates that the bias is spatially dispersed and the degree of cross-confirmation from multiple perspectives is insufficient. In this case, the reference value of the benchmark model in this area is limited, and the second confidence score is taken as a smaller value. When the spatial distribution density is in a higher range, it indicates that the bias is spatially highly clustered and the multi-perspective observations provide sufficient cross-confirmation for this area. In this case, the confidence of the benchmark model in this area is high, and the second confidence score is taken as a larger value. When the effective change area contains multiple discrete sub-regions, the spatial distribution density of each sub-region can be calculated separately, and then a weighted average based on the area of ​​each sub-region can be used to obtain the representative value of the overall spatial distribution density.

[0112] The above method enables a quantitative mapping of spatial distribution density to the credibility score of the benchmark model, providing a basis for determining the fusion strategy.

[0113] The fusion strategy is determined based on the comparison results of the first and second credibility scores: If the first credibility score is greater than or equal to the second credibility score, then the fusion strategy is determined to be tolerance fusion.

[0114] The aforementioned tolerance fusion refers to preserving the original geometric data of the initial 3D model in the effective variation region, i.e., trusting the reconstruction results of the observation model.

[0115] If the second confidence score is greater than the first confidence score, then the fusion strategy is determined to be corrective fusion; corrective fusion refers to using the digital twin benchmark model as a reference to correct the geometric deviation of the effective change area in the initial three-dimensional model before fusion.

[0116] It should be understood that a high initial confidence score indicates a small amount of deviation and a limited range of deviation in that region. This means the initial 3D model does not deviate significantly from the baseline model in that area. In this case, the deviation may simply be due to minor, normal variations between the historical state and the current actual state of the baseline model, rather than a significant error in the reconstruction result of the initial 3D model. Preserving the original geometry of the current observation data accurately reflects the current true state of the target region, avoiding unnecessary corrections to the already correct reconstruction results and preventing the introduction of new errors.

[0117] It should be understood that a higher second confidence score indicates a higher spatial density of the region, meaning the bias exhibits a clear clustering characteristic in space. Multi-view observations provide sufficient cross-confirmation of this region, and in areas with high cross-confirmation density, the reliability of the baseline model's information is higher. In this case, the bias in the initial 3D model is likely due to reconstruction errors caused by unsatisfactory imaging conditions, rather than actual changes in the target region. Using the baseline model for geometric correction can utilize the accurate geometric prior information in the baseline model to eliminate the influence of observation errors, ensuring the geometric accuracy of this region in the final 3D model.

[0118] S105-2. Based on the fusion strategy, the geometric data of the initial 3D model in the effective change region is fused with the geometric data of the digital twin reference model to obtain the fused geometric data of the effective change region.

[0119] The specific process of the above-mentioned fusion execution is as follows: When the fusion strategy is tolerance fusion, the original geometric data of the initial 3D model in the effective change region is used as the fused geometric data, that is, the observation reconstruction results are directly retained.

[0120] When the fusion strategy is corrective fusion, the following operations are performed: Extract the first set of geometric coordinates of the effective changed region in the initial 3D model; extract the second set of geometric coordinates of the corresponding position of the effective changed region in the digital twin baseline model; determine the spatial transformation relationship based on the first and second geometric coordinate sets; transform the geometric coordinates of the effective changed region using the spatial transformation relationship; and use the transformed geometric data as the fused geometric data. The spatial transformation relationship describes the rotation, translation, and scaling relationships between the two geometric coordinate sets, achieving alignment between the geometric coordinates of the observation model and the baseline model through spatial transformation.

[0121] For example, the process of determining spatial transformation relationships is as follows: Extract corresponding feature point pairs from the first geometric coordinate set and the second geometric coordinate set respectively. Each feature point pair contains a three-dimensional point from the initial three-dimensional model and its corresponding three-dimensional point in the baseline model. With the goal of minimizing the overall alignment error between all feature point pairs, the spatial transformation parameters (including rotation angles around the three coordinate axes, translation along the three coordinate axes, and overall scaling factor) are gradually adjusted through iterative optimization so that the feature points in the initial 3D model coincide as much as possible with the corresponding feature points in the baseline model after spatial transformation. When the overall alignment error is less than the preset registration accuracy threshold or the number of iterations reaches the preset maximum number of iterations, the optimization stops and the current spatial transformation parameters are used as the final spatial transformation relationship.

[0122] The aforementioned preset registration accuracy threshold is an error standard used to determine whether the optimization of spatial transformation parameters has met the accuracy requirements. Its physical meaning is: when the overall alignment error between the geometric coordinates of the effective transformation region in the initial 3D model and the corresponding geometric coordinates in the reference model after spatial transformation is less than this threshold, the two sets of geometric coordinates are considered sufficiently aligned, and the accuracy improvement brought by further iterative optimization is negligible. This threshold determines the lower limit of registration accuracy for spatial transformation.

[0123] In one possible implementation, the preset registration accuracy threshold is determined based on the combined accuracy levels of the initial 3D model and the reference model.

[0124] Specifically, if the point accuracy of both the initial 3D model and the reference model is high (e.g., both are LiDAR point cloud models with centimeter-level accuracy), the registration accuracy threshold is set to a relatively small value to ensure that the two models achieve high-precision alignment after spatial transformation. If the accuracy of the initial 3D model is relatively low, the registration accuracy threshold is appropriately relaxed to an order of magnitude that matches the model accuracy to avoid excessive pursuit of high registration accuracy, which would lead to a waste of computational resources.

[0125] Optionally, the preset registration accuracy threshold can be set differently according to the terrain complexity of the target area and application requirements.

[0126] For areas with flat terrain and simple features, the registration accuracy threshold can be set to a larger value; for areas with complex terrain and dense buildings, the registration accuracy threshold should be set to a smaller value. For high-precision military applications, the registration accuracy threshold should be set to a more stringent small value; for general terrain analysis scenarios, the threshold can be appropriately relaxed to balance computational efficiency and accuracy.

[0127] The aforementioned maximum number of iterations is a protective termination condition to prevent the spatial transformation parameter optimization process from getting stuck in an infinite loop or excessively consuming computational resources. Physically, it means that when the number of iterations reaches this upper limit, the iteration is forcibly terminated regardless of whether the current alignment error meets the accuracy threshold requirement, and the current spatial transformation parameters are output as the final result. This parameter is set to ensure that the algorithm converges within a finite time, avoiding uncontrollable computation time due to excessive data volume or large initial pose deviations.

[0128] In one possible implementation, the preset maximum number of iterations is determined comprehensively based on the computational resource constraints and real-time requirements of the actual application scenario. For offline processing scenarios with ample computational resources and low real-time requirements, the maximum number of iterations can be set to a larger value to ensure that the algorithm has sufficient opportunity to converge to the optimal solution; for online processing scenarios with limited computational resources or high real-time requirements, the maximum number of iterations is set to a smaller value to optimize the registration accuracy as much as possible while ensuring that the algorithm can complete within a limited time.

[0129] Optionally, the preset maximum number of iterations can be adaptively adjusted in conjunction with the convergence speed of the registration accuracy threshold. During the iterative optimization process, the decrease in alignment error between adjacent iterations can be monitored: if the decrease in error for several consecutive iterations is less than the preset decrease threshold, it indicates that the algorithm has converged and can be terminated early even if the maximum number of iterations has not been reached; conversely, if the decrease in error continues to be large, it indicates that the algorithm is still undergoing significant optimization and iteration can be allowed to continue until the maximum number of iterations is reached.

[0130] The above adaptive mechanism can maximize computational efficiency while ensuring registration accuracy.

[0131] After obtaining the spatial transformation relationship, the spatial transformation relationship is applied to all three-dimensional coordinate points within the effective transformation area. The geometric coordinates of the initial three-dimensional model in this area are transformed to a position aligned with the reference model. The transformed geometric data is then used as the fused geometric data for this area.

[0132] Through the above spatial transformation process, the systematic geometric deviation between the initial 3D model and the benchmark model is eliminated, and the spatial consistency between the two is achieved.

[0133] S105-3. Generate the final 3D model based on the fused geometric data.

[0134] Specifically, the fused geometric data of the effectively changed regions replaces the original geometric data of the corresponding regions in the initial 3D model, resulting in an updated 3D model. Before outputting the updated 3D model as the final 3D model, boundary smoothing can be performed to eliminate geometric discontinuities between the effectively changed regions and the unchanged regions in the initial 3D model. Boundary smoothing can employ geometric processing algorithms such as Laplacian smoothing and Poisson fusion to ensure a smooth transition between the fused region and the surrounding regions. Example 2

[0135] This embodiment provides another implementation of the confidence score calculation method in step S105, based on embodiment 1. In this embodiment, steps S201 to S204 are the same as steps S101 to S104 in embodiment 1; please refer to embodiment 1 for details, which will not be repeated here.

[0136] S205. Based on the deviation characteristics of the effective variation area, the initial 3D model, and the digital twin baseline model, determine the final 3D model, such as... Figure 2 As shown, it includes the following steps: In this embodiment, S205-1 is the same as S105-1 in Embodiment 1. For details, please refer to Embodiment 1, which will not be repeated here.

[0137] S205-2. Fusion is performed based on the fusion strategy, which is the same as S105-2 in Example 1. For details, please refer to Example 1, which will not be repeated here.

[0138] S205-3. Generate the final 3D model based on the fused geometric data. In this embodiment, before outputting the updated 3D model as the final 3D model, further iterative optimization processing is performed. The iterative optimization process is as follows: The updated 3D model is compared with the digital twin baseline model to determine whether the deviation of the effective change area is less than the preset deviation threshold.

[0139] If the deviation is less than the preset deviation threshold, the iteration stops and the current 3D model is output as the final 3D model.

[0140] If the deviation is still greater than or equal to the preset deviation threshold, and the number of iterations has not reached the preset maximum number of iterations, then a new effective change region is determined based on the updated 3D model, and the correction and tolerance processes are re-executed on the new effective change region to reconstruct the updated 3D model. The iterative optimization process continues until the preset termination condition is met.

[0141] The aforementioned preset termination condition is to stop when the deviation is less than the preset deviation threshold or when the number of iterations reaches the preset maximum number of iterations.

[0142] Through iterative optimization, the latest true state of the target area can be gradually approximated. Each iteration uses the more accurate geometric data from the previous fusion to redefine the changed areas and fusion strategies, thereby improving accuracy round by round. Example 3

[0143] This embodiment provides an image fusion and reconstruction system that uses the platform deployed on an industrial internet platform. Figure 3 This is a schematic diagram of the platform structure in this embodiment. The platform includes: The acquisition module 301 is used to acquire multi-view remote sensing images of the target area and generate an initial 3D model based on the multi-view remote sensing images; the simulation module 302 is used to acquire a pre-constructed digital twin reference model of the target area and generate predicted images at various shooting angles based on the digital twin reference model; the calculation module 303 is used to compare the remote sensing images and the predicted images step by step, calculate the imaging deviation, and decompose the imaging deviation into scene change components and observation interference components; the filtering module 304 is used to determine the intersection distribution of the multi-view observation rays in 3D space based on the spatial pose of each shooting angle, and jointly filter the target area based on the scene change components and the intersection distribution to obtain the effective change area; the fusion module 305 is used to determine the final 3D model based on the deviation characteristics of the effective change area, the initial 3D model, and the digital twin reference model; the deviation characteristics are used to indicate the magnitude and spatial distribution range of the deviation in the effective change area, including the deviation amount and deviation range of the real scene change deviation.

[0144] Each module of the system in this embodiment executes the corresponding steps in the above method embodiment. For specific implementation details, please refer to the method embodiment, which will not be repeated here.

[0145] The beneficial effects of the above technical solution are as follows: The multi-view military image fusion 3D reconstruction and accuracy calibration platform provided in this embodiment generates predicted images based on a digital twin benchmark model using a simulation module. The calculation module then compares the measured images with the predicted images step-by-step and decomposes the deviations, effectively distinguishing between real scene changes and observational interference. Secondly, a screening module introduces the intersection density of multi-view observation rays for joint screening, ensuring that only effectively changed areas with sufficient cross-confirmation from multiple views are processed, avoiding misjudgments caused by insufficient observation coverage. Finally, the fusion module dynamically selects tolerance fusion or correction fusion strategies based on deviation characteristics, achieving differentiated and refined processing of regions with different deviation characteristics. The overall combination of these technical means enables the platform to significantly improve the accuracy, robustness, and automation of 3D reconstruction while preserving physical interpretability, making it particularly suitable for scenarios with high requirements for reconstruction accuracy and reliability, such as military reconnaissance and digital modeling of battlefield environments.

[0146] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from the spirit and scope of this invention, and this invention is also intended to include these modifications and variations.

Claims

1. A method for multi-view military image fusion, 3D reconstruction, and accuracy calibration, characterized in that, The method includes: Acquire multi-view remote sensing images of the target area, and generate an initial three-dimensional model based on the multi-view remote sensing images; Obtain a pre-constructed digital twin benchmark model of the target area, and simulate and generate predicted images at various shooting angles based on the digital twin benchmark model; The remote sensing image is compared with the predicted image step by step to calculate the imaging deviation, and the imaging deviation is decomposed into scene change component and observation interference component. Based on the spatial pose of each shooting angle, the intersection distribution of multi-view observation rays in three-dimensional space is determined, and the target area is jointly screened based on the scene change components and the intersection distribution to obtain the effective change area. Based on the deviation characteristics of the effective change region, the initial 3D model, and the digital twin baseline model, the final 3D model is determined; the deviation characteristics are used to indicate the magnitude and spatial distribution range of the deviation in the effective change region, including the amount and range of deviations in real scene changes.

2. The method based on claim 1, characterized in that, Based on the spatial pose of each shooting angle, the intersection distribution of multi-view observation rays in three-dimensional space is determined, and the target area is jointly screened based on the scene change components and the intersection distribution to obtain effective change areas, including: Obtain the spatial pose parameters corresponding to each shooting angle; Based on the spatial pose parameters, the propagation path of the imaging rays in each frame of remote sensing image is tracked in three-dimensional space to determine the intersection area of ​​each imaging ray in three-dimensional space and the intersection density of imaging rays in each intersection area; the intersection density is used to characterize how many imaging rays from different shooting angles jointly cover the same spatial location. The regions in the scene change components that exceed a preset change threshold and whose intersection density exceeds a preset density threshold are identified as the effective change regions.

3. The method based on claim 1, characterized in that, The determination of the final 3D model based on the deviation characteristics of the effective variation region, the initial 3D model, and the digital twin baseline model includes: Based on the aforementioned deviation characteristics, a fusion strategy is determined between the initial 3D model and the digital twin baseline model within the effective variation region; the fusion strategy is used to indicate whether to perform corrective fusion or tolerance fusion on the effective variation region. Based on the fusion strategy, the geometric data of the initial 3D model in the effective change region is fused with the geometric data of the digital twin reference model to obtain the fused geometric data of the effective change region. The final 3D model is generated based on the fused geometric data.

4. The method based on claim 3, characterized in that, The method for determining the fusion strategy between the initial 3D model and the digital twin baseline model in the effective variation region based on the deviation characteristics includes: Based on the aforementioned deviation characteristics, a first credibility score for the initial 3D model and a second credibility score for the digital twin benchmark model are calculated respectively. If the first confidence score is greater than or equal to the second confidence score, then the fusion strategy is determined to be the tolerance fusion; the tolerance fusion refers to retaining the original geometric data of the initial 3D model in the effective change region; If the second credibility score is greater than the first credibility score, then the fusion strategy is determined to be the corrective fusion; the corrective fusion refers to using the digital twin benchmark model as a reference to correct the geometric deviation of the effective change region in the initial three-dimensional model before fusion.

5. The method based on claim 4, characterized in that, The deviation features also include the spatial distribution density of the scene change components within the effective change area; the calculation of the first credibility score of the initial 3D model and the second credibility score of the digital twin benchmark model based on the deviation features includes: Based on the deviation amount and deviation range of the effective variation region, a first confidence score of the initial 3D model is determined; the first confidence score is negatively correlated with the deviation amount and the deviation range, and is used to characterize the reliability of the initial 3D model in the effective variation region; Based on the spatial distribution density of the effective variation region, a second credibility score is determined for the digital twin benchmark model; the second credibility score is positively correlated with the spatial distribution density and is used to characterize the reliability of the digital twin benchmark model in the effective variation region.

6. The method based on claim 5, characterized in that, The determination of the first credibility score of the initial 3D model based on the deviation amount and deviation range of the effective change region includes: Based on the deviation amount of the effective change region, a first deviation influence coefficient is determined; the first deviation influence coefficient is negatively correlated with the deviation amount. Based on the deviation range of the effective change region, a second deviation influence coefficient is determined; the second deviation influence coefficient is negatively correlated with the deviation range. The first deviation influence coefficient and the second deviation influence coefficient are weighted and summed to determine the first credibility score of the initial three-dimensional model.

7. The method based on claim 3, characterized in that, Based on the fusion strategy, the geometric data of the initial 3D model in the effective change region is fused with the geometric data of the digital twin baseline model to obtain the fused geometric data of the effective change region, including: When the fusion strategy is the tolerance fusion, the original geometric data of the initial 3D model in the effective change region is used as the fused geometric data; When the fusion strategy is the corrective fusion, the first set of geometric coordinates of the effective change region in the initial three-dimensional model is extracted, and the second set of geometric coordinates of the corresponding position of the effective change region in the digital twin reference model is extracted; Based on the first set of geometric coordinates and the second set of geometric coordinates, a spatial transformation relationship is determined, and the geometric coordinates of the effective transformation region are transformed using the spatial transformation relationship; The transformed geometric data is used as the fused geometric data.

8. The method based on claim 3, characterized in that, The generation of the final 3D model based on the fused geometric data includes: The fused geometric data of the effective changed region is used to replace the original geometric data of the corresponding region in the initial 3D model to obtain the updated 3D model. The updated 3D model is output as the final 3D model.

9. The method based on claim 8, characterized in that, Before outputting the updated 3D model as the final 3D model, the method further includes: The updated 3D model is subjected to boundary smoothing to eliminate the geometric discontinuities between the effective changed region and the unchanged region in the initial 3D model.

10. A multi-view military image fusion 3D reconstruction and accuracy calibration platform, characterized in that, The platform includes: The acquisition module is used to acquire multi-view remote sensing images of the target area and generate an initial three-dimensional model based on the multi-view remote sensing images; The simulation module is used to acquire a pre-constructed digital twin reference model of the target area, and to simulate and generate predicted images at various shooting angles based on the digital twin reference model. The calculation module is used to compare the remote sensing image with the predicted image step by step, calculate the imaging deviation, and decompose the imaging deviation into scene change components and observation interference components. The filtering module is used to determine the intersection distribution of multi-view observation rays in three-dimensional space based on the spatial pose of each shooting angle, and to jointly filter the target area based on the scene change components and the intersection distribution to obtain the effective change area. The fusion module is used to determine the final three-dimensional model based on the deviation characteristics of the effective change region, the initial three-dimensional model, and the digital twin reference model; the deviation characteristics are used to indicate the magnitude and spatial distribution range of the deviation in the effective change region, including the amount and range of deviations in real scene changes.

Images