A Method and System for Intelligence Generation Based on Feature Fingerprint Storage and Spatiotemporal Geometric Correction

Abstract

This invention discloses an intelligence generation method and system based on feature fingerprint storage and spatiotemporal geometric correction, belonging to the field of intelligent remote sensing image processing technology. The method involves: accessing heterogeneous data streams generated from multi-source remote sensing images in a synchronized time sequence; inputting a shared backbone network and a modality adaptation layer to generate a current feature stream in a unified semantic space; asynchronously retrieving historical baseline features from an in-memory feature fingerprint database based on geographic coordinates; fusing satellite imaging parameters with the current feature stream and inputting it into a spatial transformation network, resampling historical baseline features using a regression transformation matrix to achieve spatiotemporal geometric correction in the feature domain; weighted fusing of the current feature streams from each modality to generate a fused feature stream; and parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report. This invention reduces memory usage and data throughput, shortens the intelligence generation cycle, reduces the false alarm rate, and achieves synergistic optimization of timeliness, accuracy, and automation.

Description

Technical Field

[0001] This invention relates to the field of intelligent processing technology for remote sensing images; and more particularly to an information generation method and system based on feature fingerprint storage and spatiotemporal geometric correction. Background Technology

[0002] Remote sensing image intelligence generation technology is a key area of ​​intersection and integration of geographic information systems, artificial intelligence, and spatial information technology, and is widely used in scenarios with extremely high timeliness requirements, such as disaster prevention and mitigation and urban planning. However, existing multi-source remote sensing (SAR, optical, infrared) intelligence generation technologies have revealed the following serious technical bottlenecks that severely restrict timeliness and accuracy in practical applications: (1) The data loading overhead under dynamic monitoring generates huge I / O latency; When performing dynamic intelligence tasks such as change detection, traditional technical solutions are based on an "image-level comparison" model, meaning the system must simultaneously load both the current real-time image and historical baseline image from the storage medium. Since remote sensing images often reach gigabyte levels, the reading, decoding, and preprocessing of massive amounts of raw pixel data generate huge I / O throughput latency. Industry estimates suggest that data loading time often accounts for more than 40% of the total process time, preventing intelligence output from converging to within minutes and making it difficult to support high-frequency rolling monitoring.

[0003] (2) Imaging geometric differences lead to serious false alarms of “spurious changes”, resulting in low intelligence reliability; Existing interpretation schemes are highly susceptible to the effects of satellite imaging angles, orbital attitudes, and illumination phase differences when processing imagery from different time periods. Because current technologies lack sophisticated alignment mechanisms for feature spaces, simple pixel-level or rudimentary feature comparisons often misinterpret changes such as building shadow movement and imaging geometric distortions as ground feature evolution. This spatiotemporal inconsistency leads to "false signals," resulting in an extremely high false alarm rate in intelligence products and severely interfering with decision-making.

[0004] (3) Redundancy in task execution chain, resulting in low feature extraction efficiency; In the existing technical architecture, interpretation tasks such as object detection, change detection, and scene classification are distributed independently in a "chimney-like" manner. When processing multi-source images of the same region, the system needs to load and run multiple independent deep learning models separately. This serial mode causes the image backbone to be executed repeatedly, resulting in severe redundancy in convolutional computation and memory usage, which can easily lead to computational crashes at edge computing power-constrained locations.

[0005] (4) The multi-source fusion mechanism suffers from a serious imbalance between "accuracy" and "timeliness"; While existing feature-level fusion methods (such as non-local attention mechanisms) offer high accuracy, their computational complexity increases exponentially, with a single fusion often taking more than 5 seconds, making them difficult to embed into real-time monitoring streams. On the other hand, simple pixel-level fusion, though fast, cannot deeply explore the complementary information between SAR's penetration and the semantic richness of optics, resulting in target recognition accuracy often falling below 70% in complex environments.

[0006] (5) Insufficient end-to-end automation and long intelligence production cycle; In the existing process, there is a lack of deep integration between the various stages, from raw data access and heterogeneous data fusion to the final structured report output. The intermediate stages still require a large amount of manual intervention for result comparison and format conversion. This "latency black hole" introduced by manual intervention means that the intelligence production cycle is usually measured in hours, which cannot meet the requirements of modern rapid decision-making for second- or minute-level responses.

[0007] In summary, existing technologies have significant shortcomings in terms of storage mode, alignment accuracy, architectural integration, and automation. There is an urgent need for a high-efficiency intelligence generation solution that can achieve feature-level incremental interpretation, sub-pixel-level geometric alignment, and end-to-end automated generation capabilities. Summary of the Invention

[0008] Based on the above analysis, the present invention aims to disclose an intelligence generation method and system based on feature fingerprint storage and spatiotemporal geometric correction, in order to solve the core problem of "the inability to simultaneously achieve timeliness, accuracy and automation".

[0009] This invention discloses an intelligence generation method based on feature fingerprint storage and spatiotemporal geometric correction, comprising: S1. Asynchronously access multi-source remote sensing images for parallel preprocessing to generate a time-synchronized heterogeneous data stream; S2. Input the heterogeneous data stream into the shared backbone network + modality adaptation layer, extract and normalize the features, and generate the current feature stream in a unified semantic space; S3. Based on the current image geographic coordinates, asynchronously retrieve historical baseline features from the memory feature fingerprint database. The historical baseline features are compressed high-dimensional features stored in the database after historical images have been processed by a shared backbone network, modality adaptation layer, and multiple downsampling. S4. After fusing the satellite imaging parameters in the image metadata with the current feature stream, input the result into the spatial transformation network and regress to obtain the transformation matrix. Use this matrix to resample the historical baseline features to achieve spatiotemporal geometric correction of the feature domain and obtain historical features that are spatially aligned with the current feature stream. S5. Dynamically calculate the fusion weights based on the quality differences of each modality, and then weight and fuse the current feature flows of each modality to generate a fused feature flow. S6. Perform parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report.

[0010] Furthermore, S2 includes: S2-1. Input the heterogeneous data streams of synchronous SAR, optical, and infrared output from S1 into a multimodal shared backbone network in parallel to extract primary features with uniform channel number and spatial size. ; S2-2, In the modal adaptation layer, through independent Convolutional layers for primary features Perform linear transformation and reconstruction along the channel dimension to complete the semantic space projection of cross-modal features and output channel-aligned transition features; S2-3. In the modal adaptation layer, for The features after convolutional projection are normalized by a layer. By standardizing the mean and variance, combined with learnable gain and bias parameters, differences in radiometric calibration between sensors are eliminated, and standardized adaptive features with cross-modal semantic alignment are output. .

[0011] Furthermore, standardized adaptation features for cross-modal semantic alignment The expression is: ; in, for Convolution operation, and These represent the mean and variance of the feature map along the channel dimension, respectively. To prevent division by zero of extremely small constants; and These are learnable gain and bias parameters; ; Among them, synchronous heterogeneous data streams , This is a single-channel SAR image; It is a three-channel optical RGB image; This is a single-channel infrared thermal radiation image; It is a multimodal shared backbone network.

[0012] Furthermore, S3 includes: S3-1. Extract geographic coordinate information from the current image-related metadata and generate a latitude and longitude grid index that matches the feature fingerprint database according to preset rules. S3-2. Using latitude and longitude grid index as the key, accurately retrieve and call up the matching historical baseline feature tensor from the memory feature fingerprint database through an asynchronous thread; The feature fingerprint stored in memory is a compressed high-dimensional feature obtained by processing historical images through the same shared backbone network and modality adaptation layer as in S2, and then using multiple downsampling.

[0013] Furthermore, S4 includes: S4-1. Extract rational polynomial coefficients (RPC) parameters containing satellite imaging geometric information from the current image's native metadata. S4-2. After the RPC parameters are characterized, encoded and normalized, they are cross-domain spliced ​​and fused with the current standardized adaptive feature stream in the channel dimension to obtain fused features that have both semantic and physical prior knowledge. S4-3. Input the fused features into the localization network of the spatial transformation network, regress to obtain the six-degree-of-freedom transformation parameters, and construct the corresponding affine transformation matrix; S4-4. Calculate the sampling coordinates of historical baseline features based on the affine transformation matrix, and complete sub-pixel level resampling of the feature domain through bilinear interpolation to obtain historical features aligned with the current feature stream.

[0014] Furthermore, the historical features aligned with the current feature stream are: ; Characteristics of historical baselines, This is a bilinear interpolation operation. It is the affine transformation matrix; These are the six-degree-of-freedom transformation parameters obtained from the regression. ; To locate the network, , for the current feature flow With RPC parameters Cross-domain splicing and fusion representation. This indicates a channel dimension splicing operation.

[0015] Furthermore, S5 includes: S5-1. Perform global average pooling on the standardized adaptation features of each modality, and extract a one-dimensional channel descriptor that represents the overall quality of the modality through spatial feature compression. S5-2. Input the channel descriptor into the lightweight bottleneck structure MLP, and learn the cross-modal dynamic weight vector in the 0~1 interval after nonlinear transformation and normalization. S5-3. Perform channel-level weighted fusion of each modal feature based on dynamic weight vector, adaptively adjust the weight coefficient according to the modal signal-to-noise ratio, and output an enhanced fusion feature stream that integrates the advantages of each modality.

[0016] Furthermore, S6 includes: S6-1. Three interpreters—detection, residual change detection, and scene—are run in parallel on the fused feature stream. The real change signal is enhanced through feature difference, ROI mask, and LeakyReLU activation gate, and multi-dimensional interpreter features are output. S6-2. Integrate the inference results of multiple interpreters to extract structured interpreter vectors, associate them with geographic coordinate information, encapsulate them into standardized input through prompt word templates, and inject them into the fine-tuned T5-Small model; S6-3: The T5-Small model automatically transforms standardized input into logically coherent natural language intelligence reports, completing end-to-end intelligence generation and output.

[0017] Furthermore, S1 includes: S1-1: Input synthetic aperture radar single-channel complex data, optical three-channel RGB data and infrared single-channel thermal radiation data, use a multi-thread pool to allocate independent computing resources for each mode and pull image slices in real time through asynchronous I / O mechanism, and output multi-source raw image data streams received in parallel. S1-2. Perform directional denoising processing based on second-order integral map on SAR modal data in multi-source raw image data stream, and transmit optical and infrared modal data in parallel to output multi-source image data stream after parallel denoising. S1-3. For the multi-source image data stream after parallel denoising, the denoising weight calculation is accelerated at the instruction level using the single instruction multiple data stream instruction set. The barrier synchronization mechanism ensures that the data of each modality is synchronized and ready, and outputs a synchronized heterogeneous data stream.

[0018] Another aspect of the present invention discloses an intelligence generation system that implements the intelligence generation method based on feature fingerprint storage and spatiotemporal geometric correction as described above, comprising: The parallel access module is used to asynchronously access multi-source remote sensing images for parallel preprocessing and generate a time-synchronized heterogeneous data stream. The shared feature extraction module is used to input heterogeneous data streams into a shared backbone network + modality adaptation layer, extract and normalize features, and generate the current feature stream in a unified semantic space. The feature fingerprint storage module is used to asynchronously retrieve historical baseline features from the memory feature fingerprint library based on the current image geographic coordinates. The historical baseline features are compressed high-dimensional features of historical images stored in the library after being processed by a shared backbone network, a modality adaptation layer, and multiple downsampling. The spatiotemporal geometric correction module is used to fuse satellite imaging parameters in image metadata with the current feature stream and input them into a spatial transformation network. The transformation matrix is ​​obtained by regression, and the historical baseline features are resampled using this matrix to achieve spatiotemporal geometric correction in the feature domain, thereby obtaining historical features that are spatially aligned with the current feature stream. The dynamic fusion module is used to dynamically calculate the fusion weights based on the quality differences of each modality, and to weight and fuse the current feature streams of each modality to generate a fused feature stream. The parallel interpretation module is used to perform parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report.

[0019] Compared with the prior art, the present invention has at least the following advantages: The intelligence generation method and system based on feature fingerprint storage and spatiotemporal geometric correction of this invention solves three core pain points in remote sensing intelligence production: I / O blocking, geometric mismatch, and computational redundancy, through the underlying logical transformation of "feature-based image replacement" and the combination of deep learning operators guided by physical imaging mechanisms. It adopts a heterogeneous shared backbone network to enable multiple tasks to share convolution parameters, reducing memory usage by approximately 65% ​​and increasing inference speed by approximately 3 times, achieving smooth deployment at the edge. A feature fingerprint database based on geographic coordinate indexes compresses data volume to 1%-5% of the original image, reducing throughput by more than 95%, and converging the intelligence generation cycle from hours to within minutes. The introduction of a spatial transformation network guided by satellite RPC parameters and dynamic channel attention fusion reduces the false alarm rate of change detection by approximately 70% and improves target recognition accuracy to over 90% in complex environments. Through deep integration of a parallel interpretation head and a pre-trained language model, it achieves an end-to-end automated closed loop from data access to report generation, completely eliminating the latency of manual intervention. Attached Figure Description

[0020] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 This is a flowchart of the intelligence generation method based on feature fingerprint storage and spatiotemporal geometric correction in an embodiment of the present invention; Figure 2 This is a schematic diagram showing the components and connections of the intelligence generation system in an embodiment of the present invention. Detailed Implementation

[0021] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and, together with the embodiments of the present invention, serve to illustrate the principles of the present invention.

[0022] Example 1 One embodiment of the present invention discloses an information generation method based on feature fingerprint storage and spatiotemporal geometric correction, such as... Figure 1 As shown, it includes: S1. Asynchronously access multi-source remote sensing images for parallel preprocessing to generate a time-synchronized heterogeneous data stream; S2. Input the heterogeneous data stream into the shared backbone network + modality adaptation layer, extract and normalize the features, and generate the current feature stream in a unified semantic space; S3. Based on the current image geographic coordinates, asynchronously retrieve historical baseline features from the memory feature fingerprint database. The historical baseline features are compressed high-dimensional features stored in the database after historical images have been processed by a shared backbone network, modality adaptation layer, and multiple downsampling. S4. After fusing the satellite imaging parameters in the image metadata with the current feature stream, input the result into the spatial transformation network and regress to obtain the transformation matrix. Use this matrix to resample the historical baseline features to achieve spatiotemporal geometric correction of the feature domain and obtain historical features that are spatially aligned with the current feature stream. S5. Dynamically calculate the fusion weights based on the quality differences of each modality, and then weight and fuse the current feature flows of each modality to generate a fused feature flow. S6. Perform parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report.

[0023] Specifically, S1 includes: S1-1: Input synthetic aperture radar single-channel complex data, optical three-channel RGB data and infrared single-channel thermal radiation data, use a multi-thread pool to allocate independent computing resources for each mode and pull image slices in real time through asynchronous I / O mechanism, and output multi-source raw image data streams received in parallel. Specifically, independent data receiving threads are established for the three heterogeneous modes: the SAR receiving thread is configured with a single-channel complex data parsing protocol to accurately calculate, verify, and normalize the real and imaginary parts of the radar echo signal, preserving the phase and amplitude characteristics of the SAR data; the optical receiving thread is configured with a three-channel RGB parallel decoding logic to support color image restoration and color gamut calibration for multiple visible light bands, ensuring the integrity of the semantic features of the optical image; and the infrared receiving thread is configured with a single-channel thermal radiation data streaming buffer mechanism to achieve continuous acquisition and data smoothing of ground object thermal infrared radiation information, avoiding abrupt distortion of thermal radiation data.

[0024] Each thread uses an asynchronous I / O mechanism to pull image slices from the satellite ground station or data push interface in real time, avoiding the accumulation of latency caused by synchronous blocking.

[0025] S1-2. Perform directional denoising processing based on second-order integral map on SAR modal data in multi-source raw image data stream, and transmit optical and infrared modal data in parallel to output multi-source image data stream after parallel denoising. SAR images inherently possess speckle noise due to their imaging mechanism, and noise suppression computationally intensive. This step employs a second-order integral image acceleration mechanism for directional optimization. The specific implementation steps are as follows: 1) Construction of second-order integral map; for input SAR image Perform a pixel-by-pixel scan, and accumulate and calculate the second-order integral image in a single traversal to generate the second-order integral image. , The calculation formula is: ; in, It is a second-order integral graph; The coordinates of the top left corner of the rectangular window. These are pixel coordinates.

[0026] 2) Fast window statistics and denoising based on integral images; weight calculation for non-local mean (NL-Means) denoising of SAR images based on the constructed second-order integral image; achieving efficient suppression of SAR speckle noise; Fast window statistics based on integral images optimize the traditional pixel-by-pixel rectangular window statistics into an integral image four-vertex lookup table method. The formula for calculating the cumulative sum of pixels within any rectangular window is: ; in, The coordinates of the top left corner of the rectangular window. and These are the window width and height, respectively. This is the cumulative sum of all pixels within the rectangular window; Based on the cumulative sum, the mean and variance of pixels within the window can be quickly derived, providing a statistical basis for the weight allocation of NL-Means denoising.

[0027] This optimization reduces the computational complexity of rectangular window statistics from the traditional (N is the number of pixels in the window) decreased to It only requires 4 table lookup operations to complete the statistical calculation of any window, which greatly reduces the weight calculation time of the NL-Means denoising algorithm and achieves efficient suppression of SAR speckle noise.

[0028] In this step, optical and infrared modal data are transmitted in parallel with the SAR denoising process to ensure the parallelism of multi-source data processing.

[0029] S1-3. For the multi-source image data stream after parallel denoising, the denoising weight calculation is accelerated at the instruction level using the single instruction multiple data stream instruction set. The barrier synchronization mechanism ensures that the data of each modality is synchronized and ready, and outputs a synchronized heterogeneous data stream. When performing NL-Means denoising weight calculations on SAR impacts, instruction-level parallel acceleration is achieved using a Single Instruction Multiple Data Stream (SIMD) instruction set (such as AVX2 or NEON). Experiments show that for large SAR images of 5000×5000 pixels, the denoising time is reduced from over 15 seconds to less than 1.8 seconds.

[0030] A synchronization barrier is set at the end of the preprocessing stage. This barrier will block the start of the subsequent feature extraction process until the data streams of all modes, including SAR (denoised + SIMD accelerated), optical, and infrared, have completed this step and reached the synchronization node.

[0031] The design of the synchronization barrier specifically addresses the issue that SAR mode processing takes longer than optical and infrared processing, completely avoiding subsequent feature fusion misalignment caused by inconsistent processing progress of different modes. It ensures that multi-source heterogeneous data streams can flow synchronously into the next-level feature extraction stage, providing a temporally consistent input basis for subsequent heterogeneous feature normalization.

[0032] Specifically, S2 includes: S2-1, the heterogeneous data streams of synchronous SAR, optical, and infrared output from S1 are input in parallel into a multimodal shared backbone network to extract primary features with uniform channel number and spatial size. ; Specifically, including: 1) Definition of multimodal input The input set of synchronous heterogeneous data streams can be represented as: , Among them, single-channel SAR imagery (amplitude feature channel): Preserve the denoised radar backscattering characteristics; Three-channel optical RGB imaging: Preserve the semantic features of visible light surface texture, color, and outline; Single-channel infrared thermal radiation image: Preserve the physical characteristics of temperature radiation of ground features.

[0033] All modal data have been spatially aligned using S1, maintaining a uniform height H and width W.

[0034] 2) Forward inference using shared backbone network features; Three types of modal data are input in parallel into a shared backbone network. The network adopts a lightweight convolutional architecture (adapted to the high-time-efficiency inference requirements of remote sensing images). Basic feature extraction operations such as convolution, pooling, and activation are performed on each modality of data. The network does not distinguish between modal types and shares all network parameters. The final output is a primary feature map with uniform channel count and spatially aligned dimensions. ; in, The height and width of the feature map after downsampling are strictly aligned with the input image space; To share the unified number of feature channels output by the backbone network; This includes primary general features encompassing SAR scattering characteristics, optical texture characteristics, and infrared radiation characteristics. It is a multimodal shared backbone network.

[0035] S2-2, In the modal adaptation layer, through independent Convolutional layers for primary features Perform linear transformation and reconstruction along the channel dimension to complete the semantic space projection of cross-modal features and output channel-aligned transition features; 1×1 convolution does not change the feature map spatial size, but only performs linear transformation in the channel dimension. It reconstructs the primary features of SAR, optical, and infrared in the channel dimension, and uniformly adapts to the input channel specifications of subsequent networks. It learns the linear weights of cross-modal features to realize the semantic association of features from different sensors. It is computationally lightweight with no additional latency overhead, and adapts to the needs of high-time-efficiency intelligence generation.

[0036] Primary features of three modes Parallel input independent Convolutional layers (without sharing parameters between modalities, preserving modal-specific features) perform feature projection transformations and output channel-aligned transition features, providing a foundation for subsequent normalization constraints.

[0037] S2-3. In the modal adaptation layer, for The features after convolutional projection are normalized by a layer. By standardizing the mean and variance, combined with learnable gain and bias parameters, differences in radiometric calibration between sensors are eliminated, and standardized adaptive features with cross-modal semantic alignment are output. ; Modal adaptation layer through Convolution and layer normalization (LayerNorm) achieve cross-modal semantic alignment, and its mathematical model is as follows: ; in, for Convolution operation, and These represent the mean and variance of the feature map along the channel dimension, respectively. To prevent division by zero of extremely small constants; and These are learnable gain and bias parameters used to force a constrained characteristic distribution and eliminate radiometric calibration differences between sensors. This is the final output of cross-modal standardized adaptation features.

[0038] Specifically, S3 includes: S3-1. Extract geographic coordinate information from the current image-related metadata and generate a latitude and longitude grid index that matches the feature fingerprint database according to preset rules. From the image metadata associated with the standardized adaptation features processed by S2, accurately extract the geographic coordinate information of the current remote sensing image. This information includes the latitude and longitude bounding boxes of the image coverage area. Geographic projection parameters and pixel-to-geographic coordinate mapping relationships; based on preset equal-precision latitude and longitude grid division rules, the latitude and longitude bounding boxes of the current image are defined. Matching to the corresponding latitude and longitude grid cells generates a unique geospatial grid index. This index is completely consistent with the storage index rules of historical features in the feature fingerprint database, providing a geolocation basis for subsequent accurate addressing.

[0039] S3-2. Using latitude and longitude grid index as the key, accurately retrieve and call up the matching historical baseline feature tensor from the memory feature fingerprint database through an asynchronous thread; Using the latitude and longitude grid index as the search keyword, the asynchronous addressing thread of the memory feature fingerprint database is triggered. The thread runs independently of the main computing process. Based on the index, it quickly matches and extracts the historical baseline feature tensor that completely corresponds to the geographic coverage of the current image in memory. The entire search process is a pure memory operation, without time-consuming steps such as disk I / O reading and image decoding, achieving millisecond-level retrieval of historical baseline features. At the same time, the asynchronous addressing mechanism avoids blocking the subsequent main processes such as feature alignment and fusion during the search process, ensuring the overall timeliness of intelligence generation. After the search is completed, the historical baseline feature tensor is pushed to the designated cache area of ​​the main process, waiting for subsequent processing.

[0040] The feature fingerprint stored in memory is a compressed high-dimensional feature obtained by processing historical images through the same shared backbone network and modality adaptation layer as in S2, and then using multiple downsampling. The historical baseline feature tensors stored in the feature fingerprint database are high-dimensional feature representations obtained by extracting primary features from historical remote sensing images through a shared backbone network, normalizing them through a modality adaptation layer, and then performing downsampling by 8x or 16x. The downsampling operation only compresses the spatial dimension of the feature map proportionally, fully preserving the semantic feature information of the channel dimension. After compression, the data volume of this feature fingerprint is only 1%-5% of the original pixel image, significantly reducing memory storage overhead. Moreover, it is pre-resident in the memory database, which is different from the traditional approach of storing historical images in the form of original pixel files on disk. This completely avoids the huge I / O latency caused by reading and decoding GB-level original pixel files, providing data throughput support for minute-level closed loops of intelligence generation.

[0041] Specifically, S4 includes: S4-1. Extract rational polynomial coefficients (RPC) parameters containing satellite imaging geometric information from the current image's native metadata. Accurately extract rational polynomial coefficients (RPC) parameters from the original metadata files of current remote sensing images during the satellite imaging process. The RPC parameters contain core physical imaging information such as satellite orbital attitude, camera imaging geometry, and sensor physical characteristics. They are key prior data characterizing the spatial geometric relationships during satellite imaging, providing precise physical constraints for subsequent geometric transformations of the feature space. Unlike purely vision-driven alignment methods, this significantly improves the accuracy and robustness of transformation matrix regression.

[0042] S4-2. After the RPC parameters are characterized, encoded and normalized, they are cross-domain spliced ​​and fused with the current standardized adaptive feature stream in the channel dimension to obtain fused features that have both semantic and physical prior knowledge. Extracted RPC parameters After feature encoding, it is adapted to the current standardized features. Perform cross-domain splicing and fusion at the channel level; the fusion expression is: ; in, This indicates a channel-level concatenation operation; First, the RPC parameters are expanded in dimension and normalized numerically to ensure consistency with the spatial size and numerical distribution of the current feature stream. Then, cross-domain splicing is performed to fuse the features. It simultaneously incorporates the semantic features of images and the physical geometric features of satellite imaging, providing a fusion input for spatial transformation networks that combines visual semantics and physical priors.

[0043] S4-3. Input the fused features into the localization network of the spatial transformation network, regress to obtain the six-degree-of-freedom transformation parameters, and construct the corresponding affine transformation matrix; Cross-domain integration features Input lightweight positioning network (Core component of the spatial transformation network) The six-degree-of-freedom spatial transformation parameters are obtained through adaptive learning and regression by the localization network. The parametric regression expression is: ; Transformation parameters obtained from regression Furthermore, an affine transformation matrix is ​​constructed to describe the spatial geometric mapping relationship between the current feature and the historical baseline feature. The matrix expression is: ; This affine transformation matrix can accurately characterize the geometric transformation operations such as translation, scaling, and rotation that need to be performed on historical baseline features, thereby enabling the quantification and correction of imaging geometric differences.

[0044] S4-4. Calculate the sampling coordinates of historical baseline features based on the affine transformation matrix, and complete sub-pixel level resampling of the feature domain through bilinear interpolation to obtain historical features aligned with the current feature stream; The completed affine transformation matrix Historical baseline features retrieved from S3 To perform feature domain spatial distortion resampling, first calculate the original coordinates of each point in the historical baseline features based on the matrix. Corresponding transformed sampled coordinates The coordinate transformation formula is: ; Then, a bilinear interpolation operator is used to perform sub-pixel level interpolation calculations on the transformed sampling coordinates. By weighted fusion of neighborhood features, accurate feature values ​​at the sampling coordinates are obtained. Finally, the spatial geometric correction of the historical baseline features is completed, resulting in historical features that are perfectly aligned with the current feature flow. The formula for calculating the alignment feature is: ; in, Characteristics of historical baselines, This is a bilinear interpolation operation.

[0045] This process directly performs sub-pixel alignment in the feature domain without returning to the original pixel level, which ensures alignment accuracy (registration accuracy less than 0.5 pixels) while significantly reducing computational overhead and effectively filtering out spurious change signals caused by imaging distortion, shadow displacement, etc.

[0046] Specifically, in S5, Dynamic Channel Attention Fusion (DCAF) multimodal feature adaptive weighted fusion includes: S5-1. Perform global average pooling on the standardized adaptation features of each modality, and extract a one-dimensional channel descriptor that represents the overall quality of the modality through spatial feature compression. Global average pooling (GAP) is performed on the normalized adaptation feature streams of SAR, optical, and infrared modes output by S2. Through spatial dimension feature compression, channel descriptors that can characterize the overall quality and semantic information of each mode are extracted. The calculation formula is as follows: ; in, and These represent the height and width of the feature map, respectively. For position eigenvalues ​​at that location This is the channel descriptor.

[0047] This operation compresses the high-dimensional spatial feature map into a one-dimensional channel vector, discarding redundant spatial details and retaining the core features that can reflect the overall imaging quality of the modality (such as sharpness, signal-to-noise ratio, and noise interference level), providing a precise basis for quality perception for subsequent dynamic weight learning.

[0048] S5-2. Input the channel descriptor into the lightweight bottleneck structure MLP, and learn the cross-modal dynamic weight vector in the 0~1 interval after nonlinear transformation and normalization. The channel descriptors zc extracted from each modality are input into a bottleneck-structured multilayer perceptron (MLP). A lightweight neural network learns dynamic activation weights across modalities to achieve feature competition and adaptive activation between modalities. The weight learning formula is as follows: ; in, and The learnable weight matrix for a multilayer perceptron (MLP) (the bottleneck structure achieves lightweight computation by reducing the dimensions of the intermediate layers, adapting to high timeliness requirements). It is a ReLU activation function (which realizes nonlinear transformation of features and enhances the expressive power of weight learning). The Sigmoid activation function (maps weight values ​​to the 0~1 range, achieving normalization constraints on modal weights). The learned dynamic weight vector contains weight coefficients that correspond one-to-one with the number of modalities, each representing the feature contribution of each modality in the current fusion scenario. The entire weight learning process is an end-to-end adaptive computation that requires no manual intervention and can update the weights in real time based on the quality status of the input features.

[0049] S5-3. Perform channel-level weighted fusion of each modal feature based on dynamic weight vector, adaptively adjust the weight coefficient according to the modal signal-to-noise ratio, and output an enhanced fusion feature stream that integrates the advantages of each modality.

[0050] Based on the dynamic weight vector W obtained by MLP learning, channel-level weighted summation and fusion are performed on the standardized adaptation features of SAR, optical, and infrared modes. The fusion weights are then adaptively and dynamically adjusted based on the actual signal-to-noise ratio of each mode. The weighted fusion formula is as follows: ; in, Standardized adaptation features for SAR, optical, and infrared modes. Let be the dynamic fusion weight coefficients for each modality, and satisfy . The normalization constraint ensures that there is no excessive amplification or attenuation of feature information during the fusion process.

[0051] This step features environmentally adaptive weight adjustment: when the optical mode is affected by factors such as clouds, nighttime, or uneven lighting, resulting in decreased image quality and signal-to-noise ratio, the system will automatically reduce the optical mode weight. Simultaneously enhance the SAR mode weights, which possess strong penetration and are unaffected by weather and lighting conditions. When the optical modal imaging quality is good, a high level of retention is achieved. This fully leverages its semantic richness and clear texture; the infrared mode weights are adjusted in real time based on the effectiveness of the thermal radiation characteristics.

[0052] The fusion operation executes in less than 3ms on the GPU, achieving millisecond-level heterogeneous modal information complementarity, and ultimately outputting an enhanced fused feature stream. It integrates the core advantages of various modalities, providing high-quality and robust unified feature inputs for subsequent interpretation tasks.

[0053] Specifically, S6 includes: S6-1. Three interpreters—detection, residual change detection, and scene—are run in parallel on the fused feature stream. The real change signal is enhanced through feature difference, ROI mask, and LeakyReLU activation gate, and multi-dimensional interpreter features are output. Enhanced fusion feature stream output by S5 The system employs three lightweight, decoupled sub-networks deployed in parallel: a detection head, a residual change detection head, and a scene head. Each sub-network infers independently while sharing the fused feature stream, eliminating redundant computation. Simultaneously, the residual change detection head utilizes feature differencing and activation gate operators to accurately enhance the real change signal and effectively suppress noise. The core processing logic is as follows: 1) Parallel inference with multiple interpreters: The detection head adopts decoupled classification and regression branches, focusing on extracting precise category, spatial coordinates and other information of ground objects; the scene head uses a large receptive field convolutional layer to determine the macro scene attributes of the image coverage area (such as disaster-affected areas, port operation areas, urban built-up areas, etc.); the residual change detection head is specifically designed to accurately identify the evolution of ground objects, providing core basis for change intelligence extraction.

[0054] 2) Core calculation for residual change detection: First, based on the aligned historical features obtained from S4... Calculate its fusion feature flow with the current fusion feature flow. The element-wise absolute difference is then compared with the region of interest mask. Perform element-wise multiplication, retaining only the feature difference information within the ROI ( (Foreground is set to 1, background to 0, to mask interference from meaningless background regions); finally, an activation gate is constructed using the LeakyReLU activation function (negative slope coefficient α=0.1) to enhance the true change signal and preserve subtle evolutionary features. The calculation formula is as follows: ; ; in, To enhance the feature map of changes, this process directly performs change detection in the feature domain, effectively filtering out spurious change signals caused by geometric distortion and noise, and improving the accuracy of change recognition.

[0055] S6-2. Integrate the inference results of multiple interpreters to extract structured interpreter vectors, associate them with geographic coordinate information, encapsulate them into standardized input through prompt word templates, and inject them into the fine-tuned T5-Small model; By integrating the parallel inference results of the detection head, residual change detection head, and scene head, a structured derivation vector containing core information such as target category / location, change type / area / range, and regional scene attributes is extracted. Simultaneously, it correlates with the geographic coordinate information extracted by S3. (Latitude and longitude, geographical range, etc.) A function is constructed using a preset prompt template. The structured vectors and geographic coordinate information are encapsulated into standardized natural language prompts according to fixed logic, and then injected into a T5-Small pre-trained language model that has been fine-tuned based on remote sensing scenes. The core expression is: ; Among them, prompt word template Designed based on the decision-making needs of remote sensing data, it realizes the standardized conversion of structured data to natural language input (such as "Detected [target category] Quantity: [X], Location: [latitude and longitude / geographic region], Change attribute: [new / disappear / deformation], Belonging scene: [macro scene type]"), ensuring that the T5-Small model can accurately parse and interpret the semantic logic of the information.

[0056] S6-3: The T5-Small model automatically transforms standardized input into logically coherent natural language intelligence reports, completing end-to-end intelligence generation and output. Based on standardized input prompts, the T5-Small model automatically generates structured information into logically coherent, complete, and decision-oriented natural language intelligence reports. The generated reports include key content such as the geographic extent of the image, details of detected targets, core information on changes in ground features, and regional scene attributes, eliminating the need for manual "image-based" report writing. Simultaneously, the generated intelligence reports can be directly pushed to the user's device, achieving a minute-level end-to-end closed loop from multi-source remote sensing data access to natural language intelligence output. This fully meets the high-timeliness and automation requirements of remote sensing intelligence in scenarios such as disaster prevention and mitigation, and national defense security.

[0057] The method in this embodiment can achieve the following effects: (1) Significantly reduces computational load and memory overhead, and improves edge adaptability. The method in this embodiment adopts a second-order integral graph acceleration operator and a heterogeneous shared-backbone architecture.

[0058] In the preprocessing stage, the second-order integral image simplifies the convolution calculation, which originally grows quadratically with the window size N, into a constant-time operation (4 table lookups). Combined with the parallelism of the SIMD instruction set, this speeds up the most time-consuming denoising process by about 8 times.

[0059] In the feature extraction stage, the traditional siloed layout of "one task, one model" has been changed. By normalizing data from different sensors through a modality adaptation layer, multiple task branches such as target detection, change detection, and scene classification can share the same set of backbone network parameters.

[0060] Experimental verification: Experimental environment: Ubuntu 20.04, PyTorch 2.1, CUDA 11.8, hardware platform is NVIDIA Jetson AGX Orin edge computing terminal (32GB video memory).

[0061] Test dataset: DIOR remote sensing target detection dataset.

[0062] Implementation: In this method, Layers 1-3 are deployed as a shared feature extraction layer, and the three interpreter heads after Layer 5 are driven in parallel by 1x1 convolutional adaptation layers. The control group uses three independent models (YOLOv8-s, Siam-ResNet, and ResNet-18) for sequential loading.

[0063] Experimental results:

[0064] Conclusion: Experimental data shows that, when handling tasks of the same scale, this invention significantly reduces memory usage by 65% ​​and increases inference speed by approximately 3 times. This enables interpretation tasks that previously required high-performance workstations to be smoothly deployed on memory-constrained edge computing nodes.

[0065] (2) Eliminate I / O throughput bottlenecks and achieve intelligence output in minutes. A feature fingerprint database residency mechanism based on geographic coordinate indexing and asynchronous feature addressing are adopted.

[0066] The method in this embodiment abandons the traditional mode of loading GB-level raw pixel files, and instead stores high-dimensional feature tensors that have been processed by a shared backbone and downsampled by 8 or 16 times.

[0067] Because feature fingerprints are highly compressed semantic representations, their data size is only 1% to 5% of the original image. When executing tasks, the system retrieves features directly from memory addresses based on latitude and longitude bounding boxes via asynchronous threads, eliminating the need for cumbersome image decoding and disk reads.

[0068] Experimental Design: Experimental subject: GF-2 satellite remote sensing imagery covering a 50km × 50km area (single temporal raw file approximately 1.8GB).

[0069] Comparison group (traditional mode): After receiving the decoding command, the system synchronously reads two sets of raw pixel files, the current time phase and the historical baseline time phase, from the disk, decodes them, and sends them into memory.

[0070] Experimental Group (this embodiment): Historical baseline data is pre-residing in the memory database in the form of feature fingerprints. The system uses the current image's latitude and longitude bounding box and geographic coordinate information... The corresponding feature block can be retrieved directly from the memory address in seconds through an asynchronous addressing thread, without the need for disk I / O.

[0071] Experimental results: The data loading time of the system was reduced from over 40% in the traditional process to less than 2%. For routine-scale remote sensing dynamic monitoring tasks, the entire intelligence generation cycle was successfully reduced from hours to less than minutes.

[0072] (3) Effectively suppresses geometric distortion interference, significantly improving intelligence accuracy and robustness. This embodiment introduces an RPC metadata-guided spatial transformation network (STN) and dynamic channel attention fusion (DCAF).

[0073] Traditional algorithms struggle to handle feature drift caused by large oblique angle satellite imaging. This invention directly reads RPC parameters from image metadata to guide STN regression, performing subpixel-level geometric resampling at the feature layer. This ensures that the current image and historical baselines are absolutely "overlapping" in feature space, eliminating false alarms caused by shadows and displacement. DCAF uses the bottleneck MLP to sense the modal signal-to-noise ratio in real time. When optical image quality degrades due to cloud cover, fog, or nighttime environmental interference, the system automatically suppresses optical channel weights and simultaneously excites highly penetrating SAR modal features.

[0074] Experimental Design: Experimental data: 500 pairs of heterogeneous images with significant differences in shooting angle were selected from AIRSAT's own satellite resource pool.

[0075] Contrast Group A (without RPC guidance): Alignment is performed using traditional SIFT feature point matching or purely vision-driven STN.

[0076] Comparison Group B (Linear Fusion): Simple pixel-level weighted fusion was used, with each modality weight fixed at 0.5.

[0077] Experimental group (this example): Enable RPC-guided STN alignment and DCAF dynamic weight awareness.

[0078] Experimental results:

[0079] Conclusion: Under complex imaging conditions, this scheme reduces the false alarm rate of change detection by approximately 70%. Target identification accuracy at night or in adverse weather conditions is improved from around 70% with traditional schemes to over 90%, significantly enhancing the decision-making value of intelligence.

[0080] In summary, the method in this embodiment addresses three core pain points in remote sensing intelligence production: I / O blocking, geometric mismatch, and computational redundancy, through a fundamental logical transformation of "feature-based image substitution" combined with deep learning operators guided by physical imaging mechanisms. It employs a heterogeneous shared backbone network, enabling multiple tasks to share convolutional parameters, reducing memory usage by approximately 65%, increasing inference speed by approximately 3 times, and achieving smooth deployment at the edge. A feature fingerprint database based on geographic coordinate indexes compresses data volume to 1%-5% of the original image, reducing throughput by over 95%, and shortening the intelligence generation cycle from hours to within minutes. The introduction of a satellite RPC parameter-guided spatial transformation network and dynamic channel attention fusion reduces the false alarm rate of change detection by approximately 70%, and improves target recognition accuracy to over 90% in complex environments. Through deep integration of a parallel interpretation head and a pre-trained language model, it achieves an end-to-end automated closed loop from data access to report generation, completely eliminating latency caused by manual intervention.

[0081] Example 2 Another embodiment of the present invention discloses an information generation system based on feature fingerprint storage and spatiotemporal geometric correction, such as... Figure 2 As shown, it includes: The parallel access module is used to asynchronously access multi-source remote sensing images for parallel preprocessing and generate a time-synchronized heterogeneous data stream. The shared feature extraction module is used to input heterogeneous data streams into a shared backbone network + modality adaptation layer, extract and normalize features, and generate the current feature stream in a unified semantic space. The feature fingerprint storage module is used to asynchronously retrieve historical baseline features from the memory feature fingerprint library based on the current image geographic coordinates. The historical baseline features are compressed high-dimensional features of historical images stored in the library after being processed by a shared backbone network, a modality adaptation layer, and multiple downsampling. The spatiotemporal geometric correction module is used to fuse satellite imaging parameters in image metadata with the current feature stream and input them into a spatial transformation network. The transformation matrix is ​​obtained by regression, and the historical baseline features are resampled using this matrix to achieve spatiotemporal geometric correction in the feature domain, thereby obtaining historical features that are spatially aligned with the current feature stream. The dynamic fusion module is used to dynamically calculate the fusion weights based on the quality differences of each modality, and to weight and fuse the current feature streams of each modality to generate a fused feature stream. The parallel interpretation module is used to perform parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report.

[0082] In this embodiment, the more specific technical details and beneficial effects are the same as those disclosed in Embodiment 1. Please refer to them for details, and they will not be repeated here.

[0083] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligence generation method based on feature fingerprint storage and spatiotemporal geometric correction, characterized in that, include: S1. Asynchronously access multi-source remote sensing images for parallel preprocessing to generate a time-synchronized heterogeneous data stream; S2. Input the heterogeneous data stream into the shared backbone network + modality adaptation layer, extract and normalize the features, and generate the current feature stream in a unified semantic space; S3. Based on the current image geographic coordinates, asynchronously retrieve historical baseline features from the memory feature fingerprint database. The historical baseline features are compressed high-dimensional features stored in the database after historical images have been processed by a shared backbone network, modality adaptation layer, and multiple downsampling. S4. After fusing the satellite imaging parameters in the image metadata with the current feature stream, input the result into the spatial transformation network and regress to obtain the transformation matrix. Use this matrix to resample the historical baseline features to achieve spatiotemporal geometric correction of the feature domain and obtain historical features that are spatially aligned with the current feature stream. S5. Dynamically calculate the fusion weights based on the quality differences of each modality, and then weight and fuse the current feature flows of each modality to generate a fused feature flow. S6. Perform parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report.

2. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 1, characterized in that, S2 includes: S2-1. Input the heterogeneous data streams of synchronous SAR, optical, and infrared output from S1 into a multimodal shared backbone network in parallel to extract primary features with uniform channel number and spatial size. ; S2-2, In the modal adaptation layer, through independent Convolutional layers for primary features Perform linear transformation and reconstruction along the channel dimension to complete the semantic space projection of cross-modal features and output channel-aligned transition features; S2-3. In the modal adaptation layer, for The features after convolutional projection are normalized by a layer. By standardizing the mean and variance, combined with learnable gain and bias parameters, differences in radiometric calibration between sensors are eliminated, and standardized adaptive features with cross-modal semantic alignment are output. .

3. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 2, characterized in that, Standardized adaptation features for cross-modal semantic alignment The expression is: ； in, for Convolution operation, and These represent the mean and variance of the feature map along the channel dimension, respectively. To prevent division by zero of extremely small constants; and These are learnable gain and bias parameters; ； Among them, synchronous heterogeneous data streams , This is a single-channel SAR image; It is a three-channel optical RGB image; This is a single-channel infrared thermal radiation image; It is a multimodal shared backbone network.

4. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 2, characterized in that, S3 includes: S3-1. Extract geographic coordinate information from the current image-related metadata and generate a latitude and longitude grid index that matches the feature fingerprint database according to preset rules. S3-2. Using latitude and longitude grid index as the key, accurately retrieve and call up the matching historical baseline feature tensor from the memory feature fingerprint database through an asynchronous thread; The feature fingerprint stored in memory is a compressed high-dimensional feature obtained by processing historical images through the same shared backbone network and modality adaptation layer as in S2, and then using multiple downsampling.

5. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 4, characterized in that, S4 includes: S4-1. Extract rational polynomial coefficients (RPC) parameters containing satellite imaging geometric information from the current image's native metadata. S4-2. After the RPC parameters are characterized, encoded and normalized, they are cross-domain spliced ​​and fused with the current standardized adaptive feature stream in the channel dimension to obtain fused features that have both semantic and physical prior knowledge. S4-3. Input the fused features into the localization network of the spatial transformation network, regress to obtain the six-degree-of-freedom transformation parameters, and construct the corresponding affine transformation matrix; S4-4. Calculate the sampling coordinates of historical baseline features based on the affine transformation matrix, and complete sub-pixel level resampling of the feature domain through bilinear interpolation to obtain historical features aligned with the current feature stream.

6. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 5, characterized in that, The historical features aligned with the current feature stream are: ； Characteristics of historical baselines, This is a bilinear interpolation operation. It is the affine transformation matrix; These are the six-degree-of-freedom transformation parameters obtained from the regression. ; To locate the network, , for the current feature flow With RPC parameters Cross-domain splicing and fusion representation. This indicates a channel dimension splicing operation.

7. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 5, characterized in that, S5 includes: S5-1. Perform global average pooling on the standardized adaptation features of each modality, and extract a one-dimensional channel descriptor that represents the overall quality of the modality through spatial feature compression. S5-2. Input the channel descriptor into the lightweight bottleneck structure MLP, and learn the cross-modal dynamic weight vector in the 0~1 interval after nonlinear transformation and normalization. S5-3. Perform channel-level weighted fusion of each modal feature based on dynamic weight vector, adaptively adjust the weight coefficient according to the modal signal-to-noise ratio, and output an enhanced fusion feature stream that integrates the advantages of each modality.

8. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to claim 7, characterized in that, S6 includes: S6-1. Three interpreters—detection, residual change detection, and scene—are run in parallel on the fused feature stream. The real change signal is enhanced through feature difference, ROI mask, and LeakyReLU activation gate, and multi-dimensional interpreter features are output. S6-2. Integrate the inference results of multiple interpreters to extract structured interpreter vectors, associate them with geographic coordinate information, encapsulate them into standardized input through prompt word templates, and inject them into the fine-tuned T5-Small model; S6-3: The T5-Small model automatically transforms standardized input into logically coherent natural language intelligence reports, completing end-to-end intelligence generation and output.

9. The information generation method based on feature fingerprint storage and spatiotemporal geometric correction according to any one of claims 1-8, characterized in that, S1 includes: S1-1: Input synthetic aperture radar single-channel complex data, optical three-channel RGB data and infrared single-channel thermal radiation data, use a multi-thread pool to allocate independent computing resources for each mode and pull image slices in real time through asynchronous I / O mechanism, and output multi-source raw image data streams received in parallel. S1-2. Perform directional denoising processing based on second-order integral map on SAR modal data in multi-source raw image data stream, and transmit optical and infrared modal data in parallel to output multi-source image data stream after parallel denoising. S1-3. For the multi-source image data stream after parallel denoising, the denoising weight calculation is accelerated at the instruction level using the single instruction multiple data stream instruction set. The barrier synchronization mechanism ensures that the data of each modality is synchronized and ready, and outputs a synchronized heterogeneous data stream.

10. An intelligence generation system implementing the intelligence generation method based on feature fingerprint storage and spatiotemporal geometric correction as described in any one of claims 1-9, characterized in that, include: The parallel access module is used to asynchronously access multi-source remote sensing images for parallel preprocessing and generate a time-synchronized heterogeneous data stream. The shared feature extraction module is used to input heterogeneous data streams into a shared backbone network + modality adaptation layer, extract and normalize features, and generate the current feature stream in a unified semantic space. The feature fingerprint storage module is used to asynchronously retrieve historical baseline features from the memory feature fingerprint library based on the current image geographic coordinates. The historical baseline features are compressed high-dimensional features of historical images stored in the library after being processed by a shared backbone network, a modality adaptation layer, and multiple downsampling. The spatiotemporal geometric correction module is used to fuse satellite imaging parameters in image metadata with the current feature stream and input them into a spatial transformation network. The transformation matrix is ​​obtained by regression, and the historical baseline features are resampled using this matrix to achieve spatiotemporal geometric correction in the feature domain, thereby obtaining historical features that are spatially aligned with the current feature stream. The dynamic fusion module is used to dynamically calculate the fusion weights based on the quality differences of each modality, and to weight and fuse the current feature streams of each modality to generate a fused feature stream. The parallel interpretation module is used to perform parallel interpretation and semantic encapsulation of the fused feature stream to generate a natural language intelligence report.

Images