A spatio-temporal aliasing event detection method for wide-area compression observation

Abstract

A space-time aliasing event detection method for wide-area compression observation; first, a double-path time sequence feature pyramid network is constructed, and multi-resolution time sequence representation and dense candidate queries are generated through cross-scale time sequence feature fusion to uniformly cover long-time continuous events and short-time transient events; second, a multi-label differentiated query generation mechanism based on uncertainty and time sequence overlap intensity joint constraint is proposed to avoid premature suppression of effective instances in high overlap areas; in the decoding stage, a deformable cross attention mechanism is introduced to realize dynamic feature aggregation and cross-scale modeling of sparse and dense events through adaptive time slice sampling and key time sequence alignment; a double-branch collaborative structure of sparse instance-level prediction and dense frame-level prediction is designed to alleviate the uneven coverage problem of the query mechanism under extreme event distribution; through the above technical cooperation, the robustness and consistency of multi-event boundary positioning and category recognition in complex long video scenes are significantly improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a spatiotemporal aliasing event detection method for wide-area compressed observation. By constructing full-scale semantic information, generating differentiated queries based on multi-label perception, and performing dense-sparse collaborative prediction, it improves the event detection and robustness of the method in complex scenarios such as coexistence of extreme duration spans, fine-grained differentiation of multiple events, and suppression of weak signals. Background Technology

[0002] Multi-event detection is a fundamental and crucial computer vision task. With the rapid development of wide-angle optical imaging technology, all-sky cameras, panoramic cameras, and other wide-angle imaging devices are widely used in space weather monitoring, meteorological environment assessment, and large-scale airspace surveillance. Long-term video sequences acquired by wide-angle imaging devices often contain various physical events or target patterns with different spatial scales and time spans (e.g., transient bright structures with clear spatial boundaries, and long-term diffuse background events with blurred spatial boundaries and low contrast). Fine-grained, high-precision temporal boundary localization and collaborative identification of these multiple events within a large field of view have significant engineering and scientific value for achieving all-day meteorological monitoring or quantitative analysis of the airspace environment.

[0003] However, in wide-angle imaging systems, due to the inherent characteristics of lenses with large fields of view or long-distance imaging, the system acquisition process faces the following three core imaging mechanism features and underlying technical bottlenecks:

[0004] 1) Spatial Scale Compression Coding Degradation: Wide-angle fields of view integrate an extremely wide spatial range within a single image plane, resulting in the presence of multiple physical targets at different spatial scales within the same image. Distant targets have a very small pixel count in the image, exhibiting weak feature representation; while near-distance targets have a large pixel count, exhibiting strong feature representation, and both are intermingled with mid-scale targets in the field of view. This spatial scale compression coding phenomenon makes it difficult for traditional single-receptive-field networks to balance feature extraction from targets at all scales.

[0005] 2) Projection and Overlap of Temporal Events: Due to the extremely large field of view of wide-angle imaging systems, multiple physically independent events typically occur simultaneously at different spatial locations within the same time window. When these multi-source concurrent events are projected onto the same video sequence, highly overlapping concurrent observations are formed on the timeline. This projection and overlap of multiple events on the same time stream causes traditional single-label forced normalization processing or class-independent non-maximum suppression (NMS) filtering mechanisms to directly fail, easily introducing systematic missed detections and biases.

[0006] 3) Spatiotemporal Attribute Coupling Interference: In wide-angle imaging, there is a severe coupling degradation among the spatial scale, temporal duration, and multi-target density of the target. Specifically, long-range concurrent events typically exhibit "small spatial scale and long duration," which are easily ignored in the feature space; while short-range concurrent events exhibit "large spatial scale and short transientity," often dominating the training and inference processes, leading to an imbalance in the competition for targets. Simultaneously, the coexistence of high-density spatiotemporal aliasing events easily induces mutual occlusion of radiation components and optical artifact interference.

[0007] In summary, overcoming the negative impacts of spatial compression, temporal overlay, and spatiotemporal observation coupling in wide-angle imaging systems, achieving full-scale temporal boundary modeling in untrimmed long-term wide-angle videos, and completing high-precision, low-redundancy, and fine-grained multi-label temporal event detection in highly overlapping and coexisting scenarios is a technical problem that urgently needs to be studied. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a spatiotemporal aliasing event detection method for wide-area compressed observation. By using a dual-path temporal feature pyramid and a multi-label perceptual differential query generation module, the model can accurately filter the queries corresponding to the events, thereby significantly improving the multi-event detection performance for spatiotemporal aliasing.

[0009] To achieve the above objectives, the technical solution adopted in this method is: a spatiotemporal aliasing event detection method for wide-area compressed observation, comprising the following steps:

[0010] Step 1: Perform grayscale compression and size normalization preprocessing on each image: linearly compress the grayscale to a preset grayscale range; scale the image to a preset size to form a standard image set. ,in The total number of frames in the image sequence is used to obtain the preprocessed video sequence images.

[0011] Step 2: Input the preprocessed video sequence images from Step 1 into a dilated 3D convolutional network for feature extraction to obtain initial temporal features. Semantic features are obtained by constructing a dual-path temporal feature pyramid consisting of a bottom-up time-scale ascending path and a top-down time-scale recovery path; wherein, the dual-path temporal feature pyramid includes:

[0012] 1) Bottom-up path: First, the feature dimension is reduced by one-dimensional convolution. Compress to Degree, then through Subsequent downsampling to construct Layered temporal feature pyramid ,in, It is a bottom-up path time series feature pyramid. The first pyramid representing the time series features of the bottom-up path Layer features;

[0013] 2) Top-down path: To begin, Layer features are upsampled to After the layers are of the same size, with the first Temporal semantic fusion is performed on layer features;

[0014] 3) The fused features from each layer are channel-aligned through pointwise convolution, and then smoothed through one-dimensional temporal convolution to construct... Layered Refined Temporal Feature Pyramid ,in, It is a refined time-series feature pyramid. It is the first of the top-down path time series feature pyramids Layer features;

[0015] Step 3, in the refined time-series feature pyramid Generate candidate queries at each layer and at each time series. And for any two queries Perform dual-condition filtering to create differentiated queries. Among them, dual-condition filtering includes:

[0016] 1) The temporal intersection-union ratio of the two queries is greater than the threshold;

[0017] 2) The two queries have the same category;

[0018] 3) When both 1) and 2) are satisfied, suppress one of the queries;

[0019] Step 4, perform the differentiated query With the refined time-series feature pyramid Attention calculations are performed to obtain the frame-level multi-class prediction probability matrix. Refined temporal feature pyramid Layer 0 features Feature extraction is performed to obtain a frame-by-frame multi-class prediction probability matrix. Finally and Linear weighted fusion is performed to obtain the final frame-level multi-class prediction output. Weighted fusion refers to:

[0020]

[0021] in, For the first image in a video sequence frame, To adjust the fusion weights, control and The relative contribution ratio to the final output, or the final frame-level multi-class prediction output obtained through other equivalent transformations of the frame-level multi-class prediction probability matrix and the frame-by-frame multi-class prediction probability matrix.

[0022] Step 1, the preprocessing includes the following steps:

[0023] Step 1a: Compress the 16-bit grayscale range of the original image to an 8-bit grayscale range;

[0024] Step 1b, will The original image of pixels is uniformly scaled to A standard image of pixels.

[0025] Step 2, the process of constructing the dual-path temporal feature pyramid, includes the following steps:

[0026] Step 2a, the initial timing features Channel dimensionality reduction is performed using a one-dimensional convolution with a kernel size of 1, reducing the feature dimension from... Compress to ;

[0027] Step 2b, bottom-up path through One-dimensional max pooling with a step size of 2 is used for step-by-step downsampling to construct... Layer-by-layer temporal feature pyramid, the first Layer feature length ;

[0028] Step 2c, top-down path from Begin, the first Layer features are upsampled to the th layer using bilinear interpolation. After the layers are of the same size, according to Add element-wise the corresponding layer features;

[0029] Step 2d: The fused features from each layer are channel-aligned using a pointwise convolution with a kernel size of 1, and then smoothed using a one-dimensional temporal convolution with a kernel size of 3, thus constructing... Layered Refined Temporal Feature Pyramid The construction of the dual-path temporal feature pyramid was completed.

[0030] In step 2, the temporal semantic fusion may take the form of element-wise addition, but is not limited to: ,in, It is the first of the top-down path time series feature pyramids Layer features, It is the first of the bottom-up path time series feature pyramid Layer features, Features Perform upsampling.

[0031] Step 3, which involves performing dual-condition filtering on any two queries, includes the following steps:

[0032] Step 3a: First, calculate the predicted label set for each candidate query. ,in, It is the first Predicted labels for each candidate query. It is an index of action categories. This represents the sigmoid activation function. It is the first Candidate queries in The original predicted score on the category, It is the category confidence threshold;

[0033] Step 3b: Suppress queries with a temporal intersection-union ratio higher than the threshold and the same predicted label:

[0034]

[0035] in, It is the first and the One query, It is a time intersection comparison screening threshold. Used to determine whether a condition is true or false. Used for calculation and Time intersection and union comparison;

[0036] Step 3c: Reorder the retained differential queries to form a differential query set. .

[0037] Compared with existing technologies, the spatiotemporal aliasing event detection method for wide-area compressed observation provided by this invention brings the following significant benefits by introducing a dual-path temporal pyramid module, a multi-label perceptual differential query generation module, and a dense-sparse collaborative head:

[0038] 1) Significantly improved the temporal boundary localization accuracy of events across all scales:

[0039] This invention constructs a dual-path temporal feature pyramid consisting of a bottom-up, time-scale-increasing path and a top-down, time-scale-recovery path. The bottom-up path captures global event features at large temporal scales through step-by-step downsampling, while the top-down path transfers strong semantic information from higher layers to lower layers through temporal semantic fusion and utilizes one-dimensional temporal convolution for smoothing, effectively eliminating temporal boundary ambiguity caused by cross-layer feature fusion. Experimental results show that this method significantly improves both the segmentation mean average accuracy (Seg-mAP) and detection mean average accuracy (Det-mAP) on the AD3 dataset, which contains different time spans and spatial scales, achieving high-precision, fine-grained temporal boundary localization for full-scale temporal events.

[0040] 2) Reduced redundancy in dense event detection:

[0041] To address the aliasing degradation problem caused by the temporal overlap and coexistence of multiple events in wide-area observation, this invention designs a multi-label-aware differential query generation module, employing a dual-condition filtering mechanism based on intersection-union ratio (tIoU) and category for redundancy removal. This mechanism accurately suppresses redundant candidate queries of the same category with high overlap, while ensuring that independent candidate queries corresponding to different event categories are retained. Ablation experiments show that after introducing multi-label-aware differential filtering (ML-DQS), the model's Seg-mAP (e.g., increasing from 25.1% to 75.5%) and Det-mAP indicators both show a significant improvement, greatly enhancing the model's accurate redundancy removal capability in complex scenarios with multiple events coexisting and high-density superposition.

[0042] 3) Achieved detection of events with both strong and weak characteristics:

[0043] This invention employs a multi-layer cross-attention decoder to enable multi-scale interaction between sparse differentiated queries and a refined temporal feature pyramid. It also designs a dense-sparse collaborative head consisting of a dense frame-level classification branch and a sparse event-level detection branch. The dense classification branch provides fine-grained frame-level supervision, capable of capturing weak signals, short-term transients, and other weak feature events; the sparse detection branch provides macroscopic event-level prediction, ensuring the overall structural integrity of long-term, wide-scale events. This synergistic complementarity suppresses background noise interference with weak signals and avoids the fragmentation or missed detection of long-term events, enabling the model to exhibit strong robustness and applicability in complex real-world observation sequences such as substorms. Attached Figure Description

[0044] Figure 1 This is an overall flowchart of the method of the present invention.

[0045] Figure 2 This is a structural diagram of multi-path temporal feature generation in the method of this invention.

[0046] Figure 3 This is a diagram showing the event detection results under environmental interference according to the present invention.

[0047] Figure 4 This is a diagram showing the detection results of this invention on complex real aurora events.

[0048] Figure 5 This is an image showing the event detection results of the present invention for scenarios where multiple subclasses coexist. Detailed Implementation

[0049] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0050] Example 1

[0051] See Figure 1 A spatiotemporal aliasing event detection method for wide-area compressed observation includes the following steps:

[0052] Step 1: Perform grayscale compression and size normalization preprocessing on each image: linearly compress the grayscale to a preset grayscale range; scale the image to a preset size to form a standard image set. ,in Given the total number of frames in the image sequence, obtain the preprocessed video sequence images;

[0053] Step 2: Input the preprocessed video sequence images from Step 1 into a dilated 3D convolutional network for feature extraction to obtain initial temporal features. Semantic features are obtained by constructing a dual-path temporal feature pyramid consisting of a bottom-up time-scale ascending path and a top-down time-scale recovery path; wherein, the dual-path temporal feature pyramid includes:

[0054] 1) Bottom-up path: First, the feature dimension is reduced by one-dimensional convolution. Compress to Degree, then through Subsequent downsampling to construct Layered temporal feature pyramid ,in, It is a bottom-up path time series feature pyramid. The first pyramid representing the time series features of the bottom-up path Layer features;

[0055] 2) Top-down path: To begin, Layer features are upsampled to After the layers are of the same size, with the first Temporal semantic fusion is performed on layer features;

[0056] 3) The fused features from each layer are channel-aligned through pointwise convolution, and then smoothed through one-dimensional temporal convolution to construct... Layered Refined Temporal Feature Pyramid ,in, It is a refined time-series feature pyramid. It is the first of the top-down path time series feature pyramids Layer features;

[0057] Step 3, in the refined time-series feature pyramid Generate candidate queries at each layer and at each time series. And for any two queries Perform dual-condition filtering to create differentiated queries. Among them, dual-condition filtering includes:

[0058] 1) The temporal intersection-union ratio of the two queries is greater than the threshold;

[0059] 2) The two queries have the same category;

[0060] 3) When both 1) and 2) are satisfied, suppress one of the queries;

[0061] Step 4, perform the differentiated query With the refined time-series feature pyramid Attention calculations are performed to obtain the frame-level multi-class prediction probability matrix. Refined temporal feature pyramid Layer 0 features Feature extraction is performed to obtain a frame-by-frame multi-class prediction probability matrix. Finally and Linear weighted fusion is performed to obtain the final frame-level multi-class prediction output. Weighted fusion refers to:

[0062]

[0063] in, For the first image in a video sequence frame, To adjust the fusion weights, control and The relative contribution ratio to the final output, or the final frame-level multi-class prediction output obtained through other equivalent transformations of the frame-level multi-class prediction probability matrix and the frame-by-frame multi-class prediction probability matrix.

[0064] Step 1, the preprocessing includes the following steps:

[0065] Step 1a: Compress the 16-bit grayscale range of the original image to an 8-bit grayscale range;

[0066] Step 1b, will The original image of pixels is uniformly scaled to A standard image of pixels.

[0067] Step 2, the process of constructing the dual-path temporal feature pyramid, includes the following steps:

[0068] Step 2a, the initial timing features Channel dimensionality reduction is performed using a one-dimensional convolution with a kernel size of 1, reducing the dimension from... Compress to ;

[0069] Step 2b, bottom-up path through One-dimensional max pooling with a step size of 2 is used for step-by-step downsampling to construct... Layer-by-layer temporal feature pyramid, the first Layer feature length ;

[0070] Step 2c, top-down path from Begin, the first Layer features are upsampled to the th layer using bilinear interpolation. After the layers are of the same size, according to Add element-wise the corresponding layer features;

[0071] Step 2d: The fused features from each layer are channel-aligned using a pointwise convolution with a kernel size of 1, and then smoothed using a one-dimensional temporal convolution with a kernel size of 3, thus constructing... Layered Refined Temporal Feature Pyramid The construction of the dual-path temporal feature pyramid was completed.

[0072] In step 2, the temporal semantic fusion may take the form of element-wise addition, but is not limited to: ,in, It is the first of the top-down path time series feature pyramids Layer features, It is the first of the bottom-up path time series feature pyramid Layer features, Features Perform upsampling.

[0073] Step 3, which involves performing dual-condition filtering on any two queries, includes the following steps:

[0074] Step 3a: First, calculate the predicted label set for each candidate query. ,in, It is the first Predicted labels for each candidate query. It is an index of action categories. This represents the sigmoid activation function. It is the first Candidate queries in category The original prediction score on the screen, It is the category confidence threshold;

[0075] Step 3b: Suppress queries with a temporal intersection-union ratio higher than the threshold and the same predicted label:

[0076]

[0077] in, It is the first and the One query, It is a time intersection comparison screening threshold. Used to determine whether a condition is true or false. Used for calculation and Time intersection and union comparison;

[0078] Step 3c: Reorder the retained differential queries to form a differential query set. .

[0079] Example 2

[0080] Reference Figure 1 A spatiotemporal aliasing event detection method for wide-area compressed observation mainly includes four stages: image sequence preprocessing, multi-scale temporal feature generation, multi-label perceptual differential query generation, and dense-sparse collaborative head prediction. The specific steps are as follows:

[0081] Step 1: Preprocess all wide-view image sequences in the database: linearly compress the grayscale range of each image to 8-bit grayscale, and scale the images to... Pixels were then used to extract one-dimensional temporal sequence features from the wide-area image sequence using an Integral 3D feature extractor (I3D). ;

[0082] Step 2, refer to Figure 2 Generate a time series feature pyramid for bottom-up paths, generate a time series feature pyramid for top-down paths, and merge the time series feature pyramids of the two different paths.

[0083] Step 2-1: Generate a bottom-up temporal feature pyramid:

[0084] Step 2-1-1, Input frame-by-frame features First, channel dimensionality reduction is performed using a one-dimensional convolution with a kernel size of 1 to generate new features. The convolution kernel is determined adaptively by the model;

[0085] Step 2-1-2: Through one-dimensional max pooling with a step size of 2 at each stage, the new features are... Downsampling is performed step by step to construct a four-layer temporal feature pyramid. ,in , The channel dimensions increase progressively, while the timing length decreases progressively.

[0086] Step 2-2: Generate a top-down path temporal feature pyramid:

[0087] Step 2-2-1, Features As the starting point of the top-down path time series feature pyramid ;

[0088] Step 2-2-2, for features After performing bilinear interpolation upsampling, and with the features Perform element-wise addition to generate features Features are generated according to the above rules. ;

[0089] Steps 2-3: Merge the temporal feature pyramids from two different paths:

[0090] Step 2-3-1, for Pointwise convolutions with a kernel size of 1 are performed separately, and channel alignment is performed, where the convolution kernel is determined adaptively by the model;

[0091] Step 2-3-2: Perform one-dimensional temporal convolution smoothing on the channel-aligned features with a kernel size of 3, and finally output a four-layer refined temporal feature pyramid. ,in , It refers to the number of levels in the pyramid.

[0092] Step 3, generating multi-label-aware differentiated queries, includes: generating dense candidate queries and filtering multi-label-aware differentiated queries.

[0093] Step 3-1, generate dense candidate queries:

[0094] Step 3-1-1, in the dual-path temporal feature pyramid network At each level and at each time sequence location, an event candidate query is generated, and each candidate query contains a time sequence interval and a category;

[0095] Step 3-1-2: Use lightweight regression and classification heads to make preliminary estimates of the time interval and category confidence of the query, and predict the time offset and multi-label category confidence of the candidates.

[0096] Step 3-1-3: Calculate the refined timing coordinates of each candidate based on the prior position and timing offset;

[0097] Step 3-1-4: Concatenate the candidate query matrices for all time-series positions across all layers to generate a total of [number missing]. A dense query.

[0098] Step 3-2, filter multi-tag perceived differential queries:

[0099] Step 3-2-1: Sort all candidate queries in descending order of overall confidence, and then filter out the top [queries]. One query;

[0100] Step 3-2-2: Calculate the predicted label set for each candidate query;

[0101]

[0102] in, It is the first Predicted labels for each candidate query. It is an index of action categories. It is the sigmoid activation function. It is the first Candidate queries in The original predicted score on the category, It is the category confidence threshold;

[0103] Step 3-2-3: For each query, perform multi-label-aware greedy differential query selection, and suppress queries with a time intersection-union ratio higher than the threshold and the same predicted labels:

[0104]

[0105] in, It is the first and the One query, It is a time intersection comparison screening threshold. Used to determine whether a condition is true or false. Used for calculation and The time intervals are compared.

[0106] Step 4, dense-sparse collaborative head prediction, includes: multi-layer cross-attention decoding, sparse prediction head prediction, dense prediction head prediction, and dense-sparse prediction fusion.

[0107] Step 4-1, multi-layer cross-attention decoding includes: performing self-attention mechanism, performing deformable feature cross-attention and nonlinear transformation;

[0108] Step 4-1-1, perform self-attention mechanism: perform self-attention operation on the multi-label perceived differential query generated in step 3 to generate context-aware differential query;

[0109] Step 4-1-2: Perform deformable cross-attention calculation on the context-aware differential query generated in Step 4-1-1 and the dual-path temporal feature pyramid generated in Step 2 to generate an instance-aware target query, given the query features. and reference timing position The calculation of deformable cross attention is as follows:

[0110]

[0111] in, For deformable cross-attention calculation, The first of the two-path time series feature pyramid Layer features, It refers to the number of levels in the pyramid. Query features exist Number of sampling points in the layer For the corresponding attention weights, It queries the learnable sampling offset predicted by a linear mapping;

[0112] Step 4-1-3: Perform a nonlinear transformation on the instance-aware target query generated in step 4-1-2 to generate a higher-order discriminative query;

[0113] Step 4-1-4: Use the high-order discriminative query output from step 4-1-3 as the input to step 4-1-1, and repeat steps 4-1-1, 4-1-2, and 4-1-3 four times to obtain a differentiated query that meets the requirements.

[0114] Step 4-2, Sparse prediction head prediction:

[0115] Step 4-2-1: Transform the time segments of the differentiated query output in Step 4-1-4 to the frame axis through back projection. The known time segment coordinates are... , representing the start and end positions on the feature map, respectively, after:

[0116]

[0117] Convert to frame axis, where and These represent the start and end frames corresponding to the time segment, respectively. It is the step size for temporal downsampling of the video;

[0118] Step 4-2-2: Generate the final sparse frame-level probability matrix. For the first in a certain batch Frame, First The probability value of a class of actions can be calculated using the following formula:

[0119]

[0120] Among them, the query set that meets the conditions Defined as: , For the first The query pair for the first Confidence probability of action prediction and They are the first The start and end frame indices of the time interval coordinates back-projected onto the original frame axis. This is a function to find the maximum value.

[0121] Step 4-3, Dense Prediction Head Prediction: During the model training phase, the dense prediction head is independently supervised and trained using a frame-level multi-label binary cross-entropy loss function. During the inference phase, the dense head uses the shallowest refined features output in step 2-3-2. As input, the network passes through three stacked one-dimensional convolutional layers, and outputs six class probabilities frame by frame while maintaining the temporal dimension:

[0122]

[0123] in, This represents the sigmoid activation function. Indicates dense head operations;

[0124] Step 4-4, Dense-Sparse Prediction Fusion: The final frame-level prediction is obtained through weighted fusion.

[0125]

[0126] in, For the first image in a video sequence frame, The fusion weights are adjustable.

[0127] Example 3:

[0128] The dataset used is AD3, a multi-label aurora detection dataset constructed from 557.7nm wavelength all-sky aurora images obtained by the Yellow River Station optical system. AD3 contains 30 uncropped long-time sequences, totaling 46,341 images, collected from December 2003 to December 2007. Twenty sequences were used as the training set, and ten sequences as the test set. The data includes veil-like diffuse auroras, irregular diffuse auroras, blocky diffuse auroras, striped diffuse auroras, diffuse auroral bands, and discrete auroras, which overlap temporally.

[0129] Experimental equipment: A server equipped with four NVIDIA Titan Xp GPUs, Python version 3.8, and PyTorch 2.4.1 (CUDA 11.8) deep learning framework.

[0130] Evaluation metrics: Two complementary metrics are used: segmentation mean accuracy (seg-mAP) and detection mean accuracy (det-mAP).

[0131] In terms of experimental setup, the images were normalized and preprocessed to 8-bit grayscale and... To improve computational speed without sacrificing detection accuracy, pixel-level optimization was implemented. Initial feature extraction was performed using I3D. The model was trained for 200 iterations with a batch size of 4. The optimizer used was AdamW with an initial learning rate of 0.0001, weight decay of 0.0001, gradient clipping maximum norm of 10, and a cosine annealing learning rate strategy. The dual-path temporal feature pyramid network had a uniform channel dimension of 256. A multi-label-aware differential query generation module generated dense queries at each temporal position of the feature pyramid layer and retained 64 differential queries after multi-label-aware differential filtering. The decoder consisted of 4 layers, each containing 8 self-attention heads and deformable cross-attention heads with 4 sampling points per query. Sparse-dense dual-head weight fusion was employed. The hyperparameter for multi-tag-aware differential query filtering is set to the tIoU threshold. Category confidence threshold All models were trained on a server equipped with four NVIDIA Titan Xp GPUs and implemented in the PyTorch deep learning framework.

[0132] To verify the effectiveness and accuracy of the proposed spatiotemporal aliasing event detection method for wide-area compressed observations, the multi-event detection results of the proposed method were compared with those of other methods. Furthermore, to analyze the impact of key modules on detection performance, ablation experiments were conducted on different benchmark models. Specifically, the temporal feature pyramid module corresponds to global and local semantic modeling, the multi-label perceptual differential query generation module corresponds to reasonable query selection, and the sparse-dense dual-head collaborative prediction module corresponds to temporal boundary and classification prediction. These three modules enhance the model's ability to detect spatiotemporal aliasing events from wide-area compressed observations. Through the above experimental design, the effectiveness of the proposed method compared to traditional methods and the contribution of different methods in the scheme to the overall detection effect can be rigorously demonstrated. The experimental results will be carefully analyzed, and the advantages of the proposed method will be intuitively demonstrated through visualization.

[0133] Experiment 1: Comparison of results of wide-area compressed observation aurora event detection using different methods to verify the superiority of the present invention.

[0134] To verify the effectiveness of the spatiotemporal aliasing event detection method for wide-area compressed observation described in this invention, the method of this invention was compared with methods such as PDAN, PointTAD, and DualDETR. The experimental results are shown in Table 1:

[0135] Table 1. Comparison of quantitative performance of this method with other methods

[0136]

[0137] Table 1 lists the comparison results of the proposed method with other methods. Experimental results show that compared with PDAN, the proposed method improves Det-mAP and Seg-mAP by 20.6 and 19.0 percentage points, respectively; compared with PointTAD, Det-mAP and Seg-mAP improve by 4.8 and 11.6 percentage points, respectively; and compared with DualDETR, Det-mAP and Seg-mAP improve by 1.9 and 12.9 percentage points, respectively. Overall, the experimental results show that the spatiotemporal aliasing event detection method for wide-area compressed observation proposed in this invention can effectively improve the detection accuracy and performance of multiple events based on existing methods, demonstrating good effectiveness and superiority.

[0138] Experiment 2: Ablation experiment to verify the effectiveness of the key modules of the event detection method for wide-area compressed observation proposed in this invention.

[0139] To verify the universality and effectiveness of the proposed method, a series of ablation experiments were conducted on the feature aggregation path of the temporal feature pyramid network, multi-label-aware differential query filtering, and dense-sparse dual-head collaborative prediction mechanism. The results were compared after the experiments, and are shown in Tables 2, 3, and 4. Due to space limitations, the multi-label-aware differential query filtering mechanism is abbreviated as ML-DQS in Table 3.

[0140] Table 2 Ablation experiments of different feature aggregation path configurations in the temporal feature pyramid network.

[0141]

[0142] Table 3 Ablation Comparison of Multi-Label Aware Differentiated Query Filtering and Standard Sparse Query Filtering Strategies

[0143]

[0144] Table 4 Dense-Sparse Two-Head Collaborative Prediction

[0145]

[0146] The ablation comparison experiments above show that the dual-path temporal feature pyramid achieves cross-scale semantic fusion, resulting in a significant improvement in Det-mAP when this module is added to the base network. The multi-label-aware differential query filtering mechanism retains independent candidate queries for concurrent events of different categories within the same time interval, leading to substantial improvements in Det-mAP and Seg-mAP when this module is added to the base network. The dense head compensates for insufficient coverage of short-term discrete aurora and weak signal subclasses, resulting in a significant improvement in Seg-mAP when this module is added to the base network. In summary, the experimental results demonstrate the strong universality and effectiveness of the proposed method.

[0147] Experiment 3: Verification of the effectiveness of the method under environmental interference

[0148] To verify the applicability of the multi-label temporal detection framework, a complete detection of a typical weak-intensity substorm event during 04:00–06:00 UT on December 28, 2007, was performed. The video sequence was input frame by frame into the proposed spatiotemporal aliasing event detection method for wide-area compressed observation. First, global and local semantic fusion of temporal features was performed using a dual-path temporal pyramid for feature extraction. Then, the temporal features were input into a multi-label perceptual differential query generation module to obtain effective differential queries. Finally, the final frame-level detection results were obtained through a dense-sparse collaborative head. The detection results are shown below. Figure 3As shown, the illustrations from top to bottom are: the AL index evolution curve (marking each stage of the substorm's evolution), the keogram, the time-series detection results of multi-label aurora events, and full-sky images of five key moments.

[0149] The negative deflection of the AL index directly reflects the intensity of the electric current collection in the substorm expansion phase. The keogram can condense and extract all the photos taken continuously over a period of time into a single two-dimensional image, making it easy to see the dynamic changes in the sky over a period of time.

[0150] against Figure 3 The experimental results, detailed explanations, and typical scenario case analyses are as follows: In the first frame of the five key moments, located in the early stage of the substorm growth phase, the entire sky image is dominated by veil-like diffuse auroras. This method accurately identifies this as a pulsating brightening region of veil-like diffuse auroras, rather than discrete auroral events. The second frame, located at the end of the substorm growth phase, shows a typical coexistence of veil-like diffuse auroras and discrete auroral arcs in the entire sky image. This method accurately detects both types of concurrent events, verifying the effectiveness of the dual constraints of multi-label perception differential query filtering in the coexistence scenario of the substorm growth phase. The third frame corresponds to the substorm expansion phase. During this period, the dominance of the strong radiation structure of discrete auroras further obscures the weak morphological information of irregular diffuse auroras within the same field of view, representing one of the typical scenarios with the highest risk of missed detection in multi-label detection. However, this method accurately detects irregular diffuse auroras, verifying the effectiveness of the dense-sparse dual-head collaborative prediction mechanism and the deformable cross-attention mechanism. The fourth frame corresponds to the early stage of the substorm recovery phase. At this time, the discrete auroral remnants gradually fade, and the veil-like diffuse aurora is in the initial stage of gradual development. This method accurately captures the gradual boundary of the initial stage of the veil-like diffuse aurora development. This is due to the dual-path temporal feature pyramid fusing global temporal features, enabling the model to perceive the gradual brightening trend of the diffuse aurora using global temporal information, rather than relying on the morphological features of a single frame in isolation. The fifth frame corresponds to the late stage of the substorm recovery phase. The all-sky image shows a veil-like diffuse aurora with extremely weak luminescence intensity, accompanied by discrete auroral structures with relatively weak luminescence intensity. This method still achieves accurate detection under these weak signal conditions without any missed detections. The fundamental source of its effectiveness lies in the continuous temporal dimension, slow drift, and quasi-periodic modulation dynamic features, as well as the weighted fusion of dense and sparse prediction heads.

[0151] This method achieves accurate multi-label detection in all the above stages, verifying the robust adaptability of the proposed framework to auroral coexistence scenarios under the typical geomagnetic disturbance condition of weak-intensity substorms.

[0152] Experiment 4: Event Detection Experiment of the Method of the Invention in Real Complex Aurora Image Sequences

[0153] To verify the effectiveness of the method in complex real-world observation scenarios, a systematic analysis was conducted on the aurora observation sequence from November 19, 2004. The aurora sequence was input frame by frame into the proposed spatiotemporal aliasing event detection method for wide-area compressed observation. First, feature extraction was performed using I3D. Then, a dual-path temporal pyramid was constructed for full-scale semantic fusion and feature extraction. The temporal features were then input into a multi-label perceptual differential query generation module to obtain effective differential queries. Finally, a dense-sparse collaborative head module was used to obtain the final frame-level prediction. The detection results are as follows: Figure 4 As shown, from top to bottom, the images are: keogram, multi-label aurora event time-series detection results, and full-sky images at five key moments.

[0154] against Figure 4 The experimental results, detailed descriptions, and typical scenario case analyses are as follows: The first frame of the all-sky image simultaneously contains both discrete auroras and diffuse auroral bands. As can be seen from the temporal detection results, the duration of the diffuse auroral band far exceeds that of the discrete auroral events in the same period. This method achieves accurate detection for both types of events. This effectiveness stems from the collaborative mechanism design of the dual-path temporal feature pyramid and the multi-label perceptual differential filtering module. The four-layer temporal feature pyramid of the dual-path temporal feature pyramid network covers the full-scale temporal receptive field. The deep feature map meets the overall temporal modeling requirements of the diffuse auroral band, while the shallow feature map retains the fine-grained control over short-term discrete auroral events. The multi-label perceptual differential filtering strategy ensures that the independent candidate queries corresponding to each of the two events are retained. The second and fourth frames simultaneously display veil-like diffuse aurora, diffuse aurora bands, and discrete aurora. Our method outputs three category labels simultaneously at these two moments, closely matching the ground truth labels. This is attributed to the synergistic effect of the dual-path temporal pyramid, the multi-label perceptual differential query module, and the decoder. The deformable cross-attention mechanism of the decoder adaptively focuses each query on the discriminative temporal segment of its respective category, effectively decoupling features in the context of the coexistence of three types of events. In the third frame, veil-like diffuse aurora and weak diffuse aurora bands coexist, posing a highly challenging fine-grained discrimination task. Our method accurately distinguishes these two subclasses, primarily due to the sparse adaptive sampling characteristics of the deformable cross-attention mechanism in the temporal dimension. The temporal-specific dynamic features of veil-like diffuse aurora are not visible in a single frame image. The queries corresponding to different subclasses in the decoder adaptively focus their attention on their respective temporal dynamic feature segments through deformable sampling, thereby achieving fine-grained discrimination based on temporal dynamic patterns. The fifth frame of the all-sky image is dominated by discrete auroras, with a weakly luminous region in the lower right corner resembling a diffuse aurora. As can be seen from the temporal detection results, this method only outputs a discrete aurora category label at the fifth frame. This correct judgment is due to the advantage of this method in temporal dynamic feature modeling, which determines that the region is not a diffuse aurora based on the dynamic features in the temporal dimension.

[0155] This method achieves accurate multi-label detection in all the above stages, verifying the collaborative effectiveness of the proposed framework and fully demonstrating the robust applicability of this method under complex real-world observation conditions.

[0156] Experiment 5: Event Detection Experiment of This Method under Multiple Subclass Coexistence Conditions

[0157] To verify the robustness of this method in scenarios where multiple diffuse aurora subclasses coexist, this experiment systematically analyzes the aurora observation sequence from 06:00 to 08:00 UT on November 15, 2007. The aurora sequence is input frame by frame into the spatiotemporal aliasing event detection method for wide-area compressed observation proposed in this invention. First, the temporal features of the sequence are extracted using a dual-path temporal pyramid. Then, an effective differential query is obtained through a query filtering mechanism. Finally, the final frame-level prediction result is obtained through a dense-sparse collaborative head.

[0158] Analysis results as follows Figure 5 As shown, the illustration from top to bottom consists of: a keogram, the results of multi-label aurora event time-series detection, full-sky images at five key moments, and a continuous full-sky aurora subsequence with an adjacent frame time interval of 20 seconds.

[0159] against Figure 5The experimental results, detailed descriptions, and typical scenario case analyses are as follows: The first frame of the all-sky image shows a veil-like diffuse aurora. However, this atypical morphology deviates from the standard appearance of a veil-like diffuse aurora in the static features of a single frame. This method can correctly identify it, and its effectiveness stems from the accurate modeling of the essential temporal dynamic features of the veil-like diffuse aurora in the deformable cross-attention of the decoder. The second frame shows the temporal coexistence of the veil-like diffuse aurora and discrete aurora. Its special feature is that it is in the early stage of the development of discrete aurora, and its discrete structure is not yet significant. In a single frame image, it is easily identified as a single veil-like diffuse aurora. This method accurately detects the coexistence of aurora events, verifying the effectiveness of the dual-path temporal feature pyramid network and the dual-constraint filtering mechanism of multi-label perception differential query. The third frame shows a transitional phase between the gradual fading of the veil-like diffuse aurora and the gradual dominance of the striped diffuse aurora. This method accurately outputs both types of labels simultaneously, primarily because the decoder uses deformable sampling to adaptively focus on the most discriminative temporal phase for each subclass, effectively decoupling the two queries in the feature space. In the fourth frame, all three diffuse aurora subclasses coexist in the field of view. At this moment, this method simultaneously outputs multi-label predictions for all three subclasses, consistent with the ground truth. This result relies on the collaborative operation of the multi-label perceptual differential query generation module, the decoder, and the dense-sparse collaborative head. The fifth frame shows that the blocky diffuse aurora has developed into the dominant subclass, while the veil-like diffuse aurora is about to fade. The core basis for this method's accurate multi-label prediction of the two coexisting classes lies in the essential differences in temporal dynamic features. The collaborative operation of the dual-path temporal pyramid and the decoder enables the system to effectively distinguish the temporal dynamic features of the two aurora classes.

[0160] To further verify the detection capability of this method under conditions of high static morphological similarity, Figure 5 The bottom shows a five-frame all-sky aurora subsequence from 07:57:40 to 07:59:00UT, with a 20-second interval between adjacent frames. Single-frame static features are almost insufficient to distinguish between blocky diffuse auroras and veil-like diffuse auroras. This method accurately detects this sequence as a blocky diffuse aurora event. Its effectiveness demonstrates the accurate modeling of the temporal dynamic features of this subclass by the deformable cross-attention in the decoder. This fundamentally verifies the feasibility of this method's recognition path based on temporal dynamic patterns rather than single-frame static shapes in such easily confused scenarios.

[0161] The parts of this invention not described in detail are common knowledge to those skilled in the art. The above description is only a preferred embodiment of the present invention and is not intended to limit the invention. Obviously, those skilled in the art, after understanding the content and principles of the present invention, may make various modifications and changes in form and detail without departing from the principles and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the protection scope of the claims of the present invention.

Claims

1. A method for detecting spatiotemporal aliasing events for wide-area compressed observation, characterized in that, Includes the following steps: Step 1: Perform grayscale compression and size normalization preprocessing on each image: linearly compress the grayscale to a preset grayscale range; scale the image to a preset size to form a standard image set. ,in Given the total number of frames in the image sequence, obtain the preprocessed video sequence images; Step 2: Input the preprocessed video sequence images from Step 1 into a dilated 3D convolutional network for feature extraction to obtain initial temporal features. Semantic features are obtained by constructing a dual-path temporal feature pyramid consisting of a bottom-up time-scale ascending path and a top-down time-scale recovery path; wherein, the dual-path temporal feature pyramid includes: 1) Bottom-up path: First, the feature dimension is reduced by one-dimensional convolution. Compress to Degree, then through Subsequent downsampling to construct Layered temporal feature pyramid ,in, It is a bottom-up path time series feature pyramid. The first pyramid representing the time series features of the bottom-up path Layer features; 2) Top-down path: To begin, Layer features are upsampled to After the layers are of the same size, with the first Temporal semantic fusion is performed on layer features; 3) The fused features from each layer are channel-aligned through pointwise convolution, and then smoothed through one-dimensional temporal convolution to construct... Layered Refined Temporal Feature Pyramid ,in, It is a refined time-series feature pyramid. It is the first of the top-down path time series feature pyramids Layer features; Step 3, in the refined time-series feature pyramid Generate candidate queries at each layer and at each time series. And for any two queries Perform dual-condition filtering to create differentiated queries. Among them, dual-condition filtering includes: 1) The temporal intersection-union ratio of the two queries is greater than the threshold; 2) The two queries have the same category; 3) When both 1) and 2) are satisfied, suppress one of the queries; Step 4, perform the differentiated query With the refined time-series feature pyramid Attention calculations are performed to obtain the frame-level multi-class prediction probability matrix. Refined temporal feature pyramid Layer 0 features Feature extraction is performed to obtain a frame-by-frame multi-class prediction probability matrix. Finally and Linear weighted fusion is performed to obtain the final frame-level multi-class prediction output. Weighted fusion refers to: ； in, For the first image in a video sequence frame, To adjust the fusion weights, control and The relative contribution ratio to the final output, or the final frame-level multi-class prediction output obtained through other equivalent transformations of the frame-level multi-class prediction probability matrix and the frame-by-frame multi-class prediction probability matrix.

2. The spatiotemporal aliasing event detection method for wide-area compressed observation according to claim 1, characterized in that, Step 1, the preprocessing includes the following steps: Step 1a: Compress the 16-bit grayscale range of the original image to an 8-bit grayscale range; Step 1b, will The original image of pixels is uniformly scaled to A standard image of pixels.

3. The spatiotemporal aliasing event detection method for wide-area compressed observation according to claim 1, characterized in that, Step 2, the process of constructing the dual-path temporal feature pyramid, includes the following steps: Step 2a, the initial timing features Channel dimensionality reduction is performed using a one-dimensional convolution with a kernel size of 1, reducing the dimension from... Compress to ; Step 2b, bottom-up path through One-dimensional max pooling with a step size of 2 is used for step-by-step downsampling to construct... Layer-by-layer temporal feature pyramid, the first Layer feature length ; Step 2c, top-down path from Begin, the first Layer features are upsampled to the th layer using bilinear interpolation. After the layers are of the same size, according to Add element-wise the corresponding layer features; Step 2d: The fused features from each layer are channel-aligned using a pointwise convolution with a kernel size of 1, and then smoothed using a one-dimensional temporal convolution with a kernel size of 3, thus constructing... Layered Refined Temporal Feature Pyramid The construction of the dual-path temporal feature pyramid was completed.

4. The spatiotemporal aliasing event detection method for wide-area compressed observation according to claim 1, characterized in that, In step 2, the temporal semantic fusion may take the form of element-wise addition, but is not limited to: ,in, It is the first of the top-down path time series feature pyramids Layer features, It is the first of the bottom-up path time series feature pyramid Layer features, Features Perform upsampling.

5. The spatiotemporal aliasing event detection method for wide-area compressed observation according to claim 1, characterized in that, Step 3, which involves performing dual-condition filtering on any two queries, includes the following steps: Step 3a: First, calculate the predicted label set for each candidate query. ,in, It is the first Predicted labels for each candidate query. It is an index of action categories. This represents the sigmoid activation function. It is the first Candidate queries in category The original prediction score on It is the category confidence threshold; Step 3b: Suppress queries with a temporal intersection-union ratio higher than the threshold and the same predicted label: ； in, It is the first and the One query, It is a time intersection comparison screening threshold. Used to determine whether a condition is true or false. Used for calculation and Time intersection and union comparison; Step 3c: Reorder the retained differential queries to form a differential query set. .

Images