Time series data new class identification method based on multi-modal fusion

Abstract

The application belongs to the technical field of data processing, and particularly relates to a time series data new class identification method based on multi-modal fusion, which comprises the following steps: constructing a dynamic sliding baseline container for each modal encoder, calculating a cross-modal information phase angle difference coefficient by modal average normalization of relative entropy deviation, and performing phase alignment on a video stream; performing singular value decomposition on a phase angle difference sequence after difference, determining a structured change according to a double threshold value of a subspace alignment residual and a disturbance amplitude; performing sparse representation on a reconstruction error vector to eliminate historical noise components, projecting a residual to a historical principal component orthogonal complement space, calculating a sparse compensation energy proportion, and applying energy truncation and order preservation; and obtaining a decision score based on a nonlinear mechanism fusion of a time-varying manifold space, and identifying a new class when the decision score exceeds a threshold value. The application can reduce modal interference and asynchronous deviation, improve new class identification accuracy, and reduce false positive rate.

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, this invention relates to a method for identifying new categories in time series data based on multimodal fusion. Background Technology

[0002] In the field of smart city traffic management, multimodal devices such as geomagnetic coils, video optical flow, and meteorological sensors are typically used to collect time-series data in order to monitor traffic conditions.

[0003] The sampling rates of each sensor differ significantly, and each modal data is feature extracted by an independent encoder to obtain latent variables that characterize the intrinsic state.

[0004] Due to the unbalanced distribution of training data, the latent variables output by different encoders may differ by several orders of magnitude in terms of numerical range and baseline distribution. Most existing new category recognition methods directly calculate the absolute deviation of the reconstruction error of each modality or simply fuse the distances of each modality, failing to eliminate the inherent baseline bias introduced by the unbalanced encoder training. At the same time, the video stream has a timestamp offset of about hundreds of milliseconds due to transmission delay. If it participates in intermodal comparison without phase alignment, it will introduce spurious differences, resulting in a large number of false alarms in the recognition process.

[0005] In addition, under normal traffic conditions, sensors are affected by factors such as ambient temperature and parameter drift, which will generate slow common-mode noise. Existing discrimination methods based on a single moment or a simple sliding window are difficult to effectively distinguish such slow drift from the structured changes in the actual traffic conditions.

[0006] Traditional processing methods do not introduce a filtering mechanism for noise components in reconstruction errors that can be explained by historical normal patterns, resulting in a lot of redundant information in residual energy analysis and reducing the specificity of new category discrimination. Summary of the Invention

[0007] To address the technical problems in the prior art of eliminating the inherent biases and interferences caused by training imbalance and asynchronous sampling among multimodal systems, as well as common-mode noise and system drift, the present invention provides solutions in the following aspects.

[0008] In a first aspect, the present invention provides a method for identifying new categories of time series data based on multimodal fusion, the method comprising: A dynamic sliding baseline container for each modal encoder is constructed, the relative entropy deviation of the current sample relative to the baseline distribution is calculated, and the cross-modal information phase difference coefficient is obtained by modal average normalization, and the video stream is phase aligned. Based on the dynamic modulation of the sliding window length of the cross-modal information phase difference coefficient, singular value decomposition is performed on the difference of the cross-modal information phase difference coefficient sequence within the adaptive sliding window, and structured change markers are output based on the dual thresholds of subspace aligned residual and perturbation amplitude. The sparse constraint regularization coefficient is adjusted by the adaptive feedback of the subspace alignment residual. Based on the adjusted sparse constraint regularization coefficient, the reconstruction error vector of the current sample is sparsely represented to remove interpretable historical noise components, and the residual vector is obtained. Then, the residual vector is projected onto the historical principal component orthogonal complement space and the ratio of the energy of the remaining projection vector to the energy of the reconstruction error vector is calculated as the proportion of sparse compensation energy. An adaptive covariance matrix is ​​constructed based on the perturbation amplitude in the time-varying manifold space. The cross-modal information phase difference coefficient and the sparse compensation energy ratio are mapped to the time-varying manifold space to calculate the adaptive Mahalanobis distance and obtain the decision score. The score threshold is then used to identify a new category.

[0009] Preferably, each modal encoder includes a geomagnetic coil encoder, a video optical flow encoder, and a weather sensor encoder.

[0010] Preferably, the construction of the dynamic sliding baseline container for each modal encoder includes: establishing a sliding baseline container for each modal encoder, storing the mean and standard deviation parameters of the hidden layer output distribution of a preset number of normal samples, and updating the baseline distribution parameters using the EM algorithm.

[0011] Preferably, the step of calculating the relative entropy deviation of the current sample relative to the baseline distribution and obtaining the cross-modal information phase difference coefficient by modal average normalization includes: calculating the KL divergence of the latent variable of the current sample of each modal encoder relative to the Gaussian mixture model of the baseline distribution, as the relative entropy deviation of that mode; calculating the average value of the relative entropy deviations of all modes; and obtaining the cross-modal information phase difference coefficient based on the ratio of the sum of the absolute values ​​of the pairwise differences of the relative entropy deviations of each mode to the average value.

[0012] Preferably, the phase alignment of the video stream includes: calculating the delay of the video stream relative to the geomagnetic coil signal using a cross-correlation function, and resampling the video stream density sequence to the time granularity of the geomagnetic coil.

[0013] Preferably, the step of performing singular value decomposition on the cross-modal information phase difference coefficient sequence after differencing includes: performing first-order differencing on the cross-modal information phase difference coefficient sequence to obtain an incremental sequence; and performing singular value decomposition on the incremental sequence within a sliding window to obtain the singular values ​​and principal component directions of the current window. Preferably, the step of outputting a structured change marker based on a dual threshold of subspace alignment residual and perturbation amplitude includes: calculating the norm difference between the singular values ​​of the current window and the singular values ​​of the historical baseline as the perturbation amplitude; calculating the subspace alignment residual between the principal component direction of the current window and the principal component direction of the historical baseline; wherein the singular values ​​of the historical baseline and the principal component direction of the historical baseline are calculated by sliding singular value decomposition of normal samples in the training set, and the standard deviation of the perturbation amplitude of the historical window is the standard deviation of the perturbation amplitude of the sliding window of normal samples in the training set; when the subspace alignment residual exceeds a first threshold and the perturbation amplitude exceeds a predetermined multiple of the standard deviation of the perturbation amplitude of the historical window, a structured change marker is output.

[0014] Preferably, the step of performing sparse representation on the reconstruction error vector of the current sample to remove interpretable historical noise components includes: using a dictionary learned from the reconstruction error vector of normal samples in the training set through the K-SVD algorithm to sparsely encode the reconstruction error vector to obtain sparse representation coefficients; and subtracting the product of the dictionary and the sparse representation coefficients from the reconstruction error vector to obtain a residual vector.

[0015] Preferably, the step of projecting the residual vector onto the orthogonal complement space of the historical principal components and calculating the sparse compensation energy ratio includes: projecting the residual vector onto the orthogonal complement space of the historical principal components to obtain a projection vector; directly using the total energy of the projection vector to calculate the sparse compensation energy ratio; obtaining a decision score, and identifying a new category if the score exceeds a threshold, includes: constructing an adaptive covariance matrix of the time-varying manifold space based on the perturbation amplitude; mapping the cross-modal information phase difference coefficient, the structured change marker, and the sparse compensation energy ratio to the time-varying manifold space to calculate an adaptive Mahalanobis distance to obtain a decision score; and identifying a new category when the decision score exceeds a threshold.

[0016] The embodiments of the present invention have at least the following beneficial effects: 1. The relative entropy deviation of the current sample is calculated based on the baseline Gaussian mixture model, and the cross-modal information phase angle difference coefficient is obtained by modal average normalization. At the same time, the asynchronous offset is eliminated by optical flow phase alignment, which reduces the inherent bias caused by encoder training imbalance and the pseudo phase angle caused by sampling delay, and avoids the continuous false alarm caused by high ground state entropy of a single mode.

[0017] 2. By performing differential preprocessing on the phase difference coefficient sequence of cross-modal information and implementing sliding window singular value decomposition, combined with the dual threshold judgment of subspace alignment residual and disturbance amplitude, only when the current principal component direction deviates significantly from the historical baseline direction and the disturbance energy exceeds the statistical confidence range is it marked as a structural change. This effectively distinguishes common-mode noise from real traffic state variations, avoids misjudgment caused by global interference, and enhances the robustness and positioning accuracy of new category detection in complex environments.

[0018] 3. A sparse energy screening mechanism is introduced into the energy calculation of the orthogonal complement space. The reconstructed error vector is sparsely represented by the historical error pattern dictionary, and noise components that can be explained by history are eliminated. The unexplainable residuals are projected to the orthogonal complement space of the historical principal components and the cumulative energy dominant direction is retained to calculate the energy ratio, thereby eliminating the interference of normal variation that is not modeled and making the index more specific. Finally, the cross-modal information phase angle difference coefficient, structured change marker and sparse compensation energy ratio are integrated into the decision score to achieve multi-dimensional collaboration, which improves the recognition accuracy of truly new categories while reducing the false alarm rate. Attached Figure Description

[0019] Figure 1 The flowchart illustrates the steps of the new category identification method for time series data based on multimodal fusion in this invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] S1: Construct a dynamic sliding baseline container for each modal encoder, calculate the relative entropy deviation of the current sample relative to the baseline distribution, and obtain the cross-modal information phase difference coefficient through modal average normalization, and perform phase alignment on the video stream.

[0023] In the multimodal time-series data scenario of smart city traffic management, the encoders of geomagnetic coils, video optical flow, and meteorological sensors suffer from imbalanced training data, resulting in output hidden layer distributions with mean and standard deviations differing by several orders of magnitude. Directly comparing intermodal deviations amplifies system noise rather than reflecting the structural changes in the actual traffic conditions. Therefore, a dynamic sliding baseline container is constructed for each modal encoder. Store the most recent Mean parameters of the hidden layer output distribution of a normal sample and standard deviation parameter and every The parameters of the Gaussian mixture model of the baseline distribution are updated using the online EM algorithm over time. The empirical value is 1000 samples. The empirical value is 10 minutes, and all of these are hyperparameters that can be adjusted by the implementer based on the amount of data in the scenario. and The dimensions of the encoder output features are consistent with the dimensions of the encoder output features.

[0024] For the multimodal samples input at the current time, calculate the latent variables output by each modal encoder. KL divergence relative to its baseline Gaussian mixture model As the degree of relative entropy deviation of this mode, From the formula Given, among which The probability density function of the baseline Gaussian mixture model is... Let Gaussian approximation of the probability density function of the latent variables of the current sample be given. It is a dimensionless quantity. For modal index, , The total number of modes is 3, which is empirically taken as a value. This value is a hyperparameter and can be adjusted according to the actual number of sensor modes connected. The probability density function of the Gaussian mixture model is a well-known technique and will not be described in detail here.

[0025] Then, the average value of the relative entropy deviation of all modes is calculated. , and according to Obtain cross-modal information phase angle difference coefficients In the formula and For different modal indices, For a very small positive constant, the empirical value is [value]. Dimensionless, used to avoid division by zero. It is also a dimensionless coefficient, and its value is [value missing] when the input sample is in a normal state. .

[0026] in, and The KL divergence value represents the degree of deviation of the relative entropy of different modes, indicating how much abnormal the data collected by a single sensor is compared with its historical normal state. This represents the total number of modes; The average deviation of all modalities represents the current overall level of anomaly. yes The total number of pairwise combinations of each modality; This calculation represents the sum of the pairwise absolute differences in the deviations of each sensor. If all sensors experienced similar deviations synchronously, the differences between them would be small. However, if a sudden event was only detected by some sensors, the differences between the individual sensors would be significant. The values ​​will vary greatly, leading to a sharp increase; It uses the overall average deviation as a benchmark, for Normalization is performed; when the overall system experiences severe oscillations, The absolute error may also be amplified, causing false alarms, when divided by the average value. After that, the common-mode noise was reduced, making It becomes a dimensionless pure coefficient that only reflects relative differences.

[0027] To address the approximately 100-millisecond offset between video frame timestamps and geomagnetic coil signals caused by transmission delays in video optical flow sensors, a cross-correlation function is used to calculate the delay between the video flow density sequence and the geomagnetic coil vehicle passage rate sequence. , sort the video stream density sequence according to The time was shifted and resampled to a 1-minute time granularity using the geomagnetic coil. The resampling was performed using linear interpolation, thereby obtaining a video stream density sequence time-aligned with the geomagnetic coil signal. The unit is milliseconds, and the calculation of the cross-correlation function is a well-known technique, which will not be elaborated further.

[0028] S2: Perform singular value decomposition on the phase difference coefficient sequence of cross-modal information after differential decomposition, and output structured change labels based on the dual thresholds of subspace aligned residuals and perturbation amplitude.

[0029] Because the cross-modal information phase difference coefficient sequence exhibits slow numerical drift under normal conditions—a drift typically caused by sensor temperature changes or minor encoder parameter updates—it does not reflect structural changes in traffic conditions. If singular value decomposition is directly performed on the original cross-modal information phase difference coefficient sequence, the resulting principal component directions will be dominated by the slowly drifting components, leading to a decrease in the sensitivity of subspace alignment to actual structural changes. Therefore, a first-order difference is performed on the cross-modal information phase difference coefficient sequence to obtain the incremental sequence. In the formula, For the current moment The cross-modal information phase angle difference coefficient, The phase angle difference coefficients of the cross-modal information at the previous moment. for The increment value at time is dimensionless. The first-order difference operation eliminates the slowly drifting common-mode component, causing the sequence to fluctuate around the zero value.

[0030] Within an adaptive sliding window of length L, the incremental sequence values ​​at all time points within the window are... Construct the observation matrix and perform singular value decomposition to obtain ,in, Let be the left singular vector matrix of the current window, and let its column vectors represent the principal component directions of the increment sequence. This is the diagonal matrix of singular values ​​for the current window. The singular values ​​on the diagonal are arranged in descending order, reflecting the energy intensity along the directions of each principal component. This is the right singular vector matrix of the current window.

[0031] The sliding window length L is determined by the phase angle difference coefficient of the cross-modal information. Dynamic modulation, the calculation formula is: ,pass Apply non-negative lower bound protection, where the base window length The empirical value is 50, minimum window length The empirical value is 10, which is the sensitivity attenuation factor. The experience value is 5, where, , , This is a hyperparameter that can be adjusted by the implementer according to the specific implementation scenario.

[0032] By constructing this phase angle-driven adaptive window, when When the window size increases, it automatically shortens to enhance the capture of mutations. When the system approaches zero, a long recovery window is used to smooth out system noise.

[0033] Retain the previous value during real-time calculation. The principal component directions are used for subspace alignment analysis. Indicates the previous window corresponding to the current window The left singular vector submatrices with the largest singular values ​​form an orthogonal basis for the current subspace. An empirical value of 5 is used as a hyperparameter, which can be adjusted by the implementer based on modal complexity. This is the historical baseline subspace. The window offset step size for the normal samples in the training set is 1 sample point, obtained by averaging the sliding singular value decomposition results of the last 2000 normal samples in the training set. This is achieved by applying the following to each normal sliding window: The matrix is ​​averaged column by column to obtain stable historical principal component directions. 2000 is a hyperparameter that can be adjusted by the implementer according to the specific implementation scenario.

[0034] Calculate the singular value matrix of the current window Singular value matrix with historical baseline The difference in Frobenius norm between them serves as the perturbation magnitude for the current window. ,in The mean matrix of the singular values ​​of the normal window in the training set. This represents the overall deviation of the energy distribution along each principal component direction from the normal baseline within the current time window, expressed as a dimensionless quantity. Simultaneously, it calculates the perturbation amplitude of the sliding window for all normal samples in the training set. Standard deviation As a benchmark reference value for normal fluctuations, It is a dimensionless quantity.

[0035] Calculate the structured projection residuals between the current subspace and the historical baseline subspace. ,in for 3D identity matrix Let be the inner product matrix of the bases of the two subspaces, reflecting the projection coefficients of the current principal component direction onto the historical baseline direction. When the two subspaces are perfectly aligned, this matrix approaches the identity matrix. Approaching zero, the principal component orientation rotates when structural changes occur. Significantly increased, It is a dimensionless quantity.

[0036] If and only if the structured projection residual Exceeding the first threshold And the amplitude of the disturbance Exceeding the standard deviation of historical window disturbance amplitude When the value is 3 times the normal value, it is determined that a real structured change in traffic status has occurred within the current window, and a structured change marker is output. Otherwise output ,in An empirical value of 0.15 is used as a hyperparameter, which can be calibrated by the implementer using a small number of labeled samples during the initial deployment phase. The three-standard-deviation criterion corresponds to the statistical principle... The outlier detection principle states that the probability of false triggering under normal fluctuations is less than 0.3%, thus excluding global disturbances caused solely by common-mode noise from the new category of triggering conditions.

[0037] S3: Perform a sparse representation on the reconstruction error vector of the current sample to remove interpretable historical noise components. Then, project the residual vector onto the orthogonal complement space of historical principal components and calculate the sparse compensation energy ratio. Apply energy truncation to preserve order.

[0038] After completing the structured change marker discrimination, the reconstruction error vector generated by the reconstruction of the current sample by each modal encoder is obtained. ,in , The original multimodal splicing features, Features reconstructed via an autoencoder Dimensions and Consistent, dimensionless.

[0039] Because the historical training samples cover a limited number of normal traffic patterns, some normal variations, such as seasonal traffic flow differences, are not fully modeled. These variations will appear as residual energy. In the middle, and when directly projected onto the orthogonal complement space of the historical principal components, virtual high energy is formed, therefore, firstly... Sparse representation is used to remove components that can be explained by historical noise patterns.

[0040] The dictionary is obtained offline using the K-SVD algorithm by reconstructing the error vectors of all normal samples in the training set. For the reconstruction error vector Sparse coding is performed, where the dictionary , To reconstruct the dimension of the error vector, the number of dictionary atoms The empirical value is set to 500, and the maximum number of iterations for the K-SVD algorithm is set to 100 to solve for the sparse representation coefficients. The solution process is as follows: ,in The empirical value for the number of atoms is 500, where the number of atoms in the dictionary is... The maximum number of iterations is a hyperparameter that can be adjusted by the implementer according to the specific implementation scenario.

[0041] The sparse constraint regularization strength coefficient is determined by the subspace alignment residual. Dynamic feedback adjustment, the calculation formula is: The basic regularization coefficient The empirical value is 0.1, and the residual response gain coefficient is... The experience value is 10. and This is a hyperparameter that can be adjusted by the implementer according to the specific implementation scenario.

[0042] This linkage mechanism enables the detection of structured deviations, i.e., occurrences. As the value increases, the dependence on the historical dictionary will automatically decrease, forcing new energy to flow into the subsequent residuals, thus achieving cross-amplification of structural and energy characteristics.

[0043] After obtaining the estimated coefficients, the portion of the error vector that can be explained by historical noise patterns will be reconstructed. Remove, forming a residual vector , This refers to the remaining error components after historical noise components have been sparsely removed.

[0044] Historical principal component orthogonal supplement space After performing principal component analysis on the normal features of the training set, the top [number] features are selected. The space spanned by each principal component is composed of orthogonal complementary spaces. The empirical value is 10, which is a hyperparameter that can be adjusted by the implementer according to the specific implementation scenario. The retained principal component directions form an orthogonal basis. , The projection operator is .

[0045] The residual vector Project to , to obtain the projection vector At this point, the projection vector contains only residual components that cannot be explained by the direction of the historical principal components.

[0046] Calculate the proportion of sparse compensation energy In the formula For a very small positive number, the empirical value is [value]. Dimensionless, used to ensure that the denominator is non-zero. It is also a dimensionless quantity, and its value reflects the proportion of energy of novel components in the residual error after removing historically explainable noise.

[0047] S4: The decision score is obtained by fusing cross-modal information phase angle difference coefficient, structured change label and sparse compensation energy ratio. If the score exceeds the threshold, it is identified as a new category.

[0048] In obtaining cross-modal information phase difference coefficients Structured change markers and the proportion of sparse compensation energy After using three dimensionless indicators, since they respectively characterize the degree of separation between the current sample and the known class from three dimensions—relative cognitive bias between modalities, temporal structure perturbation, and residual energy specificity—each indicator is susceptible to misjudgment under complex disturbances such as common-mode noise, system drift, or historical unmodeled fluctuations. Therefore, these three indicators are multidimensionally weighted and fused to obtain a comprehensive decision score. To overcome the limitations of linear weighting in dealing with complex disturbances, this invention is based on the disturbance amplitude. A nonlinear fusion mechanism is constructed for the time-varying manifold space, and the fusion computation formula is updated as follows: .

[0049] in, It is a two-dimensional state vector composed of the phase angle difference coefficients of cross-modal information and the proportion of sparse compensation energy; Let be the adaptive covariance matrix dynamically modulated by the perturbation amplitude, defined as , For very small positive numbers, such as ; This is a structured mutation penalty term, empirically set at 0.3, dimensionless, and can be adjusted by the implementer according to the specific implementation scenario.

[0050] In the above nonlinear fusion calculation formula, since It reflects the degree of consistency deviation of the output of each modal encoder relative to its respective baseline at the current moment. It directly provides a binary result indicating whether a directional structural change has occurred in the current time series window. This represents the proportion of remaining energy that cannot be fitted by historical principal components even after removing historical interpretive noise. This ensures that the final fusion result not only integrates the degree of variation across modalities, temporal sequences, and residual spaces, but also introduces... Parameter linkage was achieved: when the time sequence structure was subjected to severe disturbance ( During a surge, the metric scale of the state space is automatically distorted, nonlinearly amplifying the weight of the energy anomaly dimension; while under normal perturbations, the space is contracted, effectively suppressing false alarms.

[0051] income It is a dimensionless quantity, and in normal samples it is usually distributed in... The interval is such that when a new category is actually introduced, the three indicators rise simultaneously and exceed the preset threshold.

[0052] The obtained judgment score With preset decision threshold Comparison, The empirical value is 0.5, which is dimensionless. This value is a hyperparameter and can be tuned on the validation set by the implementer according to the degree of conservatism required for the identification of new categories.

[0053] when If the current input sample belongs to a new category that is not covered by historical data, the recognition result is output; otherwise, the current sample belongs to one of the known normal categories.

[0054] This fusion decision strategy integrates multi-dimensional clues through a weighted mechanism. Compared with single-indicator threshold decision, when common-mode noise causes a single indicator to be falsely high, the other indicators can still suppress it below the threshold, thereby keeping the false alarm rate controllable.

[0055] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0056] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0057] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying new categories in time series data based on multimodal fusion, characterized in that, Includes the following steps: A dynamic sliding baseline container for each modal encoder is constructed, the relative entropy deviation of the current sample relative to the baseline distribution is calculated, and the cross-modal information phase difference coefficient is obtained by modal average normalization, and the video stream is phase aligned. Based on the dynamic modulation of the sliding window length of the cross-modal information phase difference coefficient, singular value decomposition is performed on the difference of the cross-modal information phase difference coefficient sequence within the adaptive sliding window, and structured change markers are output based on the dual thresholds of subspace aligned residual and perturbation amplitude. The sparse constraint regularization coefficient is adjusted by the adaptive feedback of the subspace alignment residual. Based on the adjusted sparse constraint regularization coefficient, the reconstruction error vector of the current sample is sparsely represented to remove interpretable historical noise components, and the residual vector is obtained. Then, the residual vector is projected onto the historical principal component orthogonal complement space and the ratio of the energy of the remaining projection vector to the energy of the reconstruction error vector is calculated as the proportion of sparse compensation energy. An adaptive covariance matrix is ​​constructed based on the perturbation amplitude in the time-varying manifold space. The cross-modal information phase difference coefficient and the sparse compensation energy ratio are mapped to the time-varying manifold space to calculate the adaptive Mahalanobis distance and obtain the decision score. The score threshold is then used to identify a new category.

2. The method according to claim 1, characterized in that, The modal encoders include a geomagnetic coil encoder, a video optical flow encoder, and a meteorological sensor encoder.

3. The method according to claim 1, characterized in that, The construction of the dynamic sliding baseline container for each modal encoder includes: establishing a sliding baseline container for each modal encoder, storing the mean and standard deviation parameters of the hidden layer output distribution of a preset number of normal samples, and updating the baseline distribution parameters using the EM algorithm.

4. The method according to claim 1, characterized in that, The calculation of the relative entropy deviation of the current sample relative to the baseline distribution and the resulting modal average normalization to obtain the cross-modal information phase difference coefficients include: Calculate the KL divergence of the latent variables of the current sample of each modal encoder relative to the Gaussian mixture model of the baseline distribution, as the relative entropy deviation of that modality; Calculate the average of the relative entropy deviations of all modes; The cross-modal information phase angle difference coefficient is obtained by the ratio of the sum of the absolute values ​​of the pairwise differences in the relative entropy deviations of each mode to the average value.

5. The method according to claim 1, characterized in that, The phase alignment of the video stream includes: calculating the delay of the video stream relative to the geomagnetic coil signal using a cross-correlation function, and resampling the video stream density sequence to the time granularity of the geomagnetic coil.

6. The method according to claim 1, characterized in that, The step of performing singular value decomposition on the phase difference coefficient sequence of the cross-modal information after differencing includes: The incremental sequence is obtained by performing a first-order difference on the cross-modal information phase angle difference coefficient sequence; Singular value decomposition is performed on the incremental sequence within a sliding window to obtain the singular values ​​and principal component directions of the current window.

7. The method according to claim 1, characterized in that, The structured change markers output based on the dual thresholds of subspace aligned residuals and perturbation amplitude include: The norm difference between the singular values ​​of the current window and the singular values ​​of the historical baseline is calculated as the perturbation amplitude; Calculate the subspace alignment residual between the current window principal component direction and the historical baseline principal component direction; Wherein, the historical baseline singular values ​​and the historical baseline principal component directions are calculated by the sliding singular value decomposition of normal samples in the training set, and the standard deviation of the historical window perturbation amplitude is the standard deviation of the sliding window perturbation amplitude of normal samples in the training set. When the subspace alignment residual exceeds a first threshold and the perturbation amplitude exceeds a predetermined multiple of the standard deviation of the historical window perturbation amplitude, a structured change marker is output.

8. The method according to claim 1, characterized in that, The step of sparsely representing the reconstruction error vector of the current sample to remove interpretable historical noise components includes: Using a dictionary learned from the reconstruction error vector of normal samples in the training set through the K-SVD algorithm, the reconstruction error vector is sparsely encoded to obtain sparse representation coefficients; Subtracting the product of the dictionary and the sparse representation coefficients from the reconstruction error vector yields the residual vector.

9. The method according to claim 1, characterized in that, The step of projecting the residual vector onto the orthogonal complement space of historical principal components and calculating the sparse compensation energy ratio includes: The residual vector is projected onto the orthogonal complement space of the historical principal components to obtain the projection vector; the total energy of the projection vector is directly used to calculate the sparse compensation energy ratio. The process of obtaining a decision score and identifying a new category if the score exceeds a threshold includes: constructing an adaptive covariance matrix of the time-varying manifold space based on the perturbation amplitude; mapping the cross-modal information phase difference coefficients, the structured change markers, and the sparse compensation energy proportions to the time-varying manifold space; calculating an adaptive Mahalanobis distance to obtain a decision score; and identifying a new category when the decision score exceeds a threshold.

Images