Data increment-oriented sparse tensor flow back-testing method

Abstract

The invention relates to a data increment-oriented sparse tensor flow back-testing method. Increment analysis is performed on tensor flow data, channel activeness is detected based on a dimension combination which is not activated in each time step, dilution processing is performed on a high-dimensional combination which is not activated for a long time, and a shadow index is reserved; identifying potentially activated tensor dimension channels for the newly added incremental data, and judging continuous change characteristics in a preset time window; comparing the structure between any newly added tensor slice and the previous state, calculating the structure offset intensity, and if the offset intensity does not exceed a set offset entropy threshold value, skipping a backtest process and adopting a backtest snapshot; on the contrary, carrying out compression backtest on the tensor block with the structure having significant variation; and when an abnormal event is detected, backtracking a tensor dimension path which is diluted or is not activated due to an inertia strategy based on an abnormal triggering mechanism, reconstructing a structure state, executing an anti-sparse backtesting process, and identifying and tracking an abnormal propagation path so as to compensate and position a potential influence region.

Description

Technical Field

[0001] This invention relates to a sparse tensor flow backtesting method for data, specifically a sparse tensor flow backtesting method for incremental data. Background Technology

[0002] Existing technologies, such as Chinese patent CN110138614A, "An Online Network Traffic Anomaly Detection Method and System Based on Tensor Models," currently mainstream methods for tensor-based anomaly detection face a series of unavoidable technical bottlenecks in key aspects such as handling incremental data, high-dimensional sparsity, dynamic structural evolution, and anomaly tracing. This patent uses three-dimensional tensors as its modeling foundation, extracts the current-time data matrix through a fixed time window, and combines historical tensor slices with low-rank tensor decomposition to extract normality factors, ultimately using error tensors for anomaly detection. However, this design has significant shortcomings when dealing with dynamic data scenarios: First, its fixed-window strategy is poorly adaptable to real-world environments with frequent incremental data injection and unstable spatiotemporal structures, unable to flexibly adjust the window length to adapt to sudden changes or trend drifts, leading to insufficient sensitivity or frequent false alarms. Secondly, the tensor decomposition process used in this invention assumes that the data structure dimensions are stable. The three-dimensional tensor is usually preset to "IP-port-time" or "traffic source-target-time", which cannot adapt to the situation where the tensor dimensions themselves change with business logic or system topology. Therefore, when there are structural abrupt changes such as dimension addition, path rearrangement or dimension compression, this method cannot dynamically adjust the tensor modeling framework. It can only be processed by rebuilding the data structure or forcibly filling in the missing dimensions, which leads to a significant reduction in system sustainability and real-time performance.

[0003] Furthermore, this method lacks a sparsity management mechanism for low-frequency combinations within the tensor dimensions. All dimension combinations are retained for tensor construction and decomposition, which significantly increases computational complexity and memory costs in large-scale high-dimensional sparse scenarios. It also dilutes the impact of error mutations in truly critical paths, reducing the focus and effectiveness of anomaly detection. Additionally, existing anomaly detection technologies rely on the error matrix between the current tensor decomposition residual and historical normal values. The judgment logic depends on convergence and non-relaxation criteria, making it difficult to capture periodic and structural anomaly signals and lacking interpretable modeling for anomaly propagation. Moreover, existing technologies lack mechanisms to support the assessment of anomaly diffusion and path-level response in determining the scope of anomaly propagation. Finally, this patented method typically requires retraining the entire tensor or iterating through a full window. This batch reprocessing approach struggles to meet the online detection needs of high-frequency incremental data, and its iterative convergence stopping mechanism often faces delays or misjudgments in real-world complex flow fields. In summary, while existing technologies have indeed improved the static modeling and error detection capabilities of three-dimensional network traffic based on traditional tensor models, they lack sparse management, structural adaptation, path backtracking, and snapshot mechanisms when facing practical engineering challenges such as dynamic structural evolution, high-dimensional sparse tensor flow, incremental data access, abnormal path propagation, and structural recoverability. This results in problems such as system response delays, detection blind spots, and resource waste. Summary of the Invention

[0004] The purpose of this invention is to provide a sparse tensor flow backtesting method for incremental data, thereby addressing some of the drawbacks and shortcomings pointed out in the background art.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: a sparse tensor flow backtesting method for incremental data, including: performing incremental access analysis on tensor flow data organized in a multi-dimensional structure, detecting channel activity based on inactive dimension combinations in each time step, performing dilution processing on high-dimensional combinations that have not been activated for a long time, removing them from the current tensor structure, and retaining shadow indexes for future activation. When processing new incremental data, identify potentially activated tensor dimension channels, determine the continuous change characteristics within a preset time window, and if the continuous threshold is not reached, do not include them in the tensor structure, but record the evolution trend to form an inertial propagation mechanism to reduce structural fluctuations. During the evolution of tensor flow, the structure of any newly added tensor slice is compared with that of the previous state, and the structural offset intensity is calculated. If the offset intensity does not exceed the set offset entropy threshold in the corresponding historical time period, the backtesting process of that historical time period is skipped and a backtesting snapshot is used; otherwise, a compressed backtesting is performed on the tensor block whose structure has changed significantly. When anomalies are detected in the incremental tensor flow, including structural mutations or a significant surge in reconstruction residuals, the tensor dimension paths that were previously diluted or not activated due to inertial strategies are traced back based on the anomaly triggering mechanism. Their structural states are reconstructed, and an anti-sparse backtesting process is performed to identify and track the anomaly propagation path in order to compensate for and locate the potential affected areas.

[0006] Furthermore, the shadow index includes a unique key value for recording the removed dimension combination and the last active timestamp, which is used for structural reconstruction when an anomaly is triggered subsequently; the channel activity is dynamically updated by statistically analyzing the frequency of access to the dimension combination per unit time, and multiple activity thresholds are set to adjust the dilution intensity.

[0007] Furthermore, the dilution process includes marking the combination of dimensions that meet the dilution conditions as a frozen state, which is reactivated during abnormal activation or periodic re-checking; in the inertial propagation mechanism, the evolutionary trend record is modeled using an exponentially weighted moving average method to model the potential activation frequency; the determination of the persistent change characteristics is based on a joint judgment of the number of consecutive occurrences within the historical window and the maximum interval time.

[0008] Furthermore, the structural offset intensity is calculated using the difference in tensor structure hash fingerprints or the subspace overlap rate; the offset entropy threshold is an adaptive threshold dynamically adjusted based on the historical tensor evolution trajectory, and the sensitivity is adjusted in conjunction with the global sparse density factor; the backtest snapshot is a cached copy of the previous backtest calculation results, and has a fast indexing mechanism based on time stamp and tensor structure version.

[0009] Furthermore, the compressed backtesting includes local backtesting of the tensor block where the structural shift occurs, and constructing a prediction error estimate using historical backtesting residuals; the detection of the abnormal event is jointly triggered by identifying sudden increases in the residual tensor and node jump changes in the structural variation spectrum.

[0010] Furthermore, the compressed backtesting includes local backtesting of the tensor block where the structural shift occurs, and constructing a prediction error estimate using historical backtesting residuals; the detection of the abnormal event is jointly triggered by identifying sudden increases in the residual tensor and node jump changes in the structural variation spectrum.

[0011] Furthermore, when a structural mutation caused by the addition of the tensor dimension is detected, anti-sparse activation is performed on the dilute dimension combination related to the added dimension, and priority is assigned for backtesting; the dimension combination rearrangement detection includes the judgment of asymmetric activation regions caused by changes in tensor index order, and the original combination mapping relationship is reconstructed after detection to support the docking of historical trajectories.

[0012] Furthermore, the determination of activation density mutation is based on the change in the second derivative of the number of activated elements within a unit time window. If the value exceeds the threshold, an anti-sparse process is triggered. When reactivating the diluted dimensional channel, the evolution time vector of the channel is generated synchronously and aligned with the anomaly detection time to locate the target time period.

[0013] Furthermore, if two consecutive abnormal indicators exceed the tolerance limit during the anti-sparse backtesting process, the channel is marked as a persistent abnormal channel, and a warning for freezing the trigger structure is issued; the estimation of the abnormal impact range adopts the convolution propagation algorithm to spread the abnormal residual of the channel to the tensor quantum structure to identify the secondary affected region. Specifically: When the anomaly intensity of an activated sparse channel exceeds the system's set tolerance limit for two consecutive cycles during anti-sparse backtesting. At this point, the channel is marked as a persistent anomalous channel. To identify the range of influence of this channel on the surrounding tensile quantum structure, the following anomalous propagation weight integral formula is used: in: Indicates indexing in tensor dimensions The cumulative intensity of abnormal diffusion on the surface is used to determine whether the substructure belongs to the secondary affected area; Represents a time variable. This is the moment of the first detection of the anomaly. This serves as the observation time window for backtesting. Indicates the main abnormal channel In time Abnormal activation intensity; Indicates channel With target substructure The structural coupling factor between them is modeled based on the co-occurrence probability of the channel path; For anomalous diffusion, the time-domain decay kernel, Control the diffusion rate; For channel With structure In time The activation indicator function for whether a linkage occurs has a value of ; when Exceeding the system's set saturation threshold At that time, structure It is identified as a secondary affected area and included in the anti-sparse backtesting compensation range; if multiple consecutive structures meet this condition, a structure freeze early warning mechanism is triggered for the main abnormal channel, limiting its removal weight in future sparsity control.

[0014] The beneficial effects of this invention are as follows: By introducing a shadow indexing mechanism and a frozen state marking strategy, lightweight removal and recoverable mounting of long-term inactive dimension combinations are achieved, solving the problems of bloated structure and high update costs in traditional tensor methods when facing large-scale sparse dimensions, and significantly improving system operating efficiency and storage utilization. This method does not rely on a fixed time window, but combines dynamic characteristics such as activation frequency per unit time, continuous change characteristics, and structural offset intensity to construct an inertial propagation mechanism and offset entropy judgment rules, achieving predictive management of potential active paths and effectively avoiding the problem of ignoring short-period or slowly changing anomalies. It not only detects anomalies in the tensor value domain, but also identifies structural changes such as dimension additions, combination rearrangements, and abrupt changes in activation density, enhancing the system's response capability to non-numerical anomalies (such as dimension changes and path reconstruction), and providing more comprehensive monitoring and protection for complex system environments.

[0015] By constructing an anomaly propagation convolution integral function, the diffusion model of anomaly residuals in tensor quantum structures is realized, thereby identifying secondary affected regions. Combined with the evolution time vector to trace the anomaly origin and propagation chain, the system possesses strong capabilities for anomaly source localization and influence range estimation, enhancing its intelligence level. When anomaly channels exhibit unacceptable deviations for multiple consecutive cycles, the system automatically triggers structure freezing to prevent accidental removal, achieving an adaptive balance between sparsity and stability, effectively improving the robustness of the tensor system and the controllability of the backtesting strategy. The backtesting snapshot, compressed local backtesting, and sub-block residual estimation mechanisms in this method enable rapid anomaly verification without full reconstruction in incremental streams, significantly reducing computational resource consumption and making it suitable for large-scale online analysis systems with high real-time requirements. Attached Figure Description

[0016] Figure 1 This is a simplified flowchart of the sparse tensor flow backtesting process of the present invention.

[0017] Figure 2 This is a diagram showing the relationship between the sparse tensor flow shadow index and the dynamic backtesting function of this invention.

[0018] Figure 3 This is a diagram showing the relationship between the structural mutation identification and anti-sparse backtesting functions of this invention.

[0019] Figure 4 This is the dynamic dilution and anomaly backtracking process for sparse tensor flow in intelligent transportation according to Embodiment 1 of the present invention.

[0020] Figure 5 This is a schematic diagram of anti-sparse backtesting and anomalous diffusion processing of traffic sparse tensor flow under multiple types of structural mutations in Embodiment 2 of the present invention. Detailed Implementation

[0021] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] Combined with appendix Figure 1This invention presents a sparse tensor stream backtesting method for incremental data. It performs incremental access analysis on tensor stream data organized with a multi-dimensional structure. Specifically, during the continuous streaming access of tensor data, the system monitors in real-time the new tensor slices generated at each time step (e.g., per second, per minute, or smaller time granularities). These tensor slices typically have a multi-dimensional structure; for example, a three-dimensional tensor can be represented as a combination of user-behavior-time, or sensor-metric-time dimensions. The system progressively scans and statistically analyzes the activation status of each dimension combination within the current tensor slice, using the activation status of each high-dimensional combination (i.e., the coordinate position in the tensor) at that time step as an activity signal. Activation refers to the presence of a non-zero value in the current tensor slice or the system detecting that the dimension combination has generated behavioral data. Next, the system evaluates the activity of each dimension combination across time windows, including statistically analyzing whether it has exhibited activation behavior over multiple consecutive time steps, as well as the frequency and distribution trend of activation, thereby forming an activity mapping organized by dimension combinations. After the activity assessment is completed, the system performs dilution processing on high-dimensional combinations that remain inactive for a preset observation period. This involves removing these tensor dimension combinations that are determined to be long-term silent or inactive from the current tensor structure to save memory space and improve computational efficiency. However, to ensure that these diluted channels can be recovered in the event of abnormal fluctuations or structural recurrence in the future, a shadow indexing mechanism is introduced. After the removal operation is completed, the system does not completely delete the information of the channel, but retains its index identifier, last activation time, historical data summary and other meta-information in a lightweight form. This shadow index will be mounted in the external cache table of the sparse tensor structure and supports the reactivation of the channel and restoration of its structure and historical trajectory in cases of anomaly detection, anti-sparse backtesting or structural drift recovery, thereby ensuring the reversibility of sparse processing and the robustness of the system.

[0023] During the processing of newly added incremental data, the system identifies potentially active tensor dimension channels for each newly accessed tensor slice. These channels are those that, while exhibiting active behavior (e.g., non-zero values, event triggers, or data flow changes) at the current time step, have been inactive or exhibited low-frequency activity in previous consecutive time steps. The system utilizes an activity state transition detection mechanism to mark these potentially active channels and add them to a candidate activation pool. Subsequently, within a preset time window, the system dynamically tracks and extracts features from their active behavior in subsequent consecutive time slices. The time window can be set to a fixed length or a sliding window according to business needs, typically covering several time segments to ensure the capture of short-term trends and inertial behaviors of structural changes. Within this time window, the system calculates the continuous activation characteristics of each candidate channel, including statistical indicators such as activation count, continuity, activity frequency, and activity distribution skewness, and compares these with a set continuity threshold. If the continuity of a channel does not meet the threshold requirement, it is considered a short-term disturbance or noise trigger and is not included in the tensor structure for explicit modeling, but instead enters an inertial monitoring state. For channels in an inert state, the system does not immediately construct their complete channel space within the tensor structure, nor does it allocate computational resources for decomposition and backtesting. Instead, it continuously collects key parameters of their active trajectories, such as intermittent activation time points, maximum intervals, trend slopes, and rates of change, through a lightweight evolution trend recording mechanism, and constructs a time evolution vector for subsequent decision-making reference. Through this inertial propagation mechanism, the system effectively avoids frequent structural reorganization operations caused by short-term fluctuations, thereby significantly reducing the volatility and resource overhead of the tensor structure while ensuring the response to potentially activated channels. This enables rhythmic regulation and stability control of the structural evolution process, and provides a more valuable historical behavioral information basis for subsequent anomaly detection, structural upgrade, or reconstruction strategies.

[0024] During the continuous evolution of tensor flow, to effectively control the computational cost and response latency of backtesting, the system dynamically compares the structural information of each newly added tensor slice with its previous time step or historical reference state. This comparison includes, but is not limited to, the analysis of differences in the distribution of non-zero elements, dimensional activation density, range of structural coordinate changes, and sparse mode evolution. Based on this, the system constructs a structural offset intensity quantification model to measure the structural changes between the newly added tensor slice and the reference state. This offset intensity considers not only the number of added or deleted activation channels but also incorporates feature factors such as structural entropy changes, distribution centroid shift, and path topological similarity, forming a composite structural offset index that comprehensively reflects the magnitude and trend of the tensor structure's transformation from the original state to the new state. To ensure the efficiency of the backtesting strategy, the system sets an offset entropy threshold calculated based on the historical evolution trajectory. This threshold can be dynamically adjusted by combining the range of structural fluctuations in historical stable periods and the overall sparsity of the tensor. When the offset intensity is less than or equal to the threshold, it indicates that the current structural change is within the expected tolerance range. The system then skips the backtesting operation for the historical time period associated with the current tensor piece and directly uses the previously saved backtesting snapshot results to replace the calculation, thereby saving system resources and improving response speed. If the offset intensity exceeds the set threshold, the system determines that the structural change introduced by the newly added tensor piece has reached the level of significant variation, and local backtesting of its related tensor quantum blocks is required. This process adopts a compressed backtesting strategy, which only performs incremental backtesting and verification on key dimension combinations, change nodes, or path connections within the structural offset region, avoiding full recalculation. This significantly reduces the computational load while ensuring backtesting accuracy, realizing a dynamic backtesting mechanism that is sensitive to structural changes, has controllable computational accuracy, and uses resources efficiently.

[0025] During the continuous access and evolution of tensor streams, the system monitors incremental data in real time through a defined anomaly detection mechanism. When an anomaly is detected, the system initiates an anomaly-triggered structural backtesting process. Anomalies include, but are not limited to, structural mutations (such as the addition of tensor dimensions, rearrangement of dimension combinations, or mutations in activation density) and significant spikes in tensor reconstruction residuals. The system continuously monitors the dynamics of the residual matrix between the prediction error after tensor decomposition and the actual observations. If the anomaly indicators exceed the set multidimensional trigger threshold within a specific time window, it is determined to be a high-risk evolution phenomenon. At this time, the system uses the current anomaly channel and its related dimension combinations as the root node, backtracks and retrieves dimension paths that have previously undergone dilution operations or were not activated due to lazy strategies in the tensor structure history, and restores their original structural state based on the timestamp, channel status, and active trajectory records in their shadow indexes, reconstructing their position and attributes in the tensor data structure, thereby completing the retrospective activation of the historical structure. Next, the system performs an anti-sparse backtesting process based on the recovered channels. This process is not the same as regular backtesting, but only performs backtesting compensation for the activated paths. It prioritizes selecting time periods and structural regions that have historically had high coupling or high residual correlation with the current anomalous node. The system quickly estimates the potential contribution and evolutionary impact of these regions using a lightweight backtesting model, and forms a set of candidate anomaly propagation paths. The system calculates the residual diffusion rate, propagation depth, and degree of linkage with adjacent structures in these paths, and assigns dynamic impact scores to each node in the path to identify the secondary structural regions to which the anomaly has spread. Ultimately, it achieves intelligent compensation and spatial positioning of potentially affected areas, thereby improving the system's response speed to sudden structural disturbances, backtesting coverage, and structural self-recovery capability, effectively supporting the stability and anomaly self-healing mechanism of tensor flow in sparse high-frequency environments.

[0026] Combined with appendix Figure 2The shadow index mechanism is used to facilitate the dynamic management and backtracking recovery of sparse structures. Specifically, during the incremental access and structural evolution of tensor stream data, when the system identifies certain dimension combinations as continuously inactive or low-frequency active within multiple time windows based on the activity detection mechanism, and they are removed from the tensor structure due to meeting the dilution conditions, the system does not directly destroy the information of that channel. Instead, it preserves its key attributes by constructing a shadow index table. This shadow index includes at least two core fields: one is a unique key value, used to identify the coordinate position of the removed dimension combination in the original tensor structure. This key value is usually generated by the index sequence of the combined dimensions through hash mapping or concatenation encoding to ensure uniqueness and retrieval in the global structure; the other is the last active timestamp, used to record the specific time point when the system last observed the activation behavior of the channel. This timestamp can be used as the basis for subsequently judging its recovery priority or participating in abnormal backtesting scheduling. When an abnormal event occurs, the system can quickly locate and reconstruct the channel structure based on this. To ensure the dynamic adaptability of dilution judgment, the system also performs real-time assessment of channel activity for each dimension combination. Specifically, it counts the frequency of access or activation of each dimension combination within a unit time window, i.e., records the number of times it has a non-zero value or is included in the calculation. The system constructs an activity distribution based on this frequency change and uses a multi-level threshold mechanism for state determination. These multi-level thresholds include high activity threshold, critical activity threshold, and dilution threshold, which correspond to three state transition paths: regular retention, lazy tracking, and dilution removal of channels, respectively. The system can dynamically adjust these threshold parameters according to the sparsity density of the overall tensor structure, the adaptive evolution strategy, and the computing resources available. This achieves efficient compression of structural redundancy and early warning retention of potential channel activation, thereby maximizing the compression of the tensor storage structure and optimizing the efficiency of subsequent backtesting resource utilization without losing key structural information.

[0027] The dilution process not only includes removing dimension combinations that have been inactive for a long time, but also further refines it into a state management mechanism. Dimension combinations that meet the dilution conditions are not directly removed, but are marked as frozen. The frozen state is an intermediate state between activation and deletion. Channels in this state do not participate in the storage and computation of the tensor main structure, but their metadata information is retained in the system structure mapping table through shadow indexes. When the system detects an abnormal event or executes a periodic structure backtesting task, if these frozen dimension combinations meet the reactivation conditions, such as the intersection of their historical active behavior with the current abnormal path, or their cooling period reaching a set upper limit, then the channel will be restored to a computable state, reintegrated into the tensor structure, and participate in backtesting or structure evaluation. Meanwhile, in the inertial propagation mechanism, to avoid unnecessary structural oscillations caused by short-term active fluctuations, the system records the evolutionary trends of potentially activated dimensional combinations that have not yet met the structural inclusion criteria. It then models their activation frequency using an exponentially weighted moving average (EWMA) method, assigning higher weight to recently observed activation behaviors. This smooths the long-term active potential of dimensional combinations, thereby more accurately determining their future evolutionary direction and structural stability, and providing a quantitative basis for structural decisions. Furthermore, to further enhance the accuracy of determining persistent change characteristics, the system not only counts the consecutive occurrences of dimensional combinations within the historical time window but also evaluates their maximum interval time. This jointly measures whether the channel possesses continuity and stability characteristics. Specifically, only when the frequency of a dimensional combination within the window reaches a set threshold, and the maximum time interval between any two occurrences does not exceed the tolerance limit, is the channel considered to have a sufficient persistent change trend. Otherwise, it continues to maintain an inert state or enters the frozen state candidate queue. This achieves fine-grained stratification and evolutionary control of structural states, ensuring the system's dynamic adaptability and stability to sparse tensor flows.

[0028] To achieve accurate identification and response control of tensor structure changes, the system calculates the structural offset intensity for each new tensor slice and its previous state. The structural offset intensity can be achieved in two ways: First, by using the difference in tensor structure hash fingerprints, the active channels of each dimension in the tensor structure are combined to generate a feature fingerprint hash value. The degree of structural change is quantified by comparing the difference in hash signatures between the current tensor slice and historical slices. This method has the advantages of high efficiency and compressed computation, and is suitable for high-frequency structure detection scenarios. Second, the offset intensity is measured by calculating the overlap rate of the tensor structure in the feature subspace. That is, the structural projection regions of the new and old tensor slices in the low-rank decomposition or activation space are compared. If the overlap ratio is low, it indicates that the structural offset is large, thereby triggering the adjustment of the subsequent backtesting strategy. To improve the adaptability of the offset intensity judgment, the offset entropy threshold set by the system is not a static value, but an adaptive threshold that is dynamically adjusted according to the historical tensor evolution trajectory. This includes modeling the frequency of structural offsets, the range of activity fluctuations, and the trend of residual entropy changes in several past time windows. By calculating the average structural perturbation degree and introducing the fluctuation variance to construct the entropy basis reference interval, the system further combines the global sparsity density factor, i.e., the non-zero proportion of the overall tensor and the channel compression ratio, to adjust the sensitivity of the threshold. This ensures that in a data environment with strong sparsity, unnecessary backtesting tasks will not be frequently triggered due to small structural changes, thereby achieving a balance between the accuracy and robustness of offset detection. In addition, to improve the efficiency and response speed of the backtesting process, the system generates a backtesting snapshot after performing historical backtesting operations. This snapshot is a cached copy of the backtesting calculation results under the corresponding tensor structure state, and contains the bound time identifier and structure version number. Through this mechanism, when the offset intensity does not reach the threshold, the system can directly call the snapshot to reuse the results without repeating the calculation. At the same time, it supports quickly finding and restoring historical results when the structure version is switched, thereby improving the system's processing efficiency and response capability in large-scale tensor flow evolution scenarios.

[0029] The compressed backtesting design addresses the computational efficiency bottleneck caused by local rather than global tensor structure shifts, performing backtesting only on tensor quantum blocks within the region where the structural shift occurs. When the system determines, based on the structural shift intensity, that the structural change in the current tensor slice has not affected the global scope but has significant changes in certain dimension combinations or substructures, the system first locates the tensor quantum block range corresponding to the shift, including the specific activated dimension combinations, the affected time period, and the relevant channel paths. Then, it performs local backtesting within this sub-block, performing low-rank decomposition reconstruction, index recalculation, or strategy evaluation only on the relevant parts, avoiding redundant processing of the entire tensor to reduce resource overhead. After the local backtesting is completed, the system further incorporates historical backtesting residuals for prediction error estimation. That is, the system compares the results of the current sub-block backtesting with the residuals of the corresponding time period in previous snapshots or historical backtests for that sub-block. By analyzing the residual change trend, error amplification degree, and predicted shift, it infers whether the current structural evolution has a systematic impact, thereby assisting in decisions on whether to further expand the backtesting range or initiate anomaly detection procedures. In case of anomalies... In terms of event detection, the system is designed with a joint triggering mechanism that integrates residual tensor burst identification and structural variation map analysis. Residual burst identification refers to the system continuously monitoring tensor decomposition and reconstruction errors. When the error value rises rapidly compared to the historical average or the fluctuation exceeds a preset threshold within a certain period, it is judged as a residual burst anomaly. At the same time, the system maintains a structural variation map, which records the structural connections between tensor dimensions, active paths, and channel jump behaviors. Once a node in the map experiences an activation state jump, path interruption, or frequent channel reconstruction in a short period of time, it indicates that structural variation has occurred. Under the premise of simultaneously satisfying the above residual burst and map node anomaly, the system triggers the abnormal event identification process, thereby initiating the anti-sparse backtesting mechanism. This enables precise positioning, accelerated response, and structural recovery control of high-risk evolution regions, comprehensively improving the system's monitoring and processing capabilities for nonlinear evolution risks in incremental data.

[0030] Combined with appendix Figure 3The structural mutation identification and anti-sparse backtesting mechanism is used to improve the system's anomaly response capability and structural adaptability in the face of dynamically evolving data scenarios. Specifically, the system continuously monitors the stability of the tensor structure during the continuous access of tensor streams. When a structural mutation event is detected, the anomaly backtesting link is triggered. The structural mutation includes three types of situations: First, the addition of a tensor dimension, that is, the system receives a new type of dimension item for the first time, such as a new user, a new sensor, or a new indicator. This type of addition will directly expand the original tensor dimension range, constituting an expansion of the structural boundary. Second, the rearrangement of dimension combinations, that is, the combination pattern between the original dimensions changes. Due to data source switching, behavioral logic changes, or system configuration adjustments, common tensor coordinate combinations are rearranged or shifted, causing the original structural indexing system to fail. Third, the activation density mutation, which refers to a drastic change in the number of non-zero activation points of the tensor within a unit time window, such as a mutation from sparse to dense or a sudden change from dense to sparse. This behavior is usually accompanied by systemic events or policy switching and affects backtesting accuracy and policy response. The system can judge a structural mutation when any one of these three mutation types or a combination of them occurs. In response to the structural changes caused by the aforementioned mutations, the system initiates an anti-sparse backtesting process, which includes three steps: First, the previously diluted tensor dimension channels are reactivated, that is, dimension combinations that have been judged to be inactive and temporarily removed from shadow indexes or frozen states are recovered, reattached to the tensor structure, and their index positions and metadata are assigned; second, the system loads the behavioral trajectory of the channel from historical data, including its activation history, residual changes, and contextual dimension relationships involved in the calculation in multiple time segments, to complete its evolution path and restore historical influencing factors; finally, the system... The system performs periodic backtesting on the recovered channel, that is, multiple backtests are conducted within a set time window. By gradually updating its residual value and prediction offset results, the system can estimate whether the channel has a direct or indirect impact on the current anomaly. If its impact is wide or the propagation path extends to other key dimensions, the system can dynamically expand the anti-sparse range or trigger the structure freezing strategy accordingly. This enables timely response, accurate positioning and efficient compensation for sudden structural evolution behavior, ensuring that the tensor structure maintains recoverability and inference coherence when facing high-dimensional sparse dynamic changes.

[0031] To address the dynamic structural mutation problem caused by the evolution of tensor structures with incremental data, the system introduces a mechanism for identifying and backtesting two typical mutation behaviors—dimensional addition and dimension combination rearrangement—when performing structural mutation monitoring tasks. First, when the system detects a tensor dimension addition event—that is, a dimension item not previously present in the tensor structure appears for the first time in the current incremental data (e.g., a new user ID, sensor number, or behavior type is introduced into the tensor dimension set)—the system immediately identifies it as a structural boundary expansion mutation. Upon this mutation, based on the shadow index in the original sparse structure, it retrieves all historical dilution channels that have potential combination relationships with the newly added dimension. The system uses hash matching and structural topology alignment to identify whether these new combinations formed by the intersection with the new dimension have been diluted in history. These potential channels are then de-sparse activated, meaning their structural definitions and meta-information are restored from the frozen state or index table, and their status is set as high-priority backtesting candidates. This ensures that the system can promptly obtain the policy response capability related to the historical structure after the introduction of a new dimension, thereby improving the completeness of anomaly tracing. In terms of integrity and strategy compatibility, the system continuously tracks the changes in the dimension arrangement order and coordinate combination of tensors in each time slice. If it finds asymmetric changes such as misalignment, index order change, or storage rearrangement in the original dimension combination pattern of the structure, such as the user-time-behavior structure being replaced by the time-behavior-user structure, the system will judge it as a combination rearrangement event. Although this event does not introduce new dimensions, it causes changes in the tensor access path and the failure of some channel indexes. To address this, the system introduces an active region comparison mechanism. By comparing the symmetry of the distribution of active channels and the path coverage in the tensor before and after the change, it identifies whether there is a structural mapping offset. If the offset reaches a set threshold, the combination rearrangement is confirmed to be valid, and a structural mapping reconstruction operation is performed. That is, based on the historical mapping relationship of the tensor dimension index, the index order and position mapping corresponding to the original dimension combination are reconstructed to achieve the connection between the historical backtesting trajectory and the current structure. This avoids policy backtesting gaps or abnormal interpretation distortions caused by changes in combination logic, and ensures the structural continuity and analytical coherence of the system in the scenario of dynamic reconstruction of tensor dimensions.

[0032] To achieve rapid perception and response to abrupt changes in the activity of abnormal structures, the system designs a mutation identification mechanism based on the trend of activation density changes. The judgment of activation density mutation does not depend on the number of activations at a single time point, but introduces the second derivative of the number of non-zero activation elements in the tensor within a unit time window as the judgment criterion. That is, the system first samples the sequence of activation numbers within multiple consecutive time windows, then calculates the first derivative of the sequence to represent the rate of change of activation numbers, and further calculates the second derivative to measure the degree of mutation of the rate of change itself, thereby identifying nonlinear abrupt change behavior. When the result of the second derivative exceeds the activation density mutation threshold set by the system, it is considered that the tensor activity state at the current moment has abruptly changed. Such mutations usually indicate the existence of potential high-risk evolution events in the system, such as sudden policy activation, large-scale behavior switching, or high incidence of abnormal access. Based on this, the system immediately triggers the anti-sparse backtesting process. In this process, the system scans and reactivates each diluted channel associated with the mutation activation region under the current structural state. The system extracts historical metadata of the diluted channels from the shadow index and restores their channel positions and computational weights in the tensor structure. Simultaneously, after the activation action is completed, the system generates a corresponding evolution time vector for each activated channel. This vector records the time evolution trajectory of the channel from initial activation to the current moment, including its active time distribution, active interval, peak time point, and other statistical information, which is used to express the dynamic state of the channel over time. Subsequently, the system performs alignment analysis between this evolution time vector and the anomaly detection moment that triggered the activation. Through time overlap calculation or sliding window matching, it locates whether the channel had significant behavioral expression during the anomaly outbreak period, thereby clearly marking the historical influence range of the channel and backtesting compensation. Ultimately, this supports the system to achieve accurate temporal positioning, spatial path restoration, and anomaly cause tracing, improving the sensitivity of the backtesting system to nonlinear structural mutations and the timeliness of structural evolution response.

[0033] To enhance the system's ability to continuously monitor the evolution trend of anomalous channels and improve the accuracy of identifying anomalous influence regions, an anti-sparse backtesting enhancement mechanism based on periodic anomaly judgment and anomalous residual diffusion modeling is proposed. Specifically, when an activated sparse channel undergoes anti-sparse backtesting, and the anomalous indicators (such as reconstruction residuals, prediction offsets, and behavior surge rates) observed in two consecutive independent time periods all exceed the system's preset tolerance limits... At this point, the system marks the channel as a persistently anomalous channel, considering it not only the core source of the current anomaly but also a potential risk of persistent structural instability. Therefore, it is necessary to strengthen the subsequent monitoring weight of this channel and adjust its processing priority in the sparse strategy. To further identify the potential ripple effects of this anomalous channel on other regions of the surrounding tensor structure, the system introduces an anomaly impact range estimation mechanism at this stage. A convolutional propagation algorithm with a time decay factor is used to perform multi-dimensional diffusion modeling of the channel's anomalous behavior. The specific propagation model is expressed as follows: in, Indicates indexing in tensor dimensions The cumulative intensity of abnormal diffusion on the surface serves as an important basis for identifying whether the structural region is a secondary affected area; variables For time, This is the time point of the first anomaly detection. This indicates the total duration of the observation and backtesting window; Main abnormal channel In time The abnormal activation intensity reflects the intensity of its sudden residual; Indicates the main channel With target structure The structural coupling factor between them quantifies their collaborative relationship in historical paths. This parameter is dynamically generated based on the co-occurrence probability of dimension combination. A Gaussian time decay kernel is used to control the decay rate of anomaly propagation over time. (Parameters...) Indicates the decay rate. The larger the size, the more concentrated the spread becomes in the early stages of the anomaly; For linkage indicator functions, if in time Upper passage With structure If a co-activation relationship exists, the value is 1; otherwise, it is 0. After the system completes the calculation of the anomaly propagation integral, if the result... Exceeding the set ripple intensity threshold Then determine the structure As secondary affected areas, they need to be included in the subsequent anti-sparse backtesting compensation scope, and their historical behavior trajectories need to be audited and replenished. If multiple structural regions that meet the above propagation conditions are continuously detected under the dominance of a certain abnormal channel, the system will further trigger the structure freezing early warning mechanism, put the main channel into the freezing queue, and restrict its future inclusion in the sparsity elimination strategy priority. This ensures that the channel obtains a higher fault tolerance retention level in the system's tensor structure management and strategy response, thereby improving the system's early warning capability for persistent hidden anomalies and its structural stability control capability.

[0034] Example 1: Combined with appendix Figure 4 In this embodiment, a large-scale intelligent traffic management platform collects data streams in real time from tens of thousands of sensors distributed across various intersections, bus stops, and highway entrances. The collected multidimensional information includes traffic flow, vehicle speed, weather, and time. The system organizes this data into a four-dimensional tensor structure, specifically a sparse tensor stream composed of (road segment ID, sensor type, timestamp, and environmental dimension) as dimensions. Due to the prevalence of discontinuous behaviors in urban traffic scenarios, such as no traffic at remote intersections at night or temporary closure of highway entrances during holidays, many dimension combinations remain inactive for extended periods. During the initial phase of operation, incremental statistics on data from one month revealed that over 30,000 dimension combinations did not generate activation signals for most of the time. To conserve tensor structure storage and computational resources, the system initiates a dilution process, relying on dynamic statistics of channel activity to calculate the activation frequency of each dimension combination within a 10-minute time unit. Taking "Road Segment ID=R305, Sensor Type=S12, Environmental Dimension=Sunny" as an example, this combination was activated only during two peak periods within 7 days, with zero frequency at other times. The system judged it to be in a low-activity range. Based on the set multi-level activity threshold model (high-frequency retention threshold 0.8, boundary inertia threshold 0.3, dilution removal threshold 0.1), the activity frequency of this combination was only 0.02, far below the dilution threshold. The system decided to perform a removal operation. However, to support future activation rebounds or anomaly backtracking, the system built a shadow index for this combination, recording the unique key value "R305-S12-Sunny" and its last activation timestamp "2024-05-18 07:10:00" in the shadow table. Two weeks later, due to a traffic accident on this road segment, the traffic flow monitoring model detected continuous abnormal fluctuations in vehicle speed data in this area. At the same time, multiple related dimension combinations showed a significant increase in residuals. The system triggered the anomaly detection module and traced back to check if there were any historical dilution paths as potential anomaly sources. The shadow indexing mechanism immediately took effect. The system scanned the shadow table and found that the dimension combination was located in the sensor path at the location of the incident. It had not been activated for 15 consecutive days after its last activity, but the current region showed a high residual anomaly. Therefore, the system immediately reconstructed its structure, that is, restored the position and state of the dimension combination in the tensor structure, changed it from a frozen state to an active state, and included it in the anomaly anti-sparse backtesting process. Subsequently, the system retrieved the prediction error trajectory of the combination before and after the most recent activity from the backtesting snapshot, and completed the backtesting compensation with the latest data. Finally, it was found that the channel had already shown a slight structural shift signal the night before the incident, but it did not trigger the main model response due to low activity. This reconstruction compensation enabled the model to identify the early signal before the incident, thus verifying the feasibility of shadow indexing in anomaly backtesting and the necessity of the multi-level channel activity adjustment mechanism to maintain the flexibility of sparse structures.

[0035] Further demonstrate the synergistic effect of its core mechanism when processing urban-level traffic tensor data. When the system detects that the dimension combination "R305-S12-Sunny" is in a very low active state and triggers the dilution process, considering that this dimension combination has only been activated twice in the past 30 days, the activation frequency is much lower than the dilution threshold of 0.1, but it is located on the urban main road section and in the high-accident area, the system does not directly delete this dimension combination. Instead, it sets its status to the frozen state and records the unique key, the reason for freezing, and the last activation time "2024-05-18 07:10:00" of this dimension combination in the shadow index table. After entering the frozen state, this channel does not participate in the tensor decomposition and backtesting operations, but is still tracked by the system's lazy propagation mechanism. The system uses exponential weighted moving average (EWMA) to model the trend of its activation frequency. The two recent activation time points are "2024-05-08 08:30:00" and "2024-05-18 07:10:00" respectively. By setting the weight factor α = 0.6 for weighting, the system evaluates the probability of its re-activation within the next 7 days to be 0.32, which is lower than the lazy threshold of 0.5 but higher than the dilution elimination threshold. Therefore, this dimension combination is maintained in the frozen state, and its active trend vector is recorded for reference during subsequent anomaly detection. After an accident occurs, the system starts the anomaly detection process. By combining the monitoring of the sudden increase in the backtesting residual tensor and the identification of the structural variation map, it is found that there is a mutation in the path where "R305-S12-Sunny" is located. Multiple nodes in the map show abnormal activation jumps. For example, "R305-S12-Rainy" and "R304-S13-Sunny" are active simultaneously within a short period of time and do not appear in the previous path co-occurrence relationship. The system quantifies the abnormal change by calculating the structural deviation intensity. Using the hash fingerprint difference method, it is found that the Jaccard similarity between the current tensor fingerprint and the state 3 days ago has dropped to 0.72. Combined with the subspace overlap rate dropping to 64%, the system determines that the deviation degree exceeds the set deviation entropy threshold of 0.75 for this stage. This threshold is dynamically generated by combining the historical fluctuation amplitude and the current tensor sparsity rate of 0.19. At this time, the system restores the frozen channel of "R305-S12-Sunny" and extracts the backtesting result of this combination at the last activation from the snapshot cache. The binding time of this snapshot is "2024-05-18 07:10:00", and the structure version number is V13-3f. Through fast indexing, the system extracts its residual baseline of 0.022 and compares it with the current backtesting value of 0.106, confirming a significant anomaly. Subsequently, a compressed backtest is performed on this structural sub-block, and only local reconstruction is performed on the 5 dimension combinations related to "R305". The local prediction offset value is quickly obtained, and it is found that the maximum prediction offset appears in the combination "R305-S12-Sunny-High Humidity", and its error has increased by 4.7 times compared to the historical benchmark. Based on this result, the system determines that the accident signal was first manifested in this combination, and then marks it as the priority traceability path.The entire process ensures that low-activity channels are not permanently removed through a freeze-state mechanism, improves the accuracy of channel evolution trend determination through EWMA and maximum interval joint modeling, effectively identifies structural anomalies through dynamic matching of structural offset intensity and entropy threshold, ensures the system's response efficiency under high-frequency streaming data through backtesting snapshot mechanism and compressed backtesting methods, and provides intuitive localization of mutation behavior through structural spectrum changes in anomaly detection.

[0036] Example 2: Combined with appendix Figure 5One morning, the traffic data received by the system suddenly showed an anomaly in its tensor structure, triggering the structural mutation identification mechanism. First, the system discovered a new label, "dust storm," in the "Weather Status" dimension. This label originated from the newly integrated meteorological subsystem, adding a new event to the tensor dimension by combining the "Road ID-Sensor Type-Time-Weather" dimension in the original tensor structure. Second, due to the platform's rearrangement of the deployment order of some sensors after adjusting the urban road structure, the system found that the original "User-Behavior-Time" structure had been switched to "Behavior-User-Time" when receiving incremental data. Although this change did not increase the number of dimensions, it caused an asymmetric jump in the index structure, which the system used to identify the rearrangement of dimension combinations. Third, in the traffic data of the previous hour, the traffic flow data of multiple highway entrances suddenly surged during the morning rush hour. The system compared the number of non-zero tensor elements within a 10-minute sliding window and found that the number of active elements per unit time increased by 3.2 times, exceeding twice the standard deviation of the historical average density of the area, clearly indicating a mutation in activation density. Faced with the combination of these three mutation types, the system initiated an anti-sparse backtesting process. First, it retrieved frozen channels related to the new dimension "dust" combination. In the shadow index table, it found the dimension combination "R317-S14-dust". This combination had not appeared in the meteorological dimension for a long time and had been diluted within the previous 90 days, with its last active time being "2024-04-08 06:00:00". The system immediately restored this channel from its frozen state to its principal tensor structure and attached its original dimension information and data index. Subsequently, the platform loaded the channel's historical trajectory, calling its tensor data snapshots and residual records before and after the most recent activation. It found that its historical maximum residual value was 0.019, while the residual for the current detection period was 0.085, indicating a significantly abnormal change. Based on this, the system determined that the channel's anomaly level was higher than the median risk. To further quantify its potential impact on surrounding structures, the system performed periodic backtests on the active paths of this channel over the past 12 hours, employing a strategy of one backtest cycle every 15 minutes. The backtesting included comparisons with other combinations in the structurally involved paths, residual fluctuations, and offset contribution analysis. Ultimately, it was found that the residual values ​​in the two sub-combinations "R317-S14-dust-high humidity" and "R318-S14-dust-low visibility" remained consistently high, with average errors of 0.067 and 0.073, respectively. Six combinations in the corresponding propagation paths exhibited similar fluctuations. Based on the system's defined rules for determining the depth of the affected path and the propagation intensity, these combinations were included in the secondary affected area. Furthermore, the main channel underwent weight upgrades and removal of freeze restrictions to ensure that the channel would not be easily diluted in future sparsity strategies. Throughout the process, the system triggered an anti-sparse mechanism based on three types of structural mutations, establishing a complete closed-loop response encompassing path-related, periodic, and anomaly-sensitive aspects, in addition to structural recovery, data trajectory loading, and backtesting compensation.

[0037] A sudden weather-traffic linkage event occurred. At 8:00 AM on the day of the incident, a new weather status label, "Sandstorm-Warning," was added to the tensor data accessed by the platform. This label was dynamically inserted into the traffic data dimension by the National Meteorological System through emergency notifications. The system immediately identified it as a structural mutation triggered by the addition of a new tensor dimension. The platform first scanned all frozen dimension combinations related to the new dimension "Sandstorm-Warning" in the shadow index and found that combinations such as "R205-S10-Sandstorm-Warning" and "R210-S10-Sandstorm-Warning-Morning Rush Hour" had been frozen three months prior due to long-term inactivity, with an activity frequency of less than 0.01 at that time. The last active period was concentrated during the sandstorm anomaly in early February. At this point, the system performed anti-sparse activation operations on these combinations based on the newly added dimension, reattached their positions in the tensor structure, increased their status priority, added them to the emergency backtesting queue, and assigned highly sensitive backtesting parameter configurations to complete error judgment in the shortest possible time. Subsequently, the platform's structural mutation detection module further identified a rearrangement of the "behavior-time-road segment" dimension combination order in recent data. This was due to the traffic perception system prioritizing the position order of some behavioral data and timestamps during hardware upgrades. The original "R205-S10-sandstorm-warning-08:00" was changed to "08:00-S10-R205-sandstorm-warning". The system detected that the active areas in the original structure were asymmetrically distributed in the new structure, with an activation path offset rate of 68%, which exceeded the set combination rearrangement judgment threshold of 50%. Therefore, it confirmed that a structural combination jump had occurred and immediately called the historical combination mapping table to map the original dimension arrangement order to the current index path, completing the accurate docking of historical backtesting trajectories and avoiding backtesting offsets or breakpoint jumps due to structural mismatch. Meanwhile, the system detected drastic changes in activation density on multiple main urban roads within the past hour. Specifically, the activation element growth rate of the "R205-S10-sandstorm-warning-morning rush hour" dimension combination increased from 1.2 to 7.8 within a 5-minute window, while the average growth rate of activations on this road segment over the past 7 days was only 0.6. The platform calculated the second derivative change value of this indicator sequence, determining that the abrupt change slope exceeded the system's set threshold, immediately triggering an anti-sparse process and adding this structural combination to the abnormal high-frequency monitoring list. Simultaneously with the reactivation of this structure, the platform constructed a channel evolution time vector. Comparing its activity trajectory over the past 72 hours, the system found that this channel was only activated once each on March 2nd and March 6th. The current sudden activation behavior had zero overlap with historical behavior periods, thus it was judged as an independent new cycle anomaly. The evolution vector was used to align with the anomaly detection time window, and the system located the initial anomaly at 07:52, using this as a starting point for local backtesting expansion to further identify downstream affected structural areas.Through this process, the platform system, when faced with three complex structural perturbation scenarios—addition of tensor dimensions, combination rearrangement, and activation density mutation—completes the entire chain of structural reconstruction, backtesting reentry, and risk path identification by using frozen state index recovery, mapping reconstruction, and time alignment back-inference mechanisms.

[0038] Following the aforementioned response to the structural abrupt change triggered by the "dust storm warning," the platform continued to periodically backtest and monitor the anomalous intensity of the restored and activated sparse dimension channel "R205-S10-dust storm-warning-morning rush hour." The platform set the backtesting cycle to once every 10 minutes. In the two cycles at 08:00 and 08:10, the prediction error residuals for this channel were 0.081 and 0.086, respectively, far exceeding the system's currently set tolerance limits. Since the threshold was exceeded for two consecutive periods, the system immediately marked the channel as a "continuously abnormal channel" and changed its status from "high priority backtest" to "structure freeze warning" to prevent the channel from being mistakenly removed during subsequent data dilution. At the same time, the system enabled the channel to model the abnormal diffusion of surrounding tensile quantum structures and identify the affected areas.

[0039] Therefore, the platform adopts an anomaly propagation weight integral function. The influence intensity of the main channel "R205-S10-Dust Storm-Early Warning" on other channels during the current anomalous event is modeled using convolutional propagation. The system's observation time window is set as follows: Minutes, the time of the first anomaly detection is The platform's sampling frequency is 1 minute, meaning the calculation will be a discrete approximation across 30 sampling points. The anomalous activation intensity of this channel is known to be sampled as follows over the next time period: The system selected the target substructures "R205-S11-Dust Storm-Early Warning" and "R206-S10-Dust Storm-Early Warning," which have historically co-occurred with the main channel, as the simulation objects. The path co-occurrence probability of the former is... The latter is The time-domain decay kernel uses a Gaussian distributed kernel function, and the platform sets the diffusion rate parameter to be... To smooth out the diffusion intensity at times far from the anomaly center. Activation indicator function. Based on the behavior logs extracted by the system, the linkage states of the two substructures mentioned above within this time window are as follows: “R205-S11-Dust Storm Warning”: “R206-S10-Dust Storm Warning”: Substitute these actual data into the following weighted integral function: Since the sampling points are discrete and evenly spaced at 1-minute intervals, the system approximates this integral as a weighted sum using a discrete approach: For example, for "R205-S11-Dust Storm-Warning", taking the previous 10 minutes as an example, the calculation is as follows: , , , , Cumulative contributions are: The system-set saturation threshold is Therefore when This indicates that the substructure has been affected by the residual diffusion of the abnormal channel and needs to be included in the backtest compensation. The results show that the propagation intensity of "R205-S11-Sandstorm-Warning" exceeds 1.2 and "R206-S10-Sandstorm-Warning" is 0.81 after calculation, both exceeding the threshold. Based on this, the system marks them as secondary affected structures and calls their snapshots and behavior logs from the past 3 hours to perform targeted backtest compensation.

[0040] Ultimately, because more than three substructures have been affected by the abnormal propagation, the system, based on the rules of propagation chain length and number of nodes, performs a structure freeze warning for the main channel "R205-S10-sandstorm-early warning", and increases its removal weight in the sparse control strategy to the freeze level, so as to ensure that it can be continuously retained and receive stable backtesting attention in the future evolution cycle.

[0041] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A sparse tensor flow backtesting method for incremental data, characterized in that... include: Incremental access analysis is performed on tensor stream data organized in a multidimensional structure. Channel activity is detected based on inactive dimension combinations in each time step. High-dimensional combinations that have not been activated for a long time are diluted and removed from the current tensor structure, while shadow indexes are retained for future activation. When processing new incremental data, identify potentially activated tensor dimension channels, determine the continuous change characteristics within a preset time window, and if the continuous threshold is not reached, record the evolution trend to form an inertial propagation mechanism to reduce structural fluctuations. During the tensor flow evolution, the structure of any newly added tensor slice is compared with that of the previous state, and the structural offset intensity is calculated. If the offset intensity does not exceed the set offset entropy threshold in the corresponding historical time period, the backtesting process of that historical time period is skipped and a backtesting snapshot is used. Otherwise, perform compressed backtesting on tensor blocks where the structure has changed significantly; When anomalies are detected in the incremental tensor flow, including structural mutations or a significant surge in reconstruction residuals, the tensor dimension paths that were previously diluted or not activated due to inertial strategies are traced back based on the anomaly triggering mechanism. Their structural states are reconstructed, and an anti-sparse backtesting process is performed to identify and track the anomaly propagation path in order to compensate for and locate the potential affected areas.

2. The sparse tensor flow backtesting method for incremental data as described in claim 1, characterized in that... The shadow index includes a unique key value for recording the removed dimension combination and the last active timestamp, which is used for structural reconstruction when an anomaly is triggered. The channel activity is dynamically updated by statistically analyzing the frequency of access to the dimension combination within a unit of time, and multiple activity thresholds are set to adjust the dilution intensity.

3. The sparse tensor flow backtesting method for incremental data as described in claim 2, characterized in that... The dilution process includes marking dimensional combinations that meet the dilution conditions as frozen states, which are then reactivated during abnormal activation or periodic re-inspection; in the inertial propagation mechanism, the evolutionary trend record is modeled using an exponentially weighted moving average method to model the potential activation frequency; the determination of the persistent change characteristics is based on a joint judgment of the number of consecutive occurrences within the historical window and the maximum interval time.

4. The sparse tensor flow backtesting method for incremental data as described in claim 3, characterized in that... The structural offset intensity is calculated by the difference of tensor structure hash fingerprint or subspace overlap rate; the offset entropy threshold is an adaptive threshold dynamically adjusted according to the historical tensor evolution trajectory, and the sensitivity is adjusted by combining the global sparse density factor; the backtest snapshot is a cached copy of the previous backtest calculation results, which has a fast indexing mechanism based on time stamp and tensor structure version.

5. The sparse tensor flow backtesting method for incremental data as described in claim 4, characterized in that... The compressed backtesting includes local backtesting of the tensor block where structural shifts occur, and constructing a prediction error estimate using historical backtesting residuals; the detection of the abnormal event is jointly triggered by identifying sudden increases in the residual tensor and node jump changes in the structural variation spectrum.

6. The sparse tensor flow backtesting method for incremental data as described in claim 5, characterized in that... The structural mutation includes one or a combination of three cases: tensor dimension addition, dimension combination rearrangement, or activation density mutation; the anti-sparse backtesting process includes reactivating the diluted dimension channel, loading historical data trajectories, and performing periodic backtesting on the channel to estimate the impact range on the anomaly.

7. The sparse tensor flow backtesting method for incremental data as described in claim 6, characterized in that... When a structural mutation caused by the addition of the tensor dimension is detected, anti-sparse activation is performed on the dilute dimension combination related to the added dimension, and priority is assigned for backtesting; the dimension combination rearrangement detection includes the judgment of asymmetric activation regions caused by changes in tensor index order, and the original combination mapping relationship is reconstructed after detection to support the docking of historical trajectories.

8. The sparse tensor flow backtesting method for incremental data as described in claim 7, characterized in that... The determination of activation density mutation is based on the change in the second derivative of the number of activated elements within a unit time window. If the value exceeds the threshold, the anti-sparse process is triggered. When the diluted dimensional channel is reactivated, the evolution time vector of the channel is generated synchronously and aligned with the anomaly detection time to locate the target time period.

9. The sparse tensor flow backtesting method for incremental data as described in claim 8, characterized in that... If two consecutive abnormal indicators exceed the tolerance limit during the anti-sparse backtesting process, the channel is marked as a continuous abnormal channel, and a warning for freezing the trigger structure is issued. The estimation of the abnormal influence range adopts the convolution propagation algorithm to spread the abnormal residual of the channel to the tensor quantum structure to identify the secondary affected region.

Images