City calculation basic model and equipment based on Mama time sequence

Abstract

The invention provides a city calculation basic model and equipment based on a Mama time sequence, and relates to the technical field of data processing. In the model provided by the embodiment of the invention, a Mama framework is combined with multi-task time sequence analysis. Aiming at the non-stationarity of urban time series data, a frequency prompt network is designed for decoupling in combination with a Koopman operator theory and dynamic mode decomposition, a steady mode and non-stationary fluctuation are analyzed from the angles of time and frequency, and a frequency mode memory pool (FPMP) is introduced for realizing cross-scene feature reuse. The advantage of a Mama model in capturing long-term time dependence is utilized, and an attention mechanism similar to Mama is designed to simulate channel dependence in a multivariable time sequence. A learnable supplementary sequence and a multi-task prompt mark are introduced to support dynamic multi-task adaptation, and the contradiction between urban data isomerism and task diversity is relieved. Shared knowledge and specific task knowledge are balanced through shared experts and specific task experts of the multi-task attention module, the negative migration problem in multi-task learning is relieved, and the generalization ability of the model is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a city computing basic model based on Mamba time series and equipment. BACKGROUND

[0002] With the acceleration of urbanization, city computing has become an emerging interdisciplinary field aiming to solve complex urban problems by advanced computing technology and data analysis methods, thereby improving the efficiency of urban operation and the quality of life of residents. Various infrastructures in the city system continuously generate a large amount of dynamic time series data, providing important insights for urban planning, management and optimization. Time series analysis, as a core technology of city computing, not only reveals the dynamic behavior of the city system, but also provides prediction and early warning support for decision-making.

[0003] However, traditional time series models are difficult to capture the inherent complex patterns and long-term dependencies in city data. Although deep learning models (such as Transformer, CNN) improve the time modeling ability through end-to-end learning, they still face major bottlenecks in the city computing scenario: First, the scale and complexity of city time series data grow exponentially, increasing the quadratic computational complexity of traditional attention mechanisms, which is insufficient to meet real-time processing requirements. Second, the demand for multi-task collaborative optimization conflicts with the insufficient generalization ability of the model, as traditional parameter sharing mechanisms often lead to negative transfer problems. In addition, the frequency domain features of city data (such as traffic flow periodicity, energy consumption seasonality) contain rich convertible features, but existing methods lack systematic decoupling of time-varying and non-time-varying components, limiting the efficiency of cross-scene knowledge reuse.

[0004] In recent years, the rise of time series basic models provides a new perspective to address these challenges. These models achieve efficient transfer learning through pre-training strategies, enabling rapid adaptation to different tasks and datasets, particularly suitable for the heterogeneity of multi-domain data in city computing. Existing models such as TimesNet, GPT4TS and TEMPO have made progress in capturing complex temporal patterns and cross-domain learning, but they still require task-specific modules and fine-tuning, which limits their scalability and generalization ability. At the same time, new state space models such as Mamba have shown advantages in long sequence modeling due to their linear complexity and dynamic state adjustment mechanism, but they still have deficiencies in modeling multivariate channel dependencies and dynamic multi-task adaptation. SUMMARY

[0005] The present application provides a city computing basic model based on Mamba time series and equipment, aiming to meet the diverse time series analysis needs in city computing, at least partially solving the above problems.

[0006] The first aspect of the present application provides a Mamba-based time series urban computing base model, comprising: a sequence reconstruction module, a frequency prompt network, a Mamba network, and a multi-task attention module; The sequence reconstruction module is configured to supplement an input time series with a learnable supplementary sequence, generate an indication label and a response label for a corresponding task of the input time series by multi-task prompt labeling, and obtain a reconstruction feature. The frequency prompt network is configured to decouple a steady-state mode component and a transient fluctuation component of the reconstruction feature from the time and frequency angles by combining a Koopman operator theory and dynamic mode decomposition, obtain a time-varying feature and a non-time-varying feature, and fuse the time-varying feature and the non-time-varying feature via a frequency domain mode memory pool to obtain an enhanced feature. The Mamba network is configured to perform cross-domain multi-task time series analysis, and a time Mamba block models a time dependence of the enhanced feature by a selective state space mechanism, and a Mamba-like channel attention models a channel dependence between variables of the enhanced feature. The multi-task attention module comprises a shared expert and a task-specific expert, the shared expert is configured to learn a cross-task general feature, and the task-specific expert is configured to capture a unique mode of a specific task, and the multi-task attention module is configured to output a prediction result adapted to a task corresponding to the input time series.

[0007] Optionally, the indication label is configured to indicate a time series analysis task, and the time series analysis task comprises a prediction task, a classification task, an interpolation task, and an anomaly detection task, and the response label is dynamically extended according to the analysis task and adapted to a task output format. In a case where the analysis task is a prediction task, the response indication is configured to extend a label corresponding to a future time step to output a complete prediction sequence. In a case where the analysis task is a classification task, the response indication is configured to embed category information. In a case where the analysis task is an interpolation task, the response indication is configured to locate a missing value position to be supplemented. In a case where the analysis task is an anomaly detection task, the response indication is configured to define an anomaly threshold based on a reconstruction error distribution.

[0008] Optionally, the sequence reconstruction module comprises a feature extraction module, a sequence supplement module, and a position encoding module. The feature extraction module is configured to divide an input time series into a plurality of feature blocks , to obtain a plurality of feature block embeddings , wherein is a number of time steps.N is the number of variables, M is the number of feature blocks, D 0 is the initial size of each feature block; the sequence complement module is configured to generate a new feature embedding based on a learnable complement sequence and the feature block embedding V wherein L denotes the number of complement sequences, is the length of each complement sequence: the linear projection layer converts the concatenated feature embedding into reconstructed features :

[0009] wherein concat(.) denotes a concatenation operation; D denotes the embedding dimension; linear(.) denotes a linear projection; the position encoding module is configured to apply position encoding to the original input time series S to obtain corresponding feature blocks: .

[0010] Optionally, the frequency prompt network comprises a time-varying feature extraction unit, a non-time-varying feature extraction unit, and a frequency domain mode memory pool. The time-varying feature extraction unit is configured to fit a transient fluctuation component through a Koopman operator for a Fourier-transformed feature to obtain a time-varying feature . The non-time-varying feature extraction unit is configured to map a steady-state mode component through a global Koopman operator for a Fourier-transformed feature to obtain a non-time-varying feature . The frequency domain mode memory pool is configured to capture dynamic mode features of the time-varying feature and the non-time-varying feature through two learnable parameter matrices, store dynamic modes in a key-value pair structure, generate frequency domain enhanced features through a similarity retrieval, and splice the enhanced features FP i .

[0011] Optionally, the Mamba network comprises a time Mamba block and a Mamba-like channel attention unit. The time Mamba block is configured to introduce nonlinearity to the enhanced features through an activation function to obtain activated features ; a selective state space mechanism parameter Δ,​​B and C is dynamically generated as follows: ; where, denotes the Softplus function, ensuring that ∆ > 0; N denotes the hidden state dimension of the SSM. The continuous-time parameter is discretized into a discrete-time parameter using a discretization rule: ; where, and are the discretized state transition matrix and input matrix, respectively, I is the identity matrix, and ∆ is the time step for input dynamic adjustment; The recursive relationship is calculated using the parallel scan algorithm, and the output is calculated according to the obtained hidden state: ; After mapping the selective state space mechanism output back to the original dimension, the input and output y out are combined through a residual connection: ; The Mamba-like channel attention unit is used to calculate channel dependency through proxy attention mechanism: ; where, denotes the Softmax function, and the proxy token A acts as an intermediary, calculates the similarity between the query Q and the proxy token, calculates the similarity between the proxy token and the key K , B 1 and B 2 represent proxy bias, used for proxy aggregation and proxy broadcast steps, respectively, DWC (.) denotes a deep convolution.

[0012] Optionally, the multi-task attention module further comprises a gating network; The gating network is used to calculate the weighted sum of shared experts and task-specific experts: ; The weight vector is calculated through a linear transformation and a Softmax activation function: ; is a selection matrix containing shared experts and current task-specific experts: ; wherein, is a parameter matrix, m k and m s are the number of specific experts and shared experts respectively, and are the outputs of specific experts and shared experts respectively, d is the dimension of input representation.

[0013] Optionally, the multi-task attention module further comprises a tower network; the tower network comprises a generation task tower and a classification task tower; the generation task tower outputs a time series by wherein, represents a complete time series prediction task sample obtained in advance, and the MLP consists of two linear layers, Linear 1 represents an adaptive linear layer, Project is a projection layer for mapping embedding information back to a time series patch; the classification task tower outputs a category label by matching a response prompt with a category embedding.

[0014] The second aspect of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the Mamba-based urban computing basic model of time series when executed.

[0015] The third aspect of the present application provides a computer readable storage medium, which stores a computer program, wherein the program is executed by a processor to implement the Mamba-based urban computing basic model of time series as described in the time aspect of the present application.

[0016] The fourth aspect of the present application provides a computer program product, comprising computer programs / instructions, wherein the computer programs / instructions are executed by a processor to implement the steps in the Mamba-based urban computing basic model of time series as described in the time aspect of the present application.

[0017] In the embodiment of the present application, the Mamba architecture is combined with multi-task time series analysis to solve the diversity and dynamics of urban time series data, and to adapt to various urban computing scenarios. In view of the non-stationary nature of urban time series data, the present application combines the Koopman operator theory and dynamic mode decomposition to design a frequency prompt network for decoupling, to analyze the steady-state mode and non-stationary fluctuations from the perspective of time and frequency, and to introduce a frequency pattern memory pool (FPMP) for realizing cross-scene feature reuse.

[0018] The embodiment of the application also utilizes the advantage of the Mamba model in capturing long-term time dependence, and designs a Mamba-like attention mechanism to simulate channel dependence in multivariate time series.

[0019] In the embodiment of the application, the learnable supplementary sequence and the multi-task prompt label are introduced to support dynamic multi-task adaptation, and the contradiction between city data heterogeneity and task diversity is relieved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the description of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0021] Figure 1 is a structural block diagram of the city computing basic model of time series based on Mamba provided by the present application; Figure 2 is a detailed structural diagram of the city computing basic model of time series based on Mamba provided by the present application; Figure 3 is a principle diagram of the prompt mechanism for sequence data of the city computing basic model of time series based on Mamba provided by the present application; Figure 4 is a data processing flowchart of the frequency prompt network of the city computing basic model of time series based on Mamba provided by the present application; Figure 5 is a data processing flowchart of the frequency mode storage pool of the city computing basic model of time series based on Mamba provided by the present application; Figure 6 is an architecture diagram of the Mamba network of the city computing basic model of time series based on Mamba provided by the present application; Figure 7 is an architecture diagram of the Mamba-like channel attention of the city computing basic model of time series based on Mamba provided by the present application; Figure 8 is a data processing flowchart of the Mamba-like channel attention of the city computing basic model of time series based on Mamba provided by the present application; Figure 9A data processing flow schematic diagram of a multi-task attention module of a Mamba-based time series urban computing basic model provided by the application; Figure 10 A data processing flow schematic diagram of a task tower network of the Mamba-based time series urban computing basic model provided by the application; Figure 11 A hardware structure diagram of any device with data processing capability where the Mamba-based time series urban computing basic model provided by the application is located. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0023] The embodiment of the application proposes a Mamba-based time series urban computing basic model, aiming to meet the diversified time series analysis needs in urban computing. By introducing a learnable complementary sequence, a multi-task prompt label and a frequency domain feature enhancement mechanism, combining an efficient state space model Mamba and an expert hybrid (MoE) task head, efficient modeling of complex time series data and multi-task adaptability are realized. The model performs well in time series prediction, classification, interpolation and anomaly detection tasks, and shows its potential for wide application in urban computing.

[0024] Specifically, the Mamba-based time series urban computing basic model first combines the Mamba architecture with multi-task time series analysis to solve the diversity and dynamics of urban time series data, and adapts to various urban computing scenarios.

[0025] In view of the non-stationarity of urban time series data, the embodiment of the application combines the Koopman operator theory and dynamic mode decomposition to design a frequency prompt network for decoupling, analyzes the steady-state mode and non-stationary fluctuations from the time and frequency perspectives, and introduces a frequency pattern memory pool (FPMP) for realizing cross-scene feature reuse.

[0026] In the embodiment of the application, the advantages of the Mamba model in capturing long-term temporal dependencies are also utilized, and a Mamba-like attention mechanism is designed to simulate the channel dependency in multivariate time series.

[0027] In the embodiment of the application, the learnable complementary sequence and the multi-task prompt label are introduced to support dynamic multi-task adaptation, and the contradiction between urban data heterogeneity and task diversity is alleviated. Further, the shared experts and task-specific experts of the multi-task attention module are used to balance shared knowledge and task-specific knowledge, alleviate the negative transfer problem in multi-task learning, and enhance the generalization ability of the model.

[0028] Universal temporal data analysis refers to the ability of a model to effectively handle different temporal task scenarios. To achieve this goal, it is necessary to overcome the distribution differences between datasets and integrate heterogeneous time series data into a unified model. Given that time series data from different sources vary in length and number of variables, the model F It must be designed to be transparent to all input samples, which means that it should not be affected by the specific characteristics of the input.

[0029] In order to facilitate multitasking and rapid adaptation to new tasks, this paper proposes a general model paradigm applicable to multitasking scenarios. F(X,θ) →Y. In addition, sharing weights across tasks enables the model to handle multiple tasks simultaneously and eliminates the need to train separate models for each specific task. An ideal general-purpose temporal data analysis model should have two key capabilities: First, it should be able to leverage rich and diverse data from different task scenarios for training, sharing weights across tasks to avoid the need to train task-specific models from scratch. Second, it should exhibit strong cross-task generalization capabilities, especially when training data is limited, ensuring robust adaptation to tasks while maintaining high performance.

[0030] Therefore, the present invention proposes a basic model of urban computing based on Mamba time series, which is used to construct a general model for multi-task time series analysis, such as Figure 1 As shown, it shows a structural block diagram of the urban computing basic model based on Mamba time series provided by an embodiment of the present invention, and the model includes: a sequence reconstruction module 101, a frequency prompt network 102, a Mamba network 103 and a multi-task attention module 104.

[0031] The sequence reconstruction module 101 is used to supplement the input time sequence through a learnable supplementary sequence, generate an indicator tag and a response tag of a corresponding task for the input time sequence through a multi-task prompt tag, and obtain a reconstructed feature; The frequency prompt network 102 is used to combine Koopman operator theory and dynamic mode decomposition to decouple the steady-state mode component and transient fluctuation component of the reconstructed feature from the perspectives of time and frequency to obtain time-varying features and non-time-varying features, and to fuse the time-varying features and non-time-varying features through the frequency domain mode memory pool to obtain enhanced features; The Mamba network 103 is used for cross-domain multi-task time series analysis. The temporal Mamba block models the temporal dependency of the enhanced features through a selective state space mechanism, and the Mamba-like channel attention models the channel dependency between the variables of the enhanced features. The multi-task attention module 104 includes shared experts for learning cross-task general features and task-specific experts for capturing unique patterns of specific tasks, and is configured to output a prediction result adapted to a task corresponding to the input time sequence.

[0032] In the embodiment of the present application, a learnable complementary sequence and a prompt token are introduced in the sequence reconstruction module to enhance the description of cross-domain multi-task time sequences. The processed input sequence includes instruct tokens, sequence tokens, complementary tokens and response tokens, each of which plays a unique role in time sequence analysis.

[0033] The present application designs a novel frequency prompt network to decouple the steady-state mode component and the transient fluctuation component of the feature from the time and frequency angles. Inspired by the advanced state space model Mamba, the present application proposes a backbone network Mamba network for cross-domain multi-task time sequence analysis. The Mamba network utilizes the powerful function of Mamba in capturing long-term dependencies to model time dependencies. The present application also designs a special Mamba-like channel attention, which aims to effectively capture the channel dependency between variables. Finally, the embodiment of the present application introduces a customized gated hybrid expert network task head, which adopts a shared structure design to effectively integrate multi-task information and realize accurate output of multi-task results.

[0034] Specifically, as shown in Figure 2 The detailed structure diagram of the city computing basic model based on the Mamba time sequence provided by the embodiment of the present application is shown; in the embodiment of the present application, the sequence reconstruction module includes a feature extraction module, a sequence complementary module and a position encoding module.

[0035] Recent studies show that the expressiveness of the sequence representation is a key factor affecting the performance of time series analysis models. There is almost a linear relationship between the entropy of the sequence representation and the mean squared error, indicating that more diverse sequence representations can lead to better model performance. In theory, training models on large datasets is the most effective way to learn rich feature representations. However, in real cross-domain multi-task scenarios, this approach is limited. Each task usually contains only a limited number of samples, which is not enough to meet the requirements of training models on large datasets. To solve this problem, the embodiment of the invention introduces a sequence supplement module, which enhances data diversity by merging learnable auxiliary sequences into the original input sequence, thereby more effectively revealing the underlying patterns and dependencies in multi-task sequences. In addition, the invention introduces a dispersion loss function to maximize the orthogonality between the supplementary sequences, significantly improving sequence diversity.

[0036] Specifically, given an input time series wherein, is the number of time steps, N is the number of variables, the input sequence is first divided into multiple feature blocks embeddings wherein, M is the number of feature blocks, D 0 is the initial size of each feature block, then a learnable supplementary sequence is defined, wherein V represents the number of supplementary sequences, L is the length of each supplementary sequence. The supplementary sequence X is concatenated with the feature embeddings to form a new feature embedding : (1) Then, the concatenated feature embedding is converted to an embedded feature : (2) where D is the embedding dimension. To preserve the positional information of the original sequence, position encoding is only applied to the part derived from the original input sequence S : (3) where concat(.) represents the concatenation operation; D represents the embedding dimension; linear(.) represents linear projection; POS (.) represents position encoding.

[0037] In addition to enhancing data diversity through complementary sequences, the sequence reconstruction module provided by the embodiments of the present application also integrates two core hint mechanisms for sequence data: instruction hints and response hints, as shown in Figure 3 For different task datasets, the model is equipped with unique hint tokens that serve as customized "identifiers" to help the model adjust and optimize data during training according to the specific requirements of each task. The present application has conducted in-depth research on the application scenarios of the Mamba algorithm in four typical time series analysis tasks, including forecasting, classification, imputation, and anomaly detection, each task having a corresponding instruction hint. In the forecasting task, the goal of the model is to generate predictions for multiple future time steps. Inspired by UNITS, the present application adjusts the response hints by expanding tokens to adapt to forecasting tasks of different time spans, where each expansion token corresponds to a specific prediction target, enabling the model to output a complete prediction sequence at once. As shown in Figure 3 Each time series in the analysis task includes instruction hints, response hints, sequence tokens, and complementary tokens. In the classification task, the response hints are used to embed task-related class information without generating prediction results. The structure of the imputation task is similar to that of the forecasting task, but the response hints are used to locate missing values that need to be filled in to guide the model to perform imputation operations. For the anomaly detection task, the model defines an anomaly threshold based on the reconstruction error distribution and distinguishes whether a data point is normal or abnormal based on the reconstructed sequence.

[0038] Specifically, in the embodiments of the present application, the instruction tokens are used to indicate time series analysis tasks, including forecasting tasks, classification tasks, imputation tasks, and anomaly detection tasks, and the response tokens are dynamically expanded according to the analysis tasks to adapt to the task output format.

[0039] In the case where the analysis task is a forecasting task, the response tokens are used to expand tokens corresponding to future time steps to output a complete prediction sequence.

[0040] In the case where the analysis task is a classification task, the response tokens are used to embed class information. In the case where the analysis task is an imputation task, the response tokens are used to locate the positions of missing values that need to be filled in.

[0041] ​In the case that the analysis task is an anomaly detection task, the response prompt is used to define an anomaly threshold based on a reconstruction error distribution.

[0042] In the embodiments of the present application, the analysis task can specifically include at least one of power consumption prediction, traffic flow analysis, meteorological data filling, and sensor network anomaly detection.

[0043] In practical applications, time series inherently have non-stationarity, however, on a local time scale, these sequences usually exhibit weak stationary characteristics. This situation is very similar to the case handled by time-varying dynamics. Inspired by the Koopman theory in the analysis of complex dynamic systems, the present application designs a new type of frequency prompt network, as shown in Figure 4 The frequency prompt network effectively decomposes the time-varying component and the non-time-varying dynamic component in the complex non-stationary sequence from the perspective of frequency domain statistical analysis, extracts the time-varying feature and the non-time-varying feature respectively, and calculates the fitting matrix through the extended dynamic mode decomposition (eDMD) algorithm. The goal of the present application is to infinitely approximate the Koopman operator. In addition, the present application introduces a frequency pattern memory pool to store and extract various dynamic patterns embedded in the time-varying and non-time-varying components, thereby realizing deep mining and enhancement of the sequence features.

[0044] In the embodiments of the present application, the frequency prompt network comprises a time-variant feature extraction unit, a time-invariant feature extraction unit, and a frequency pattern memory pool.

[0045] The time-variant feature extraction unit is configured to fit a transient fluctuation component through a Koopman operator for the Fourier-transformed feature, to obtain a time-variant feature .

[0046] The time-invariant feature extraction unit is configured to map a steady-state mode component through a global Koopman operator for the Fourier-transformed feature, to obtain a time-invariant feature .

[0047] The frequency pattern memory pool is configured to capture dynamic mode features of the time-variant feature and the time-invariant feature through two learnable parameter matrices, to store the dynamic modes in a key-value pair structure, to generate frequency domain enhanced features through similarity retrieval, and to splice the enhanced features FP i .

[0048] In embodiments of the present invention, time-varying features refer to those that change over time, typically reflecting dynamic changes within a time series. In time series analysis, time-varying features may include seasonal fluctuations, trend changes, or other patterns that oscillate over time. In the analysis of complex dynamic systems, the observed state of the system at a given moment is generally defined as a system snapshot. Considering the continuous development of time-varying dynamic characteristics, constructing a temporal neighborhood based on local snapshots within a time window facilitates linearization.

[0049] Therefore, assuming the input after Fourier transform (FFT) is , then divide it into part ( Figure 4 The example is X var1 、X var2 、X var3 ): (4) in W represents the look-back window length, N Represents the total number of variables, and each segment is considered as a state snapshot of the system.

[0050] Then, the encoder Each snapshot Mapping to Koopman space embedding Specifically, given a set of Koopman embedding snapshots , using extended dynamic mode decomposition (eDMD) to calculate the system fitting matrix , to asymptotically approximate the Koopman operator: (5) in yes The Moore-Penrose pseudoinverse.

[0051] In obtaining back, The Koopman embedding iterative fitting is as follows: (6) in L is the length of the fitting result. Finally, the decoder Used to reconstruct the fitted embedding back to the original space, we get as follows: (7) Non-time-varying features refer to features that remain constant over time, often reflecting global patterns or static properties in time series. In time series analysis, non-time-varying features manifest as stability across the time dimension, such as long-term mean or constant periodic patterns that do not evolve over time. These features can be effectively characterized by globally learned Koopman operators. Non-time-varying features have special significance in the state transition of the system, as they establish a direct state mapping relationship between the lookback window and the fitting window. Specifically, the system embedding obtained by Fourier transform Figure 4 X inv ) is mapped to a low-dimensional Koopman lookback window embedding space by an encoder , where D denotes the dimension of the embedding space. This process helps to learn a universal embedding for pairs of sliding windows. The dynamic evolution of the system is described by a learnable operator , which establishes a state transition relationship from the lookback window to the embedding of the fitting window: (8) where and are pairs of running windows. Finally, a decoder maps the fitting embedding back to the original observation space , resulting in .

[0052] To effectively store and extract dynamic patterns of time-varying and non-time-varying components in time features, thereby enhancing the ability to mine and represent sequence features, the present invention draws inspiration from the design of memory networks and proposes a frequency pattern memory pool (FPMP), the structure of which is shown in Figure 5 .

[0053] FPMP captures time-varying features and non-time-varying features through two learnable parameter matrices (Learnable Keys Embed, Learnable Values Embed) and organizes them using a key-value pair structure. Specifically, FPMP can be formally defined as: (9) where are learnable parameters. Based on existing research, the dynamic pattern memory of frequency components is organized in a key-value structure, where denotes the key in the pattern memory pool, denotes the value in the pattern memory bank.

[0054] During feature extraction, the system uses a frequency component embedding as a query. By computing the similarity to the keys in the schema memory pool, it retrieves the values most relevant to the current frequency component. These retrieved values, together with the task-specific prompt tokens, form the augmented features that are fed into the downstream network, thus improving the model's ability to represent and predict time-series data. The implementation of this mechanism is as follows: (10) wherein, and are the retrieved frequency component dynamic schema prompts.

[0055] In cross-domain multi-task time series analysis, time and channel (intra-variable and inter-variable) dependencies have a significant impact on model performance. Time dependency manifests as dynamic patterns of individual variables over time, such as trends, periodicity, and lag effects, while channel dependency reflects complex interactions and potential causal relationships between multiple variables, such as nonlinear effects of covariates on target variables. To fully capture these complex dependencies, the model needs to model time dependency and channel dependency separately. Based on this, the present invention proposes a cross-domain multi-task time series analysis backbone network based on an improved state space model Mamba (hereinafter referred to as the Mamba network). The Mamba network utilizes the strong ability of Mamba to capture long-term dependencies for time dependency modeling, while introducing a novel Mamba-like channel attention mechanism to effectively capture inter-variable channel dependency. This design not only significantly improves model accuracy, but also reduces computational complexity, improving model flexibility and generalization ability.

[0056] In the field of time series analysis, accurately capturing time dependencies is crucial for model prediction accuracy. Although traditional Transformer models can effectively handle sequence data, their quadratic computational complexity in the attention mechanism leads to a dramatic increase in computational overhead when processing long sequences, limiting their scalability in real-world scenarios. In contrast, Mamba, as an efficient state space model, shows unique advantages in modeling long-term time dependencies. By introducing a parameterized mechanism for input sequences, Mamba enables the model to dynamically adjust the propagation and forgetting process of internal states based on current input, thus maintaining effective information transmission and updating when processing long sequences. This mechanism not only effectively addresses the performance bottleneck of traditional state space models when processing long sequences, but also significantly improves the model's ability to model time dependencies. Notably, Mamba's linear computational complexity gives it a significant computational efficiency advantage when processing large-scale time series data.

[0057] The application adopts a Mamba architecture based on a selective state space mechanism (Selective SSM, S6). The mechanism dynamically adjusts the attention of the model to the input information, effectively filters and utilizes key information in long sequences, thereby accurately capturing time-dependent relationships and improving the modeling efficiency of time series patterns. The architecture of the Mamba network is as shown in Figure 6 .

[0058] The selective SSM is the core of the Mamba architecture. By introducing a selection mechanism, the model can dynamically adjust the state update method according to the input. The core idea of the selection mechanism is to make the parameters of the SSM dependent on the input, thereby realizing selective propagation and forgetting of information. The input projection introduces nonlinearity through an activation function. Assuming that the activated features are , the SSM parameters ∆, B, and C are dynamically generated as follows: (11) where is a Softplus function that ensures ∆>0. N is the dimension of the hidden state of the SSM. Next, the continuous-time parameters are discretized into discrete-time parameters using a discretization rule: (12) where and are the discretized state transition matrix and input matrix, respectively. I is the identity matrix, and ∆ is the time step of input dynamic adjustment. Since ∆ is now a function of the input , the discretized parameters and also depend on , thereby giving the model its time-varying characteristics. Finally, the recursive relationship is calculated using a parallel scan algorithm, and the output is calculated based on the obtained hidden state: (13) After mapping the SSM output back to the original dimension, the input and output are combined through the remaining connections: (14) Channel dependency, a key feature of intervariate interactions in multivariate time series, has a decisive impact on the model's predictive performance. While Mamba offers significant advantages in modeling long-term temporal dependencies, it still suffers from significant shortcomings in modeling channel dependency in multivariate time series. Mamba's limitations are primarily reflected in two aspects: First, its selection mechanism employs a unidirectional processing model, similar to the sequence processing of recurrent neural networks (RNNs). This limits the model's ability to simultaneously capture global intervariate relationships, unlike the Transformer architecture, which can comprehensively model channel dependencies. Second, Mamba lacks a global attention mechanism, which directly impacts its ability to model global dependencies between variables. These structural limitations make it difficult for Mamba to fully capture nonlinear interactions between variables when handling complex multivariate time series tasks.

[0059] Based on this, embodiments of the present invention propose an improved attention mechanism (Mamba-like channel attention) for time series analysis. This mechanism effectively incorporates the structural advantages of Mamba in state-space modeling, significantly reducing the computational complexity of traditional attention mechanisms. At the same time, it effectively preserves the attention mechanism's ability to model global channel dependencies, achieving a balance between computational efficiency and modeling performance.

[0060] Figure 7 The following diagram shows the architecture of Mamba-like channel attention, which is defined as follows: (15) in δ represents a linear transformation, and Represents element-wise multiplication (Hadamard product).

[0061] The input is subjected to feature extraction through the convolution layer (Conv) and activated by the activation function (SiLU) δ Introducing nonlinearity to generate feature maps F 1. For the original input x Apply a linear transformation δ ,get F 3. Linear transformation of the original input δ ( x ) and convolutional features F 1 Perform element-by-element multiplication (Hadamard product) to generate fusion features F 2. This step enhances the interaction signal between channels by combining the original features with the convolutional features. In this embodiment of the present invention, an activation function is also applied to the output of the agent attention. δ , and mapped back to the original feature dimension through linear transformation to ensure compatibility with subsequent modules.

[0062] In terms of attention mechanism, the embodiment of the present invention introduces proxy attention, which is a special linear attention mechanism designed to balance computational efficiency and representation ability. It reduces the computational complexity of the traditional Softmax attention mechanism by introducing a set of proxy tags while maintaining the global context modeling capability. Figure 8 As shown in Figure 3, this design ensures that proxy attention outperforms traditional linear attention in terms of efficiency and expressiveness, while also inheriting the high expressiveness of Softmax attention.

[0063] In the embodiment of the present invention, a learnable proxy tag is introduced A Acts as an intermediary between queries and keys to reduce computational complexity.

[0064] The proxy tag first acts as a "proxy" for the query, aggregating information from the key (K) and value (V), and then broadcasting that information back to the query. The process starts with proxy aggregation, where the proxy tag A is used as the query, and attention calculations are performed using the key K and value V to generate proxy features V A : (16) in represents the Softmax function, Compute the similarity between the surrogate tag and the key.

[0065] Next, a proxy broadcast is performed, where the proxy tag A is used as the key and the proxy feature V A As the value, the attention calculation is performed using the query Q to generate the final output O. This step is also a standard Softmax attention operation, expressed as: (17) in Calculate the similarity between the query and the proxy token. Through these two steps, the proxy token A acts as an intermediary between the query and the key, avoiding direct interaction between the query and the key, thereby reducing the computational complexity. The output O can be expressed as: (18) This formula demonstrates that Agent Attention can be viewed as a combination of two Softmax Attention operations, effectively serving as a generalized linear attention mechanism. The forget gate plays a crucial role in the effectiveness of the Mamba architecture, with its core function being to provide local bias and position information. Notably, the agent tag itself has the ability to encode position information and can therefore serve as a replacement for the forget gate. Furthermore, by introducing agent bias and deep convolutional computation, the utilization of position information is further improved while maintaining feature diversity. The final agent attention module can be expressed as: (19) where B1 and B2 are proxy biases for proxy aggregation and proxy broadcasting steps, respectively. The proxy biases are constructed by three bias components (column bias, row bias, and block bias) to reduce the number of parameters. In the embodiments of the present application, the proxy biases B1, B2 and deep convolution (DWC) are introduced to enhance the utilization of position information: Before passing through the task head, the extracted feature vector is processed using an adaptive feedforward network that dynamically adjusts the weight matrix according to the changes in input and output feature dimensions, ensuring compatibility with any feature size. Bilinear interpolation is used to update the weight matrix in real time, while a weight distribution mechanism is introduced to balance the weight distribution between fixed and dynamic components through a parameter vector.

[0066] To effectively improve the performance of multi-task learning models and solve the negative transfer problem caused by traditional parameter sharing mechanisms, a multi-task head network based on a hybrid expert system is proposed. The network explicitly separates shared expert modules and task-specific expert modules and introduces a gating network to achieve dynamic knowledge fusion, thereby optimizing the efficiency of multi-task learning. Specifically, the multi-task attention module consists of two core components: the shared expert module is responsible for learning the general feature representation of the task, while the task-specific expert module focuses on capturing the unique pattern features of each task. Each task integrates the knowledge representation of the two types of experts through an independent tower network, where the gating network dynamically adjusts the weight distribution between the shared expert and the task-specific expert according to the input features. This mechanism supports adaptive knowledge sharing across tasks and independent learning of task characteristics, ensuring knowledge transfer between tasks while effectively alleviating task conflicts and sample dependency issues. The structure of the multi-task attention module is shown in Figure 9 .

[0067] In the embodiments of the present application, each task k has a gating network for calculating the weighted sum of shared experts (ExpertsShared) and task-specific experts (Experts A, Experts B). The feature vector extracted by the Mamba network is input to the gating network and denoted as x， The output of the gating network is a weight vector representing the importance of each expert. The output of the gating network is the weighted sum of the shared expert and the task-specific expert, calculated as follows: (20) where the input x is calculated through a linear transformation matrix and a Softmax activation function to obtain the weight vector : (21) is the selection matrix containing all selected vectors (shared experts and task k-specific experts): (22) where, is the parameter matrix, m k and m s and and are the number of specific experts and shared experts, respectively, d are the outputs of specific experts and shared experts, respectively,

[0068] Prediction by task-specific tower networks (Tower A, Tower B) is calculated as: (23) For the tower networks, two different networks are used for the generation and classification tasks. The generation task tower network (Tower A) is mainly used for prediction, imputation, and anomaly detection, as shown in Figure 10

[0069] The fused features can be converted or reconstructed into the corresponding time series, converting the representation learned inside the model into specific time series outputs (such as prediction values, imputation values, or reconstructed sequences) as follows: (24) where represents the complete time series sample representation learned by the model (i.e., the fused features described above), the MLP consists of two linear layers to extract nonlinear features, Linear 1 represents a self-adaption linear layer (Self-adaption Linear) that dynamically adjusts the feature dimension to adapt to different task requirements, Project is a projection layer that maps the embedded information back to a time series segment. For the imputation task and anomaly detection, the corresponding sample sequence labels are modified. The output forms include: prediction task: sequence segment at future time steps; imputation task: filling value segment at missing positions; anomaly detection: reconstructed complete sequence (used to calculate reconstruction error).

[0070] ​The classification tower network (Tower B) can convert the response label of the classification task into the corresponding time series classification result. The response label (Response Prompt) is used as a query, and cross attention is performed on all output labels. Then the processed response prompt is similarity measured with class embeddings (ClassEmbeddings) to determine the current predicted output class. The specific processing flow is as follows: (25) where the global feature representation extracted by the model The response label of the classification task as key and value As a query for cross attention calculation. The class embedding representation generated by training. Finally, the processed is matched with the class embedding to find the nearest class embedding vector, thereby obtaining the classification result.

[0071] In the embodiment of the application, the classification tower network dynamically fuses the global information of the time series by cross attention. Instead of directly outputting classification probabilities, the class is determined by embedding space distance.

[0072] The embodiment of the city computing basic model for time series based on Mamba provided by the application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The apparatus embodiment can be realized by software, or by hardware or a combination of software and hardware. Taking software realization as an example, as a logical device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability. From the hardware level, as shown in Figure 11 , it is a hardware structure diagram of the device with data processing capability where the city computing basic model for time series based on Mamba provided by the application is located. In addition to the processor, memory, network interface, and non-volatile memory shown in Figure 11 , the device with data processing capability where the apparatus is located in the embodiment usually includes other hardware according to the actual function of the device with data processing capability, which will not be described here.

[0073] Based on the same inventive concept, the application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the city computing basic model for time series based on Mamba when executed.

[0074] Based on the same inventive concept, the present application also provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the steps in the Mamba-based time series urban computing infrastructure model according to any one of the above embodiments.

[0075] Based on the same inventive concept, the present application provides a computer program product, comprising computer program / instructions, which, when executed by a processor, implements the steps in the Mamba-based time series urban computing infrastructure model according to any one of the above embodiments.

[0076] Each of the embodiments in the present specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be mutually referred to.

[0077] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present application is described with reference to flowcharts and / or block diagrams of the method, terminal device (apparatus), and computer program product according to the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, special-purpose computer, embedded processor, or other programmable terminal device to produce a machine, so that the instructions executed by the computer or other programmable terminal device produce the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0079] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0080] These computer program instructions can also be loaded into a computer or other programmable terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 These computer program instructions can also be loaded into a computer or other programmable terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 These computer program instructions can also be loaded into a computer or other programmable terminal device to cause a series of operational steps to be performed on the computer or other programmable terminal device to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal device provide steps for implementing the functions specified in the flowchart block or blocks.

[0081] Although preferred embodiments of the application have been described, a person of ordinary skill in the art can make additional changes and modifications to these embodiments once armed with the present disclosure. Therefore, the appended claims are intended to cover all changes and modifications of the preferred embodiments which fall within the scope of the application.

[0082] Finally, it should also be noted that in the present application, the relationship terms such as time and variable are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or terminal device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article or terminal device including the element.

[0083] The above describes in detail the Mamba-based time series urban computing foundation model provided by the present application. The principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed. In summary, the content of the present description should not be understood as a limitation of the present application.

Claims

1. A basic model for urban computing based on Mamba's time series, characterized by: The model includes: a sequence reconstruction module, a frequency cue network, a Mamba network and a multi-task attention module; The sequence reconstruction module is used to supplement the input time sequence through a learnable supplementary sequence, generate an indication mark and a response mark of the corresponding task for the input time sequence through a multi-task prompt mark, and obtain a reconstructed feature; The frequency cue network is used to combine Koopman operator theory and dynamic mode decomposition to decouple the steady-state mode component and transient fluctuation component of the reconstructed feature from the perspectives of time and frequency to obtain time-varying features and non-time-varying features, and to fuse the time-varying features and non-time-varying features through the frequency domain mode memory pool to obtain enhanced features; The Mamba network is used for cross-domain multi-task time series analysis. The temporal Mamba block models the temporal dependency of the enhanced features through a selective state space mechanism, and the Mamba-like channel attention models the channel dependency between the variables of the enhanced features to obtain fused features. The multi-task attention module includes shared experts and task-specific experts. The shared experts are used to learn common features across tasks, and the task feature experts are used to capture unique patterns of specific tasks. The multi-task attention module is used to output a prediction result that is adapted to the task corresponding to the input time series based on the fused features.

2. The city computing basic model based on Mamba time series according to claim 1 is characterized in that: The indicator tag is used to indicate a time series analysis task, which includes: a prediction task, a classification task, an interpolation task, and an anomaly detection task. The response tag is dynamically expanded according to the analysis task and adapted to the task output format; In the case where the analysis task is a prediction task, the response indication is used to extend the marker corresponding to the future time step to output a complete prediction sequence; In the case where the analysis task is a classification task, the response indication is used to embed category information; In the case where the analysis task is a filling task, the response indication is used to locate the position of the missing value to be filled; In the case where the analysis task is an anomaly detection task, the response prompt is used to define an anomaly threshold based on a reconstruction error distribution.

3. The city computing basic model based on Mamba time series according to claim 1 is characterized in that: The sequence reconstruction module includes: a feature extraction module, a sequence supplement module and a position encoding module; The feature extraction module is used to extract the time series , divided into multiple feature blocks embedded ,in, is the number of time steps, N is the number of variables, M is the number of feature blocks, D 0 is the initial size of each feature block; The sequence supplement module is used to supplement the sequence based on the learnable and the feature block embedding Generate new feature embeddings ,in V represents the number of complementary sequences, L is the length of each complementary sequence: ； The linear projection layer embeds the concatenated features into Convert to reconstruction features : Where concat (.) represents the cascade operation; D represents the embedding dimension; linear (.) represents linear projection; The position encoding module is used to transform the original input time series S The corresponding feature blocks apply position encoding: 。 4. The city computing basic model based on Mamba time series according to claim 1 is characterized in that: The frequency prompt network includes: a time-varying feature extraction unit, a time-invariant feature extraction unit and a frequency domain pattern memory pool; The time-varying feature extraction unit is used to fit the transient fluctuation component through the Koopman operator for the features after Fourier transformation to obtain the time-varying features. ; The time-invariant feature extraction unit is used to map the steady-state mode components through the global Koopman operator to obtain the time-invariant features after Fourier transformation. ; The frequency domain pattern memory pool is used to capture time-varying features through two learnable parameter matrices and time-invariant features Dynamic pattern features are stored in a key-value pair structure, and frequency domain enhancement features are generated by familiarity retrieval and spliced ​​to obtain enhanced features. FP i .

5. The city computing basic model based on Mamba time series according to claim 1 is characterized in that: The Mamba network includes: a temporal Mamba block and a Mamba-like channel attention unit; The temporal Mamba block is used to introduce nonlinearity into the enhanced features through the activation function to obtain the activated features. ;Selective state space mechanism parameters ∆, B and C Dynamically generated as follows: ； in, Indicates the Softplus function, ensuring ∆>0; N Denotes the hidden state dimension of the SSM. The continuous-time parameters are discretized into discrete-time parameters using the discretization rule: ； in, and are the discretized state transfer matrix and input matrix respectively, I is the identity matrix, ∆ is the time step of input dynamic adjustment; Use the parallel scanning algorithm to calculate the recurrence relation and calculate the output based on the obtained hidden state: ； After mapping the selective state space mechanism output back to the original dimension, the input is merged through the residual connection and output y out : ； The Mamba-like channel attention unit is used to calculate channel dependencies through the proxy attention mechanism: ； in, represents the Softmax function, and the proxy marker A acts as an intermediary. Calculation query Q and the similarity between proxy tags, Computing surrogate tags and keys K The similarities between B 1 and B 2 represents the proxy bias, which is used in the proxy aggregation and proxy broadcast steps respectively, and DWC(.) represents depthwise convolution.

6. The city computing basic model based on Mamba time series according to claim 1 is characterized in that: The multi-task attention module also includes a gating network; The gating network is used to compute the weighted sum of the shared expert and the task-specific expert: ； Calculate the weight vector through linear transformation and Softmax activation function : ； is a selection matrix containing shared experts and task-specific experts: ； in, is the parameter matrix, m k and m s are the number of specific experts and shared experts, and are the outputs of specific experts and shared experts respectively, d is the dimensionality of the input representation.

7. The city computing basic model based on Mamba time series according to claim 6 is characterized in that: The multi-task attention module also includes a tower network; the tower network includes a generation task tower and a classification task tower; The generation task tower is achieved by Output time series, where Represents the complete time series prediction task sample obtained previously. The MLP consists of two linear layers. Linear 1 represents an adaptive linear layer, Project It is the projection layer that maps the embedded information back to the time series patch; The classification task outputs a category label by matching the response prompt with the category embedding.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the city calculation basic model based on Mamba's time series according to any one of claims 1 to 7 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the city calculation basic model based on the Mamba time series described in any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the city computing basic model based on the Mamba time series described in any one of claims 1 to 7 is implemented.

Images