A spatiotemporal data dual-modal multi-task general analysis method and system

Abstract

The application discloses a kind of spatio-temporal data bimodal multi-task general analysis method and system.The method comprises obtaining spatio-temporal data of different modalities;Different modalities of spatio-temporal data are converted into data sequences of the same format, while generating road network representation vectors according to different modalities of spatio-temporal data;Upsampling is performed on data sequences and road network representation vectors to obtain spatio-temporal data feature sequences;Determine the text instruction and task placeholder, combine the spatio-temporal data feature sequence, and analyze the spatio-temporal data using the spatio-temporal data analysis model.The application effectively solves the incompatibility problem between different data modalities and can flexibly adapt to the needs of different tasks.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and more specifically to a general method and system for spatiotemporal data dual-modal multi-task analysis. Background Technology

[0002] Spatiotemporal data analysis has wide applications in intelligent transportation systems (ITS), smart cities, and location-based services (LBS). However, existing spatiotemporal data analysis methods are mainly task-specific, which limits the generalization ability of models across tasks.

[0003] Currently, multi-task general-purpose deep learning models have achieved significant success in fields such as natural language processing (NLP), computer vision (CV), and multimedia (MM), but they still face many challenges in the field of spatiotemporal data analysis. These include:

[0004] 1) The Challenge of Constructing a Unified Representation of Spatiotemporal Data: Two important types of data in spatiotemporal data analysis—individual trajectories and group traffic states—are often considered as two incompatible data modalities. Trajectory data is typically represented as a sequence of geographic units, such as road segments or points of interest (POIs); while traffic state data is represented as a graph structure with dynamic signals (such as traffic speed). Unifying the representation of these two different modalities is a challenge.

[0005] Existing methods generally support various downstream tasks by constructing general spatiotemporal data representations. However, most of these methods focus on individual static geographic elements, such as road networks and points of interest (POIs), or only focus on trajectory data.

[0006] In recent years, trajectory representation models such as TremBR and START have introduced periodicity by incorporating temporal information, while traffic state models such as T-wave, TrajNet, and TrGNN capture multi-hop spatial dependencies by propagating information along the trajectory. Although current representation models overlap in the trajectory and traffic state domains, developing a unified representation perspective for these two types of data remains an unresolved research area.

[0007] 2) Challenges of unifying heterogeneous spatiotemporal data analysis tasks: Spatiotemporal data analysis tasks are highly heterogeneous, including:

[0008] The data formats differ for different tasks; for example, the output of a travel time estimation task is continuous time, while the output of a path planning task is a different trajectory.

[0009] Heterogeneous tasks vary greatly in complexity; that is, the supervision signals require annotations of different granularities, which leads to a diversity of training formats.

[0010] Although general models have been built, these general models use specialized models as tools and call these tools during training to achieve cross-task learning. This paradigm builds a general spatiotemporal data analysis system rather than a single model, and its ability to reveal the correlation between different tasks is limited.

[0011] Therefore, developing a general single model that can simultaneously process trajectory and traffic status data and perform spatiotemporal data analysis across multiple heterogeneous tasks remains a major challenge. Summary of the Invention

[0012] In view of this, in order to at least partially solve the above-mentioned technical problems, the present invention provides a generalized method and system for spatiotemporal data bimodal multitasking, which aims to achieve cross-dataset generalization based on traffic spatiotemporal data and to process various spatiotemporal data-based tasks, thereby realizing the analysis and prediction of traffic conditions.

[0013] To achieve the above objectives, the present invention adopts the following technical solution:

[0014] In a first aspect, this application discloses a general method for spatiotemporal data dual-modal multi-task analysis, including:

[0015] Acquire spatiotemporal data of different modalities;

[0016] Spatiotemporal data of different modalities are converted into data sequences of the same format, and road network representation vectors are generated based on spatiotemporal data of different modalities.

[0017] Using the data sequence as an index, extract the corresponding spatiotemporal data feature sequence from the road network representation vector;

[0018] By identifying text instructions and task placeholders, and combining them with spatiotemporal data feature sequences, spatiotemporal data analysis models are used to analyze the spatiotemporal data.

[0019] Furthermore, the spatiotemporal data comes from the road network of the area to be analyzed, where each node contains static road trajectory information and dynamic traffic conditions.

[0020] Furthermore, the sequence format is: [geographic location, instantaneous time index, state interval index];

[0021] The geographical location is the specific road segment currently in which the user is located, the instantaneous time is the current time, and the status interval is the traffic status information within the preset time interval for statistics.

[0022] Furthermore, road network representation vectors are obtained based on spatiotemporal data of different modalities; the steps include:

[0023] Static and dynamic road network features were extracted from spatiotemporal data of different modalities using a characterizer and then stitched together.

[0024] The K and V values ​​are determined based on the concatenated features, and the attention-weighted features are output through a cross-attention mechanism, combined with the learnable Q value.

[0025] The attention-weighted features are further extracted using an MLP network to obtain the road network representation vector.

[0026] Furthermore, the characterizer consists of a first FFN network, a GAT network, and a second FFN network connected in sequence.

[0027] Furthermore, task placeholders include classification and regression.

[0028] Furthermore, the spatiotemporal data analysis model employs a hierarchical training strategy, and the training steps include:

[0029] The spatiotemporal data analysis model is pre-trained based on the spatiotemporal data feature sequence and task placeholders;

[0030] Text commands are incorporated to fine-tune the spatiotemporal data analysis model;

[0031] Introducing reinforcement learning to improve the performance of spatiotemporal data analysis models.

[0032] Furthermore, the spatiotemporal data analysis model is composed of multiple stacked blocks, which include parallel Value, Query and Key networks. The outputs of each network are connected to a multi-head attention network, and the outputs of the multi-head attention network are connected to a normalization layer and a feedforward neural network in sequence.

[0033] Secondly, this application discloses a general-purpose spatiotemporal data bimodal multi-task analysis system. This system applies the spatiotemporal data bimodal multi-task general-purpose analysis method described above. The system includes:

[0034] The data acquisition module is used to acquire spatiotemporal data of different modalities;

[0035] The format conversion module is used to convert spatiotemporal data of different modalities into data sequences of the same format;

[0036] The road network representation vector generation module is used to generate road network representation vectors based on spatiotemporal data of different modalities.

[0037] The spatiotemporal data feature sequence extraction module is used to extract road network representation vectors using data sequences as indexes.

[0038] The spatiotemporal data analysis module is used to analyze spatiotemporal data based on text instructions and task placeholders, combined with spatiotemporal data feature sequences, using a spatiotemporal data analysis model.

[0039] Furthermore, it also includes a spatiotemporal data analysis model training module, used to train the spatiotemporal data analysis model using a hierarchical training strategy. The training steps include:

[0040] The spatiotemporal data analysis model is pre-trained based on the spatiotemporal data feature sequence and task placeholders;

[0041] Text commands are incorporated to fine-tune the spatiotemporal data analysis model;

[0042] Introducing reinforcement learning to improve the performance of spatiotemporal data analysis models.

[0043] As can be seen from the above technical solutions, the present invention discloses a general method and system for spatiotemporal data bimodal multitasking analysis. It mainly achieves unified representation of spatiotemporal data of different modalities through spatiotemporal units (ST units) and spatiotemporal tokenizers, and introduces interactive prompts and hierarchical training strategies to address the challenges of task heterogeneity.

[0044] Compared with existing technologies,

[0045] 1. This invention successfully constructs a unified spatiotemporal data representation framework by converting individual trajectory and group traffic state data into data sequences of the same format and generating road network representation vectors. This framework effectively solves the incompatibility problem between different data modalities, enabling both types of data to be analyzed on a common basis.

[0046] 2. Unlike existing technologies that use general models as tools, the spatiotemporal data analysis model proposed in this invention can directly handle multiple heterogeneous tasks, such as classification and regression, using a single model without requiring a dedicated model for each task. This helps to reveal the correlations between different spatiotemporal data analysis tasks. The combination of task placeholders and text instructions allows the model to flexibly adapt to the needs of different tasks, greatly improving task processing efficiency.

[0047] 3. By adopting a hierarchical training strategy, including pre-training, fine-tuning, and reinforcement learning, the generalization ability of the model is significantly improved, enabling the model to not only perform well on specific datasets, but also maintain high performance across datasets. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 A schematic diagram of a general analysis method for spatiotemporal data in a dual-modal, multi-task manner;

[0050] Figure 2 This is a schematic diagram of the ST Tokenizer structure;

[0051] Figure 3 Example image of input data for trajectory generation task;

[0052] Figure 4 This is a schematic diagram of the three-stage training method;

[0053] Figure 5 This is a diagram showing the correspondence between data, models, and tasks used in spatiotemporal analysis based on the current benchmark model.

[0054] Figure 6 This is a diagram showing the relationship between the data, model, and tasks obtained using the solution of this invention. Detailed Implementation

[0055] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] To overcome the aforementioned problems in the prior art, this invention first discloses a general method for spatiotemporal data dual-modal multi-task analysis; referring to... Figure 1 , Figure 1 This is a flowchart illustrating a general method for bimodal, multi-task analysis of spatiotemporal data.

[0057] The analytical method steps include:

[0058] S1. Acquire spatiotemporal data of different modalities; in one embodiment, it is used to process common spatiotemporal applications based on road networks, including trajectory and traffic status tasks; that is, in the dataset of this application, all heterogeneous spatiotemporal data come from the road network of the area to be analyzed. The network is constructed as a graph structure, and each node contains static road trajectory information and dynamic traffic status.

[0059] S2. Convert spatiotemporal data of different modalities into data sequences of the same format, and generate road network representation vectors based on spatiotemporal data of different modalities.

[0060] In this application, both trajectory data and traffic state data are essentially sequential data sampled from a dynamic road network. From this perspective, the main difference between trajectory data and traffic state data lies in the sampling method.

[0061] Therefore, in one embodiment, in response to the challenge of unified data representation, this application proposes spatiotemporal units (STUnits) to convert spatiotemporal data of two different modalities, trajectory and traffic state sequence, into a unified format; that is, spatiotemporal data of both modalities can be represented as ST unit sequences.

[0062] ST cell sequence is a data format, specifically...

[0063] The format structure is: [geographic location, instantaneous time index, state interval index];

[0064] Geographic location refers to the specific road segment currently in which the location is situated, while instantaneous time refers to the current time accurate to the second. The status interval is used to measure traffic conditions, and is calculated in half-hour intervals to gather traffic status information within that time range.

[0065] In another embodiment, this application designs a spatiotemporal tokenizer (ST Tokenizer) to generate road network representation vectors based on spatiotemporal data of different modalities. The road network representation vectors are graph structures used to represent dynamic road networks.

[0066] The generation steps include:

[0067] Static and dynamic road network features were extracted from spatiotemporal data of different modalities using a characterizer and then stitched together.

[0068] The K and V values ​​are determined based on the concatenated features, and the attention-weighted features are output through a cross-attention mechanism, combined with the learnable Q value.

[0069] The attention-weighted features are further extracted using an MLP network to obtain the road network representation vector.

[0070] In one embodiment, the structure of the ST Tokenizer is referenced. Figure 2 It includes static characterizers, dynamic characterizers, and fused characterizers.

[0071] The static and dynamic characterizers share the same structure, comprising a first FFN network, a GAT network, and a second FFN network connected in sequence. This invention enables deep fusion of spatiotemporal data through these characterizers. This fusion not only enhances the comprehensiveness of data analysis but also improves the model's interpretability and prediction accuracy.

[0072] The first FFN network (fully connected feedforward network) is used to initially process static road network information and extract features;

[0073] GAT networks (Graph Attention Networks) are used to capture spatial dependencies in road networks and emphasize important road segment features.

[0074] The second FFN network (fully connected feedforward network) is used to further process and abstract the features output by the GAT network.

[0075] Furthermore, the features processed by the static and dynamic representative models are concatenated and merged into a richer feature representation, which helps the model better understand the overall situation of the road network.

[0076] Further, based on the concatenated features, K and V values ​​are determined, and combined with the learnable Q value, attention-weighted features are output through a cross-attention mechanism. K (Key) and V (Value) values ​​represent important parts of the road network features. By performing interactive attention calculation with the learnable Q value, the importance of different features can be determined.

[0077] The fusion representative includes cross-attention and MLP networks;

[0078] The role of cross attention is to calculate the importance weight of each feature by calculating the similarity between Q and K, and then sum the weighted values ​​of V to obtain the final attention-weighted features.

[0079] An MLP network is used to process the features output by the cross-attention mechanism, further extracting and abstracting information, and finally outputting a road network representation vector. This vector can be a comprehensive representation of the entire road network, or an intermediate representation used for a specific prediction task.

[0080] In this embodiment, ST Tokenizer combines static and dynamic information, utilizes graph neural networks and attention mechanisms, and can effectively capture the complex relationships of road networks, providing strong feature support for subsequent road network analysis or prediction tasks.

[0081] In one embodiment, the learnable Q is a trainable variable, i.e., the gradient of the loss with respect to Q is calculated by computing the loss through attention output and labels, and the parameters of Q are updated using an optimization algorithm. A learnable Q enables the model to extract useful information more effectively from input features.

[0082] S3. Using the data sequence as an index, obtain the corresponding data in the road network representation vector to obtain the spatiotemporal data feature sequence;

[0083] In this embodiment, it can be viewed as sampling spatiotemporal data of different modalities in different ways. Specifically, the data sequence formed by trajectory and traffic flow data provides a data index, and spatiotemporal feature information is extracted from the dynamic road network representation according to the index, thereby obtaining a unified representation of trajectory and traffic status (spatiotemporal data feature sequence).

[0084] S4. Determine the text instructions and task placeholders, and analyze the spatiotemporal data using a spatiotemporal data analysis model in conjunction with the spatiotemporal data feature sequence.

[0085] To address the challenge of task heterogeneity, this embodiment introduces interactive prompts, which uniformly label specific task data from different heterogeneous tasks, including input data and task-related specifications.

[0086] For the input data, based on the spatiotemporal data feature sequence, this application adds text instructions to guide the model to perform task types;

[0087] For output tasks, since multiple tasks may share the same spatiotemporal input, the model struggles to determine the specific task type based solely on spatiotemporal data. Therefore, to address this issue, this application introduces a task instruction mechanism as a task identifier, used to indicate the output type and quantity of each task. Based on this, data from multiple spatiotemporal tasks can be integrated into a single dataset for joint training.

[0088] In one embodiment, the spatiotemporal tasks are divided into four categories, and their output forms are summarized into two categories: classification of static road segment IDs and regression of dynamic features.

[0089] As a preferred approach, task placeholders are defined as follows: [CLS] represents classification, and [REG] represents regression.

[0090] In one exemplary embodiment, a personalized text instruction template is provided for each task to clearly define the task type. Therefore, the format of all input data can be standardized in this way. Specifically, the model input consists of three parts: text instructions, a spatiotemporal data feature sequence, and task placeholders.

[0091] To further illustrate the relationship between text instructions, spatiotemporal data, and task placeholders, this application provides a specific example of input data, using trajectory generation as an example. See details in the attached document. Figure 3 .

[0092] In one embodiment, heterogeneous tasks differ in complexity and training paradigm. For example, generation tasks output sequences and are supervised using sequence labels, while classification and regression tasks rely on single-label supervision. To address these challenges, this application designs a three-stage training process to meet the needs of tasks with varying complexity, including trajectory reconstruction during model pre-training; task-oriented cue fine-tuning to adapt the model to the task; and generative reinforcement learning to enhance performance on trajectory generation tasks. (See also...) Figure 4 The specific training steps include:

[0093] 1) Pre-train the spatiotemporal data analysis model based on the spatiotemporal data feature sequence and task placeholders;

[0094] At this stage, training is performed using only spatiotemporal data (ST data) and task placeholders. Specifically, the model input consists of three parts: text instructions, spatiotemporal data, and task placeholders. For the model output, only the task placeholders are taken as the final result. The number of task placeholders corresponds one-to-one with the number of results the model needs to output. Regression tasks are obtained using regression placeholders ([REG]), and classification tasks are obtained using classification placeholders ([CLS]).

[0095] 2) Incorporate text commands to fine-tune the spatiotemporal data analysis model; with the help of text commands, the model can be jointly fine-tuned across multiple tasks. After this stage, the model can handle classification and regression tasks.

[0096] Since the input data for all tasks is standardized to three parts: text instructions, spatiotemporal data, and task placeholders, all tasks can be trained using a unified model architecture. Each task type has its own text instruction template, serving as a specific task identifier.

[0097] 3) Introduce reinforcement learning to improve the performance of spatiotemporal data analysis models.

[0098] The final stage introduces reinforcement learning, specifically designed to improve the model's performance on sequence labeling tasks (such as generation tasks).

[0099] This embodiment uses the PPO algorithm in reinforcement learning; it includes:

[0100] First, the distance between the generated trajectory and the ground truth trajectory is calculated using DTW (Deep Dive Theory Warping). This distance calculation method assigns a distance value to each token in the trajectory. Then, these distance values ​​are accumulated to generate the reward value in reinforcement learning, which in turn yields the loss value.

[0101] In this application, all tasks can be trained using sequence modeling. Given GPT-2's powerful capabilities in sequence modeling, it can be used as the foundational architecture for constructing spatiotemporal data analysis models.

[0102] In one embodiment, the spatiotemporal data analysis model is a Bimodal Interactive General Model (BIGST) for spatiotemporal data analysis, the BIGST model architecture of which is as follows: Figure 1 As shown.

[0103] BIGST comprises multiple stacked blocks to learn complex sequence-to-sequence mappings. Each block includes parallel Value, Query, and Key networks, with their outputs connected to a multi-head attention network. The output of the multi-head attention network is then connected to a normalization layer and a feedforward neural network. This architecture is highly efficient, capable of capturing long-range spatiotemporal dependencies, and improves the model's computational efficiency and prediction accuracy.

[0104] in,

[0105] Value network: used to extract "value" information from input data;

[0106] Query network: Used to extract "query" information from input data;

[0107] Key network: Used to extract "key" information.

[0108] These three networks are typically fully connected layers that process the input data in parallel, producing corresponding values, queries, and key vectors.

[0109] In a multi-head attention network, the outputs from the Q, K, and V networks are split into multiple "heads," each of which computes a portion of the attention output. The final output of the multi-head attention network is the concatenation of the outputs from all heads, capturing the relationships between different parts of the input sequence.

[0110] Following the multi-head attention network, the output data passes through a normalization layer to normalize each feature of each sample, thereby stabilizing the training process and accelerating convergence.

[0111] Then, the features that have undergone multi-head attention and layer normalization are further processed by a feedforward neural network (FFN network) to increase the nonlinearity of the model.

[0112] In this embodiment, there are residual connections (skip connections) between the multi-head attention network and the FFN network, as well as after the FFN network. These connections add the input data directly to the output of each submodule, which helps prevent the gradient vanishing problem and allows the model to train deeper networks.

[0113] In one implementable embodiment, this application discloses a spatiotemporal data bimodal multi-task general analysis system. This system applies the spatiotemporal data bimodal multi-task general analysis method described above. The system includes:

[0114] The data acquisition module is used to acquire spatiotemporal data of different modalities;

[0115] The format conversion module is used to convert spatiotemporal data of different modalities into data sequences of the same format;

[0116] The road network representation vector generation module is used to generate road network representation vectors based on spatiotemporal data of different modalities.

[0117] The spatiotemporal data feature sequence extraction module is used to extract road network representation vectors using data sequences as indexes.

[0118] The spatiotemporal data analysis module is used to analyze spatiotemporal data based on text instructions and task placeholders, combined with spatiotemporal data feature sequences, using a spatiotemporal data analysis model.

[0119] Furthermore, it also includes a spatiotemporal data analysis model training module, used to train the spatiotemporal data analysis model using a hierarchical training strategy. The training steps include:

[0120] The spatiotemporal data analysis model is pre-trained based on the spatiotemporal data feature sequence and task placeholders;

[0121] Text commands are incorporated to fine-tune the spatiotemporal data analysis model;

[0122] Introducing reinforcement learning to improve the performance of spatiotemporal data analysis models.

[0123] Since the detailed execution steps of this system are consistent with the general analysis method for spatiotemporal data bimodal multitasking described above, they will not be repeated here.

[0124] This application presents a general bimodal, multi-task analysis method for spatiotemporal data, capable of integrating multiple data types and simultaneously processing multiple tasks. To clarify the achievable effects of this application, it is further compared with three different benchmark models. Details of the datasets, models, and tasks for each benchmark model are provided below. Figure 5 ;

[0125] Depend on Figure 5 It is evident that the three baselines in existing methods differ in model architecture and training paradigm. Furthermore, this application's method or system attempts to simultaneously process these different tasks, and the relationships between the resulting data, model, and tasks are as follows: Figure 6 As shown;

[0126] Depend on Figure 6As can be seen, the advantage of this application lies in its powerful multi-tasking capability. BIGST can process three different baselines using only one model, and can achieve state-of-the-art performance in all of them.

[0127] This invention represents a significant technological breakthrough in the field of spatiotemporal data analysis, providing a powerful analytical tool for intelligent transportation systems, urban planning, and other related fields. It has broad application prospects and significant technological advantages.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0129] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A spatio-temporal data dual-modal multi-task general analysis method, characterized in that, acquiring spatio-temporal data of different modalities; the spatio-temporal data comes from a road network in a region to be analyzed, each node in the road network contains static road trajectory information and dynamic traffic state; converting the spatio-temporal data of different modalities into data sequences of the same format, the sequence format being: [geographical location, instantaneous time index, state interval index]; the geographical location is a specific road segment currently located, the instantaneous time is the current time, and the state interval is traffic state information within a preset time interval for statistics; generating a road network representation vector according to the spatio-temporal data of different modalities; the step includes: extracting static road network features and dynamic road network features in the spatio-temporal data of different modalities through a representation device, and splicing them; the representation device includes a first FFN network, a GAT network and a second FFN network connected in turn; determining K and V values according to the spliced features, combining a learnable Q value, and outputting attention weighted features through a cross-attention mechanism; further extracting features of the attention weighted features through an MLP network to obtain a road network representation vector; extracting corresponding spatio-temporal data feature sequences from the road network representation vector with the data sequences as indexes; determining a text instruction and a task placeholder, combining the spatio-temporal data feature sequences, and analyzing the spatio-temporal data by using a spatio-temporal data analysis model; the text instruction is a text instruction guiding the model to execute a task type, and the task placeholder includes classification and regression; the spatio-temporal data analysis model is stacked by a plurality of Blocks, the Block includes a parallel Value network, a Query network and a Key network, outputs of the networks are commonly connected to a multi-head attention network, and an output of the multi-head attention network is connected to a standardization layer and a feedforward neural network in turn.

2. The spatio-temporal data dual-modality multi-task general analysis method according to claim 1, characterized in that, The spatio-temporal data analysis model adopts a hierarchical training strategy, and the training steps include: pre-training the spatio-temporal data analysis model according to the spatio-temporal data feature sequences and the task placeholder; fine-tuning the spatio-temporal data analysis model by incorporating the text instruction; introducing reinforcement learning to improve the performance of the spatio-temporal data analysis model.

3. A spatio-temporal data dual-modality multi-task general analysis system, characterized in that, The spatio-temporal data dual-modal multi-task general analysis method of any one of claims 1-2, the system comprising: a data acquisition module for acquiring spatio-temporal data of different modalities; a format conversion module for converting the spatio-temporal data of different modalities into data sequences of the same format; a road network representation vector generation module for generating a road network representation vector according to the spatio-temporal data of different modalities; a spatio-temporal data feature sequence extraction module for extracting the road network representation vector with the data sequences as indexes; a spatio-temporal data analysis module for analyzing the spatio-temporal data by using a spatio-temporal data analysis model according to a text instruction and a task placeholder, and combining spatio-temporal data feature sequences.

4. The spatio-temporal data dual modality multi-task general analysis system of claim 3, wherein, It also includes a spatio-temporal data analysis model training module for training the spatio-temporal data analysis model by using a hierarchical training strategy, and the training steps include: pre-training the spatio-temporal data analysis model according to the spatio-temporal data feature sequences and the task placeholder; fine-tuning the spatio-temporal data analysis model by incorporating the text instruction; Introduce reinforcement learning to improve the performance of spatio-temporal data analysis models.

Images