Urban area dynamic representation learning method based on context representation learning

Abstract

The invention provides an urban area dynamic representation learning method based on context representation learning, and belongs to the field of urban function and spatio-temporal data mining, and the method comprises the steps: S1, constructing a continuous dynamic graph according to public transport data; s2, an urban area dynamic representation extractor under multiple time dimensions is constructed, the extractor is used for extracting structure information and semantic information in the continuous dynamic graph, nodes in the continuous dynamic graph are converted into representation vectors of fixed dimensions, and the representation vectors are dynamically updated along with time; s3, pre-training the extractor by using a plurality of prediction tasks, including designing a corresponding loss function to guide the extractor to learn more universal urban region characterization; and S4, utilizing the trained plurality of extractors to obtain the dynamic representation of the urban area, and combining the dynamic representation of the urban area with context representation learning to be applied to a plurality of downstream tasks. According to the method, comprehensive space-time semantic information can be captured, and the dynamic region representation of the city is dynamically captured from multiple time granularities.

Description

Technical Field

[0001] The present invention belongs to the field of urban function and spatio-temporal data mining, and particularly relates to a method for dynamically representing urban regions based on context representation learning. Background Art

[0002] Large-scale group movement data plays an extremely important role in contemporary urban function research. By recording the movement trajectories and behavior patterns of people in urban spaces, this type of data provides a solid analysis foundation for several downstream urban tasks such as urban region traffic flow prediction, regional housing price prediction, regional crime frequency prediction, etc. Since a city consists of various different regions, including commercial areas, residential areas, etc., understanding urban functions requires effectively learning high-quality regional representations.

[0003] In recent years, many researchers have modeled and learned urban region representations based on large-scale group movement data, aiming to solve various downstream urban tasks by obtaining a general urban region representation. Therefore, the requirements for the accuracy and generalization of urban region representation modeling are increasing. Currently, existing methods mainly rely on using movement events between regions as a measure of the strength of the relationship between regions. However, this process ignores the actual time of movement, making it impossible to model urban region representations at different time scales, and thus leading to a serious loss of group movement information. In real downstream task scenarios, due to the dynamics of movement activities, the relevant downstream tasks of urban functions are also constantly changing, and traditional methods cannot meet the needs of fine-grained and dynamic tasks. In addition, existing methods need to further fine-tune the learned urban representations for each downstream task, consuming more time and resources. Therefore, how to dynamically capture and learn the regional representations of cities and directly apply these representations to various downstream tasks without fine-tuning has become an urgent problem to be solved. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a method for dynamically representing urban regions based on context representation learning, including the following steps:

[0005] Step S1: Construct a continuous dynamic graph according to public transportation data, where the continuous dynamic graph is a set of movement data sorted by time ;

[0006] Step S2: Construct a dynamic representation vector for each urban region , construct a short-term information feature extractor, a medium-term information fusion extractor, and a long-term information update extractor, and respectively extract a basic dynamic feature vector, a medium-time granularity urban region dynamic representation vector, and a current urban dynamic representation vector based on and ​ ;

[0007] Step S3: Based on and construct a historical region representation sequence and a current representation, and train the short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor based on the current representation in combination with multiple pre-training tasks;

[0008] Step S4: Use the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor to obtain the dynamic representation of the urban region, and apply the dynamic representation of the urban region combined with context representation learning to multiple downstream tasks.

[0009] Beneficial effects:

[0010] 1. The present invention proposes a dynamic learning method for urban region representation. Through spatio-temporal information extractors with multi-level time granularities, spatio-temporal information at different time granularities is extracted from the original large-scale population movement data, realizing multi-dimensional and multi-level modeling of the city. For the problem that previous methods cannot be directly applied to dynamic urban downstream tasks, such as predicting the urban region flow for multiple days, this paper proposes a dynamic urban region representation coding framework. By maintaining the dynamic urban region memory and continuously updating it with each fine-grained travel data, dynamic modeling of the region representation is achieved, providing more accurate support for market prediction, traffic flow analysis, etc.

[0011] 2. The present invention proposes to predict urban downstream tasks based on context representation learning, which can automatically adjust the model according to the input task information and historical data, greatly reducing the dependence on a large amount of labeled data and demonstrating strong multi-task processing capabilities. Since it does not require separate fine-tuning for each new task, it can quickly respond to various task requirements in a changing environment. The present invention can easily handle multiple related tasks such as price prediction and user demand prediction without the need for specialized training for each task. In the scenario of resource scheduling, the present invention can optimize resource allocation in real time according to dynamically changing demands, while the personalized recommendation system can quickly provide accurate recommendation services for users through context information. Through this efficient task switching and information extraction mechanism, the present invention can exhibit significant performance advantages in practical applications, greatly reducing the time and computational costs of enterprises in dealing with complex tasks. Description of the Drawings

[0012] Figure 1 is a schematic flow chart of a method for dynamic representation learning of urban regions based on context representation learning of the present invention;

[0013] Figure 2Schematic diagram of downstream task application based on context representation learning in the embodiments of the present invention. Detailed implementation manners

[0014] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0015] Embodiment 1

[0016] As Figure 1 shown, a method for dynamic representation learning of urban areas based on context representation learning provided by an embodiment of the present invention includes the following steps:

[0017] Step S1: Construct a continuous dynamic graph according to public transportation data, where the continuous dynamic graph is a set of moving data sorted by time ;

[0018] Step S2: Construct a dynamic representation vector for each urban area , construct a short-term information feature extractor, a medium-term information fusion extractor, and a long-term information update extractor, and extract a basic dynamic feature vector, a medium-time granularity urban area dynamic representation vector, and a current urban dynamic representation vector based on and respectively ;

[0019] Step S3: Based on and construct a historical area representation sequence and a current representation, and train the short-term information feature extractor, the medium-term information fusion extractor, and the long-term information update extractor based on the current representation combined with multiple pre-training tasks;

[0020] Step S4: Use the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor to obtain the dynamic representation of the urban area, and apply the dynamic representation of the urban area combined with context representation learning to multiple downstream tasks.

[0021] In one embodiment, the above step S1: Construct a continuous dynamic graph according to public transportation data, where the continuous dynamic graph is a set of moving data sorted by time , specifically includes:

[0022] Step S11: Eliminate public transportation data with missing information according to simple rules, and construct moving data based on this where Denote the i-th individual movement data record which respectively represent the starting longitude, starting latitude, ending longitude, ending latitude, departure time, and arrival time;

[0023] Step S12: Sort all movement data in ascending order according to the departure time, and divide the processed movement data into several travel data blocks according to the fine-grained time , and its calculation formula is as follows:

[0024] (1)

[0025] wherein, is the current time, is the starting time of the current travel data block , is the set of all movement data whose travel time belongs to and ;

[0026] Step S13: Divide the city map into several regions according to the current urban planning basis, and map the starting longitude and latitude, ending longitude and latitude in to their corresponding region numbers through a mapping function, and divide the region number where it is located according to the starting longitude and latitude in the data, to obtain the processed large-scale group movement data , wherein, represents the i-th processed movement data, which respectively represent the starting region, arrival region, and departure time.

[0027] In one embodiment, the above Step S2: Construct a dynamic representation vector for each urban region , construct a short-term information feature extractor, a medium-term information fusion extractor, and a long-term information update extractor, and respectively extract a basic dynamic feature vector, a medium-time granularity urban region dynamic representation vector, and a current urban dynamic representation vector based on and , specifically including:

[0028] Step S21: Use an urban region encoder to maintain a dynamic representation vector for each urban region

[0029] to store the dynamic changes of region i; Step S22: Construct a short-term information feature extractor to encode the fine-grained movement data into a basic dynamic feature vector

[0030] First, divide the travel data time by hour and calculate the basic dynamic feature vectors for the coding area within the starting time period of to the fine-grained as follows: The formula is as follows:

[0031] (2)

[0032] (3)

[0033] (4)

[0034] Among them, and are trainable parameters; and both represent intermediate variables; is a function that concatenates and together; and respectively represent the dynamic characterization vectors of area and area ; and respectively represent all travel events with the departure area of and the arrival area of within ;

[0035] Then, encode the time information of the travel data according to the time decay coefficient, and preliminarily aggregate this information according to the departure area and arrival area of the travel to obtain the basic dynamic feature vector ;

[0036] Step S23: Construct a medium-time-information fusion extractor to perform data enhancement on the basic dynamic feature vector and obtain the enhanced medium-time granularity urban area dynamic characterization vector :

[0037] For the basic dynamic feature vector perform cross-time attention aggregation, and the calculation formula is as follows:

[0038] (5)

[0039] (6)

[0040] Among them, and are both trainable parameters, The normalization function is used to reduce the internal offset of the input data. is the dynamic representation vector of the urban area at the medium time granularity obtained after cross-time aggregated data augmentation;

[0041] Step S24: Construct a long-term information update extractor to perform selective learning and preservation to generate the current urban dynamic representation vector :

[0042] According to and the urban historical dynamic representation vector, and the learnable parameters, the urban dynamic representation is dynamically updated. The formula is as follows:

[0043] (7)

[0044] where are the trainable parameters, is the urban historical dynamic representation vector, is the element-wise product operation;

[0045] Introduce learnable parameters to control the flow and update of the urban area representation information, ensuring that the model can capture the urban area features for a long time.

[0046] In one embodiment, the above step S3: Based on and construct the historical area representation sequence and the current representation, and train the short-term information feature extractor, the medium-term information fusion extractor and the long-term information update extractor based on the current representation combined with multiple pre-training tasks, specifically including:

[0047] Step S31: Input the medium-time granularity urban area dynamic representation vector into the long-term information update extractor, and combine the obtained current urban dynamic representation vector with the urban area dynamic representation vector to obtain the historical area representation sequence;

[0048] Step S32: Take the first C representations from the historical area representation sequence in chronological order, generate the current representation through the area representation attention fusion mechanism, and add this representation to the historical area representation sequence;

[0049] Step S33: Design pre-training tasks to adjust the parameters of the extractor, including: area input, output flow prediction task and inter-area flow data prediction task, as shown in the following formulas:

[0050] (8)

[0051] (9)

[0052] (10)

[0053] Among them, and and respectively represent the input flow data, output flow data of region i at time t, and OD transfer data from region i to region j; and respectively represent selecting from time to start time interval;

[0054] Step S34: Pre-train the short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor through the current representation, construct the loss function L and backpropagate to update the model parameters:

[0055] (11)

[0056] (12)

[0057] Among them, m represents the number of pre-training tasks, n represents the number of urban regions, respectively represent the true value and predicted value of pre-training task k in region i, represents the parameter for adjusting the weight of the pre-training task; K represents the total number of tasks;

[0058] Finally, the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor are obtained.

[0059] In one embodiment, the above step S4: Use the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor to obtain the dynamic representation of the urban region, and apply the dynamic representation of the urban region combined with context representation learning to multiple downstream tasks, specifically including:

[0060] Step S41: Initialize the historical region representation sequence for storing the historical region representation of the city, and store the representation of urban region i at time t obtained by the urban region encoder into ;

[0061] Step S42: Extract the historical region representations of the previous C days from the historical representation sequence , and take out the task data of the corresponding previous C days according to the current task as the predicted support node representation:

[0062] (13)

[0063] Among them, k represents that the sample data belongs to the k-th downstream task. represents the dynamic representation vector of region i at time t. represents the historical representation vector of the downstream task k at time t.

[0064] Step S43: According to the attention mechanism, first construct a query vector based on the current time representation of the urban region:

[0065] (14)

[0066] Among them, is the weight matrix and is a learnable parameter.

[0067] Step S44: Construct a support vector library based on the historical representation vector of the urban region and the historical data of a specific downstream task , and calculate the association weight between the current time urban region representation and the historical representation and data according to the query vector :

[0068] (15)

[0069] (16)

[0070] Among them, represents the key-value vector of region i at time, are two different weight matrices and are learnable parameters.

[0071] Step S45: Through perform weighted aggregation and linear transformation on the data of the k-th downstream task in the previous c days to obtain the target task data for prediction:

[0072] (17)

[0073] Among them, represents the predicted value of the k-th downstream task for region i that needs to be predicted, represents the learnable parameter, and MLP is the linear prediction layer for tuning parameters during the pre-training process, and the parameters do not change during the downstream task inference.

[0074] Use the above three trained extractors to perform downstream tasks for multiple urban regions in combination with context representation learning, such as crime rate prediction, housing price prediction, and regional traffic prediction.

[0075] Figure 2 Shows a schematic diagram of the downstream task application based on context representation learning.

[0076] The specific embodiments described above further elaborate on the objective, technical solution and beneficial effects of the present invention. It should be understood that the above description is only for the specific embodiments of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A dynamic representation learning method for urban areas based on context representation learning, characterized in that Including: Step S1: Construct a continuous dynamic graph based on public transportation data, where the continuous dynamic graph is a set of a series of mobile data sorted by time ; Step S2: Construct a dynamic representation vector for each urban area , construct a short-term information feature extractor, a medium-term information fusion extractor, and a long-term information update extractor, and based on and extract the basic dynamic feature vector, the medium-time granularity urban area dynamic representation vector, and the current urban dynamic representation vector respectively ; Step S3: Based on and construct a historical region representation sequence and a current representation, where the current representation is the dynamic representation of the urban area at the current moment, and train the short-term information feature extractor, the medium-term information fusion extractor, and the long-term information update extractor based on the current representation in combination with multiple pre-training tasks; Step S4: Obtain the dynamic representation of the urban area by using the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor, and apply the dynamic representation of the urban area combined with context representation learning to multiple downstream tasks.

2. The method for dynamic representation learning of urban areas based on context representation learning according to claim 1, characterized in that The step S1: constructing a continuous dynamic graph according to public transportation data, where the continuous dynamic graph is a set of a series of mobile data sorted by time , specifically including: Step S11: Eliminate the public transportation data with missing information according to simple rules, and construct mobile data based on this , where represents the i-th individual mobile data record, respectively represent the starting longitude, starting latitude, ending longitude, ending latitude, departure time, and arrival time; Step S12: Sort all the movement data in ascending order according to the departure time, and divide the processed movement data into several travel data blocks according to the fine-grained time , and its calculation formula is as follows: (1) Wherein, is the current time, is the start time of the current travel data block and is the set of all movement data whose travel time belongs to and ; Step S13: Divide the city map into several regions according to the current urban planning basis, and map the starting longitude and latitude and the ending longitude and latitude in to the region numbers where they are located through a mapping function, and divide the region numbers where they are located according to the starting longitude and latitude in the data, so as to obtain the processed large-scale group movement data , where represents the i-th movement data after processing, represent the starting region, the arrival region and the departure time respectively.

3. The method for dynamically representing and learning urban areas based on context representation learning according to claim 2, wherein The step S2: constructing a dynamic representation vector for each urban area , constructing a short-term information feature extractor, a medium-term information fusion extractor, and a long-term information update extractor, and based on and , respectively extracting a basic dynamic feature vector, a medium-time granularity urban area dynamic representation vector, and a current urban dynamic representation vector , specifically including: Step S21: Use the urban area encoder for each urban area Maintain a dynamic representation vector for the urban area , which is used to store the dynamic changes in area i; Step S22: Construct a short-term information feature extractor to encode fine-grained mobile data into a basic dynamic feature vector , and the specific steps are as follows: First, divide the travel data time by hour and calculate the basic dynamic feature vectors for the fine-grained within the starting time period to within the , and the formula is as follows: ​ (2) (3) (4) Among them, and are trainable parameters; and both represent intermediate variables; is a function that and concatenates together; and respectively represent the dynamic characterization vectors of region and region ; and respectively represent all travel events with the departure region being and the arrival region being and ; Then, encode the time information of the travel data according to the time decay coefficient, and preliminarily aggregate this information based on the departure area and arrival area of the travel to obtain the basic dynamic feature vector for this time period ; Step S23: Construct a medium-time-information fusion extractor to perform data augmentation on the basic dynamic feature vectors, and obtain an enhanced medium-time granularity urban area dynamic representation vector : For the basic dynamic feature vector perform cross-time attention aggregation, and the calculation formula is as follows: (5) (6) Among them, and are both trainable parameters, is a normalization function, is the dynamic representation vector of the urban area at the medium time granularity; Step S24: Construct a long-term information update extractor for performing selective learning and saving to generate a dynamic representation vector of the current city : According to and the dynamic representation vector of the city's history, as well as learnable parameters, the dynamic representation of the city is dynamically updated. The formula is as follows: (7) Among them, is a trainable parameter, is the dynamic representation vector of the city's history, is the element-wise product operation.

4. The method for dynamically representing and learning urban areas based on context representation learning according to claim 3, characterized in that The step S3: Based on and construct a historical region representation sequence and a current representation, and train the short-term information feature extractor, the medium-term information fusion extractor, and the long-term information update extractor based on the current representation in combination with multiple pre-training tasks, specifically including: Step S31: Input the medium time granularity urban area dynamic representation vector into the long-term information update extractor, and combine the obtained current urban dynamic representation vector with the urban area dynamic representation vector to obtain a historical area representation sequence; Step S32: Take the first C representations from the historical area representation sequence in chronological order, generate the current representation through the area representation attention fusion mechanism, and add this representation to the historical area representation sequence; Step S33: Design a pre-training task to adjust the parameters of the extractor, including: area input, output flow prediction task, and inter-area flow data prediction task, as shown in the following formula: (8) (9) (10) Among them, , , respectively represent the input flow data, output flow data of region i at time t, and OD transfer data from region i to region j, and respectively represent the selection starting from time interval; Step S34: Pre-train the short-term information feature extractor, the medium-term information fusion extractor, and the long-term information update extractor through the current representation, construct a loss function L and backpropagate to update the model parameters: (11) (12) Among them, m represents the number of pre-training tasks, and n represents the number of urban areas. respectively represent the true value and the predicted value of the model pre-training task k in region i. represents the parameter used to adjust the weight of the pre-training task; K represents the total number of tasks. Finally, obtain the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor.

5. The method for dynamic representation learning of urban areas based on context representation learning according to claim 4, wherein The said Step S4: Obtain the dynamic representation of the urban area by using the trained short-term information feature extractor, medium-term information fusion extractor, and long-term information update extractor, and apply the dynamic representation of the urban area combined with context representation learning to multiple downstream tasks, specifically including: Step S41: Initialize the historical area representation sequence For storing the representation of the urban historical area, and obtaining the representation of urban area i at time t by the urban area encoder Stored entry ; Step S42: Extract the historical regional representations of the previous C days from the historical representation sequence and take out the task data corresponding to the previous C days according to the current task as the support node representation for prediction: (13) where k represents that the sample data belongs to the k-th downstream task, represents the dynamic representation vector of region i at time t, represents the historical representation vector belonging to the downstream task k at time t; Step S43: According to the attention mechanism, first construct a query vector based on the current moment representation of the urban area: (14) Among them, the weight matrix is a learnable parameter; Step S44: Construct a support vector library based on the historical representation vectors of the urban area and the historical data of a specific downstream task, and calculate the association weights between the current urban area representation and the historical representations and data according to the query vector : (15) (16) Among them, represents the key value vector of region i at moment, are two different weight matrices, which are learnable parameters, and T represents the transpose operation of the matrix; Step S45: By performing weighted aggregation and linear transformation on the k-th downstream task data of the previous c days to obtain target task data for prediction: (17) Among them, represents the value of the k-th downstream task for the region i to be predicted, represents the learnable parameter. The MLP is tuned during the pre-training process of the linear prediction layer, and the parameters do not change during the downstream task inference.

Images