Urban POI distribution prediction method based on context learning

Abstract

This invention proposes a method for predicting the distribution of urban POIs based on contextual learning, belonging to the fields of mobile communications and spatiotemporal data mining. The method comprises the following steps: S1: inputting user signaling data and POI data into a mask model based on contextual learning to obtain a mask sequence; S2: using the mask sequence to train and fine-tune an urban POI distribution prediction encoder, extracting semantic information from the mask sequence and obtaining a confidence probability that each region contains a certain type of POI; S3: encoding data on a small number of regions containing known POI types to obtain a mask sequence to be predicted, which is then input into the trained urban POI distribution prediction encoder. The posterior probability is then combined with the initial prior probability of the POI type and the confidence probability output by the model to determine whether each region contains a POI of that type. This method requires no additional training or fine-tuning and is applicable to regional POI prediction tasks of different categories, improving prediction efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of spatiotemporal data mining, and in particular relates to a method for predicting urban POI distribution based on context learning. Background Art

[0002] Artificial general intelligence (AGI) is a key goal in AI research. It refers to a pre-trained model that can generalize to a wide range of tasks without additional training. Contextual learning plays a crucial role in achieving AGI. Contextual learning involves the ability of a model to generate appropriate results based on a set of specific prompts. A classic example is providing a large language model (such as a language model) with examples of English-French translations and then inputting an English sentence and expecting the model to output the corresponding French translation. To achieve this goal, pre-training is typically performed on a rich dataset, followed by fine-tuning on a rich dataset covering over a thousand downstream tasks.

[0003] In the field of urban perception, designing a universal model remains challenging. Urban perception encompasses a diverse range of POI tasks, each with complex underlying relationships. Current technologies primarily focus on designing specialized models for specific tasks, or generating task-specific representations and then fine-tuning them to solve downstream tasks. Building a city POI distribution prediction model that generalizes to a wide range of tasks without requiring additional fine-tuning or training has become a pressing challenge in the field. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a method for predicting urban POI distribution based on context learning, comprising the following steps:

[0005] Step S1: Input user signaling data and POI data into the mask model based on context learning to obtain a mask sequence;

[0006] Step S2: Using the mask sequence to train and fine-tune the city POI distribution prediction encoder, extracting the semantic information in the mask sequence, and obtaining the confidence probability that each area contains a certain type of POI;

[0007] Step S3: Encode the data of known POI types contained in a small number of areas to obtain the mask sequence to be predicted, and input it into the trained city POI distribution prediction encoder. The posterior probability is obtained by combining the initial prior probability of the POI type and the confidence probability output by the model to determine whether each area contains the POI of this type.

[0008] Beneficial effects:

[0009] 1. This paper proposes a novel mask model, which converts continuous data of different categories into 0 / 1 sequences and then performs masking operations, which can effectively improve the training effect of the model.

[0010] 2. This invention uses different types of data for pre-training and fine-tuning, fully considering the characteristics of the data itself. This allows the model to capture important information such as the data distribution, and makes the regional representations generated by the model more versatile. The trained model can be generalized to a wide range of tasks without additional fine-tuning or training. This invention has achieved good results on more than 100 downstream tasks.

[0011] 3. Existing technologies still mainly focus on designing proprietary models for dedicated tasks, or generating regional representations and then fine-tuning them according to specific tasks to solve downstream tasks. This results in poor versatility of the model and inability to generalize to a large number of unknown tasks without additional training and fine-tuning. However, the present invention can construct a mask sequence based on group mobility data and POI data, and use quantiles to unify the data format, and then train the model with the goal of restoring the masked part, thereby obtaining a universal representation of urban areas, making the representation extremely versatile. The universal representation of urban areas can be applied to different categories of regional POI prediction tasks without additional training or fine-tuning, greatly improving the prediction efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A flow chart of a method for predicting urban POI distribution based on context learning according to the present invention;

[0013] Figure 2 Schematic diagram of the city POI distribution prediction process based on context learning;

[0014] Figure 3 Schematic diagram of the training and fine-tuning process of the encoder for predicting city POI distribution. DETAILED DESCRIPTION

[0015] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only intended to illustrate the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0016] Example 1

[0017] like Figure 1 As shown, an embodiment of the present invention provides a method for predicting urban POI distribution based on context learning, comprising the following steps:

[0018] Step S1: Input user signaling data and POI data into the mask model based on context learning to obtain a mask sequence;

[0019] Step S2: Use the mask sequence to train and fine-tune the city POI distribution prediction encoder, extract the semantic information in the mask sequence, and obtain the confidence probability that each area contains a certain type of POI;

[0020] Step S3: Encode the data of known POI types contained in a small number of areas to obtain the mask sequence to be predicted, and input it into the trained city POI distribution prediction encoder. The posterior probability is obtained by combining the initial prior probability of the POI type and the confidence probability output by the model to determine whether each area contains the POI of this type.

[0021] In one embodiment, the above step S1: inputting user signaling data and POI data into a mask model based on context learning to obtain a mask sequence specifically includes:

[0022] Step S11: Divide urbanization into multiple 1km 1km area;

[0023] Step S12: Obtain user signaling data, including: starting longitude, starting latitude, ending longitude, ending latitude, departure time, and arrival time; calculate OD data based on user signaling data: calculate the influx of people in each area within a certain time period, that is, OD amount = time period Number of people-time period the number of personnel;

[0024] User signaling data offers significant advantages for regional analysis and user profiling. First, this data source, based on massive trajectory records, covers a wide range of spatial dimensions across time and location, providing precise geolocation and timestamp information. Second, the data's semantic dimensions are extremely rich, capturing key information such as travel frequency, driving paths, and regional traffic flow. The multi-layered and multi-dimensional nature of this data makes it extremely valuable for regional POI prediction.

[0025] Step S13: Obtain POI data, including: POI category, longitude, and latitude; calculate the number of POIs of each category in different areas based on the POI category, longitude, and latitude;

[0026] Step S14: inputting the POI data and OD data into the mask model, performing quantile division according to a predefined distribution, and converting them into the corresponding 0 / 1 matrix set H;

[0027] The mask model first processes the POI data and OD data to unify the OD data and POI data types, that is, converting the continuous data into a 0 / 1 sequence representing whether it belongs to the quantile range (based on the number of POI categories in a certain area or whether the OD volume belongs to the category, or the top N% of the OD volume of all areas) to obtain a set of enhanced data. For example, If the number of a certain POI category in the area is the top N% of the number of POI categories in all areas, then = 1, otherwise 0. The final data is a matrix set H, each matrix in the set H is an N The matrix of M, N M corresponds to the regional grid of each city, and the 0 / 1 value in the matrix represents whether it belongs to a certain distribution. Figure 2 As shown, the predefined distribution in the embodiment of the present invention includes: whether the area belongs to the top 10% of the areas with the number of ODs in period A, whether it belongs to the top 30% of the areas with the number of ODs in period A, whether it belongs to the top 50% of the areas with the number of ODs in period B, and whether it belongs to the top 70% of the areas with the number of Category C POIs.

[0028] Step S15: Randomly select the true value of a part of the data in each matrix in H for masking to obtain a mask matrix set ,right Flatten the image to obtain a one-dimensional mask sequence, including a POI mask sequence and an OD mask sequence.

[0029] In one embodiment, the above step S2: using the mask sequence to train and fine-tune the city POI distribution prediction encoder, extracting the semantic information in the mask sequence, and obtaining the confidence probability that each area contains a certain type of POI, specifically includes:

[0030] Step S21: Construct a city POI distribution prediction encoder based on the Transformer encoder, and each area is randomly initialized to a vector representation;

[0031] Step S22: inputting the POI mask sequence into the city POI distribution prediction encoder for pre-training, using a cross entropy loss function to learn the flow patterns between different areas in the city;

[0032] Step S23: Input the OD mask sequence into the city POI distribution prediction encoder for fine-tuning to better capture the detailed features of the city;

[0033] Step S24: Through the training and fine-tuning of steps S22 and S23, the vector representation is gradually updated according to the model's loss function, so that the model can gradually learn the semantic features of the area and improve the model's performance in unknown tasks; finally, the vector representation is restored to the mask part through a linear layer and a sigmoid function, and the confidence probability of each predicted area containing a certain type of POI is output.

[0034] like Figure 2 The Transformer encoder architecture shown above uses a residual connection to ensure gradient propagation and network depth. The encoder input is added to the self-attention output. This is then processed again through layer normalization and a feedforward layer to produce a new vector representation. After obtaining the final vector representation for each region, it is mapped to the (0, 1) range through a linear layer and a sigmoid activation layer, serving as the confidence probability that each region contains a certain type of POI.

[0035] Figure 3 The training and fine-tuning process of the city POI distribution prediction encoder is demonstrated.

[0036] In one embodiment, step S3 above encodes the data of known POI types contained in a small number of regions to obtain a mask sequence to be predicted, and inputs the mask sequence into a trained city POI distribution prediction encoder. The posterior probability is obtained by combining the initial prior probability of the POI type with the confidence probability output by the model to determine whether each region contains a POI of the type. Specifically, the following steps are performed:

[0037] Step S31: Determine the initial prior probability of a certain type of POI:

[0038] (1)

[0039] in, represents the number of regions where the POI of this type is known to exist, and N represents the number of regions where this POI category has true values in the region;

[0040] For example, if a certain POI type exists in one of the 10 areas in a city (the true value is 1), does not exist in one area (the true value is 0), and the other areas are unknown, then =1, , .

[0041] Step S32: constructing a mask sequence to be predicted based on a small number of samples, marking the area with the presence of POI of this type as 1, marking the area without the presence of POI of this type as 0, and marking the remaining unknown areas as masks;

[0042] Step S33: Input the mask sequence into the trained city POI distribution prediction encoder to obtain the probability of all regions containing POIs of this type; Set as the probability threshold of the final prediction result. When the probability output by the model is greater than the probability threshold, the area is considered to contain the POI of this type, otherwise it does not exist.

[0043] Step S34: Use the Bayesian formula to determine the posterior probability of this type of POI in all areas:

[0044] (2)

[0045] in, represents the proportion of areas in the known area in step S32 where the model predicts that there is no POI of this type;

[0046] Step S35: The posterior probability As the benchmark probability of the final prediction result, sort the output probability of the final prediction result from large to small, and select the one with the highest ranking If the area has a POI of this type, it is determined that there is a POI of this type; otherwise, it is determined that there is no POI of this type in the area.

[0047] For downstream tasks to be predicted, the present invention only requires the ground truth values of a small number of regions (this ground truth value must be a 0 / 1 sequence, indicating whether the region contains a certain category of POI). The city POI distribution prediction encoder can then recover the values of other regions. This method captures the distribution characteristics of the input data rather than the task itself, enabling the model to generalize to unknown tasks without fine-tuning. This addresses the universality issue of urban intelligence and has achieved good results in over 100 downstream tasks.

Claims

1. A method for predicting urban POI distribution based on context learning, characterized in that: include: Step S1: Input user signaling data and POI data into the context-based learning mask model to obtain a mask sequence, which specifically includes: Step S11: Divide the city into multiple 1km 1km area; Step S12: Obtain user signaling data, including: starting longitude, starting latitude, ending longitude, ending latitude, departure time, and arrival time; calculate OD data based on the user signaling data: calculate the influx of people in each area within a certain time period, that is, OD amount = time period Number of people-time period the number of personnel; Step S13: Obtain POI data, including: POI category, longitude, and latitude; calculate the number of POIs of each category in different areas based on the POI category, longitude, and latitude; Step S14: Input the POI data and the OD data into the mask model, perform quantile division according to the predefined distribution, and convert them into the corresponding 0 / 1 matrix set H, where each matrix in the matrix set H is an N The matrix of M, N M corresponds to the regional grid of each city, and the 0 / 1 value in the matrix represents whether it belongs to a certain distribution; Step S15: Randomly select the true value of a part of the data in each matrix in H for masking to obtain a mask matrix set ,right Flatten the image to obtain a one-dimensional mask sequence, including the POI mask sequence and the OD mask sequence; Step S2: using the mask sequence to train and fine-tune the city POI distribution prediction encoder, extracting semantic information from the mask sequence, and obtaining the confidence probability that each area contains a certain type of POI; Step S3: Encode the data of known POI types contained in a small number of areas to obtain the mask sequence to be predicted, and input it into the trained city POI distribution prediction encoder. The posterior probability is obtained by combining the initial prior probability of the POI type and the confidence probability output by the model to determine whether each area contains the POI of this type.

2. The method for predicting urban POI distribution based on context learning according to claim 1, characterized in that: Step S2: using the mask sequence to train and fine-tune the city POI distribution prediction encoder, extracting semantic information from the mask sequence, and obtaining the confidence probability that each area contains a certain type of POI, specifically includes: Step S21: Construct a city POI distribution prediction encoder based on the Transformer encoder, and each area is randomly initialized to a vector representation; Step S22: inputting the POI mask sequence into the city POI distribution prediction encoder for pre-training, using a cross entropy loss function to learn the flow patterns between different areas in the city; Step S23: inputting the OD mask sequence into the city POI distribution prediction encoder for fine-tuning to better capture detailed features in the city; Step S24: Through the training and fine-tuning of steps S22 and S23, the vector representation is gradually updated according to the model's loss function, so that the model can gradually learn the semantic features of the area and improve the model's performance in unknown tasks; finally, the vector representation is restored to the mask part through a linear layer and a sigmoid function, and the confidence probability of each predicted area containing a certain type of POI is output.

3. The method for predicting urban POI distribution based on context learning according to claim 2, characterized in that: Step S3: Encode the data of known POI types contained in a small number of regions to obtain a mask sequence to be predicted, and input it into the trained city POI distribution prediction encoder. Combine the initial prior probability of the type of POI with the confidence probability output by the model to obtain the posterior probability, and finally determine whether each region contains the type of POI. Specifically, it includes: Step S31: Determine the initial prior probability of a certain type of POI: （1） in, represents the number of regions where the POI of this type is known to exist, and N represents the number of regions where this POI category has true values in the region; Step S32: constructing a mask sequence to be predicted based on a small number of samples, marking the area with the presence of POI of this type as 1, marking the area without the presence of POI of this type as 0, and marking the remaining unknown areas as masks; Step S33: Input the mask sequence into the trained city POI distribution prediction encoder to obtain the probability of all regions containing POIs of this type; Set as the probability threshold of the final prediction result. When the probability output by the model is greater than the probability threshold, the area is considered to contain the POI of this type, otherwise it does not exist; Step S34: Use the Bayesian formula to determine the posterior probability of this type of POI in all areas: （2） in, represents the proportion of areas in the known area in step S32 where the model predicts that there is no POI of this type; Step S35: The posterior probability As the benchmark probability of the final prediction result, sort the output probability of the final prediction result from large to small, and select the one with the highest ranking If the area has a POI of this type, it is determined that there is a POI of this type; otherwise, it is determined that there is no POI of this type in the area.

Images