Human mobility prediction method and device based on cooperation of large language model and small model plug-in

Abstract

The invention discloses a human mobility prediction method and device based on cooperation of a large language model and a small model plug-in, and the method comprises the steps: preprocessing original user sign-in data, and constructing a time sequence knowledge graph which comprises the global interaction relation between a user and a place in each time period, and is used for representing the sign-in behavior of the user; meanwhile, constructing a long-term historical sign-in track and a recent sign-in track of the user; a user memory vector is constructed, and a plug-in small model composed of a plurality of encoders is trained. The plug-in small model is not limited by the form described in the application, and other plug-in small models can be freely replaced; extracting a historical sign-in track and a recent sign-in track of a user part, and generating cue words by combining data such as first K prediction tags, confidence scores and reference truth values of various preferences generated by the small model; and supervising and finely tuning the large language model by using historical data, and predicting the track of the user according to the prompt without the reference truth value to obtain a final prediction result. According to the invention, a more accurate prediction result can be provided.

Description

Technical Field

[0001] The present invention relates to the technical field of data mining and recommendation prediction, and in particular to a method and device for predicting human mobility based on a large language model in collaboration with a small model plug-in. Background Art

[0002] With the rapid development of mobile Internet and location-based services, users are increasingly relying on location-based applications to discover and explore places around them in their daily lives. Therefore, location-based service providers are able to obtain the mobility trajectory of users. Human mobility refers to people moving from one place to another, usually within a geographical area, such as a city or region. Human mobility prediction is of great significance in controlling the spread of diseases, alleviating traffic congestion, improving travel efficiency, and commercial applications (such as business location selection and location recommendation), and is an important part of smart cities.

[0003] The uniqueness of human mobility lies in its inherent regularity, randomness, and complex spatiotemporal dependencies, which make it extremely challenging to accurately predict people's whereabouts. Existing research is dedicated to revealing the laws of human mobility through a variety of methods such as sequence models and graph models, that is, leveraging the ability of deep learning models in spatiotemporal modeling to achieve more accurate prediction results. However, the prediction accuracy of these models still has room for improvement. More importantly, they are insufficient in directly explaining their prediction results, which limits the interpretability and practicality of these models in real-world applications.

[0004] In recent years, with the rapid development of large language model technology, large models have accumulated rich common sense knowledge covering many fields. This not only enables them to achieve outstanding results in various natural language processing tasks, but also shows impressive performance in many professional fields in the case of zero-shot learning. However, large language models have certain limitations in theory. First, these models have a limited effective context length when processing input data, and cannot process all mobile trajectory data and its associated data at one time, which limits the model's ability to perform overall statistical evaluation of user behavior at a macro level. Second, although these models have extensive general knowledge, they still find it difficult to fully capture and learn individual segmented preferences in the task of predicting human mobility in specific scenarios. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, a method and device for predicting human mobility based on a large language model and a small model plug-in are provided. The present invention improves the existing solution based on practical problems in real scenarios, builds a large language model with an embedded small model, and combines the advantages of both, thereby improving the accuracy of human mobility prediction.

[0006] To achieve the above object, the first aspect of the present invention relates to a method for predicting human mobility based on a large language model in collaboration with a small model plug-in, comprising the following steps:

[0007] S1: Preprocess the original user check-in data and construct a temporal knowledge graph containing the global interaction relationship between users and places in each time period to represent the user's check-in behavior. At the same time, construct the user's long-term historical check-in trajectory and recent check-in trajectory;

[0008] S2: construct a user memory vector and train a plug-in model consisting of multiple encoders. The plug-in model is not limited to the form described in this application and can be freely replaced by other plug-in models;

[0009] S3: Extract some historical check-in trajectories and recent check-in trajectories of the user, and use the small model to generate the top K predicted labels, confidence scores, and benchmark truth values ​​for each preference to generate prompt words;

[0010] S4: Use historical data to supervise and fine-tune a large language model, predict the user's whereabouts based on prompts without ground truth, and obtain the final prediction result.

[0011] Preferably, step S1 specifically includes:

[0012] The user's check-in is processed into a check-in quadruple (u, r, l, d), where u and l are the user and location entities, r is the time period relationship, and d represents the date. Specifically, the check-in quadruple reflects the check-in behavior of user u in a specific time period on a certain date. The present invention divides the knowledge graph into snapshots of different dates according to the user's check-in time, and divides it into multiple equally divided time periods according to the time of a snapshot, so as to obtain the time period relationship of the user's check-in. In addition, the previous check-in of each user's check-in is saved, and the human flow relationship between global locations is constructed based on this. In addition, the present invention needs to construct a historical check-in trajectory of length M and a recent check-in trajectory of length N for each user.

[0013] Preferably, step S2 specifically includes:

[0014] S2.1: Construct the historical check-in frequency of all locations of the user in different time periods into a user memory vector. Specifically, for each user check-in record, the present invention traverses all historical check-in data before its timestamp t to calculate the user's check-in memory vector for each location As shown in formula (1).

[0015]

[0016] Among them, t i and t j represents the check-in record of user u and ti >t j timestamp, r represents the user's check-in time period, and d represents the time information of the check-in quadruple, that is, the date number. Next, we use the following formula to convert Convert to

[0017]

[0018] S2.2: A learning strategy based on a replication mechanism is adopted to model the user's personal preferences from two aspects. One is the user's habits, and the other is the user's potential exploration needs for checking in to new places. Regarding the user's habitual preferences, existing studies have observed that users' daily lives have certain regularities, which means that researchers can capture the user's habitual preferences through the time information of the current query context and the user's historical check-in data. Therefore, the present invention connects the user entity embedding and the temporal relationship embedding as query features, and then uses it to generate a potential habit preference vector as follows:

[0019]

[0020] Where tanh is the activation function, and is a trainable parameter, L is the location embedding vector, represents the embedding splicing operation. It is worth noting that the present invention will Adding it to the habit preference vector divides the candidate space into two parts, making the habit preference vector pay more attention to the places that have been checked in before.

[0021] However, due to factors such as daily life needs, curiosity about new things, and the influence of popular events, users may also be interested in places they have never checked in before. Therefore, in contrast to the user's habitual preferences, the present invention also needs to capture the user's novel preferences. According to the same logic as modeling the user's habitual preferences, the user's novelty preference vector is obtained as follows:

[0022]

[0023] Contrary to conventional preference, the present invention will The negative number of is added to the early adopter preference vector, making the early adopter preference vector pay more attention to the places that the user has never checked in before.

[0024] S2.3: For each sign-in quadruple (u, r, l, d), the present invention saves the previous sign-in record, i.e. (u, r i ,l i ,d i ,r j ,l j ,d j). To obtain the time information between the sequential check-ins, the present invention uses the date difference between the two check-ins and the unit date vector d u Finally, the present invention combines the embedding of sequential check-ins to obtain the location flow transfer feature vector:

[0025] Δd=(d i -d j )×d u (5)

[0026]

[0027] S2.4: The various preference vectors obtained are activated by the Soft-max function to obtain the phased prediction results, as shown in the following formula:

[0028] P=σ(V) (7)

[0029] Preferably, in step S3, to construct the prompt word, the present invention selects M records from the historical check-in to form a historical check-in sequence, and selects N records from the recent check-in to form a recent check-in sequence. In addition, the present invention uses a large language model to generate a user profile summary of limited length based on the user's historical data. The profile summarizes the user's personal information, including key information such as gender, age, education level, economic status, and hobbies.

[0030] Then, the present invention performs a top-k accuracy analysis on each preference prediction in the plug-in model. We find that when the plug-in model predicts the top 30 most likely locations, it can achieve an accuracy of more than 80%, but when more prediction items are selected, the relative accuracy improvement is not obvious. Therefore, after comprehensive evaluation, the present invention selects the top 30 most likely prediction results as candidates to avoid generating too long prompt word contexts.

[0031] Ultimately, these data are used as cue words to predict human mobility.

[0032] Preferably, step S4 specifically includes:

[0033] S4.1: Fine-tuning a large language model can be costly, so the present invention applies parameter-efficient fine-tuning (PEFT) techniques during the fine-tuning phase. The present invention applies the Low-Rank Adaptation (LoRA) algorithm to freeze the dense layers in a large language model and update the weights with a rank decomposition matrix, so that only a small number of parameters receive gradient updates during training.

[0034] S4.2: To reduce excessive video memory overhead, the present invention adopts a double quantization method, that is, Normal Float 4-bit quantization is used for storage, and Brain Floating 16-bit quantization is used for forward and backward propagation.

[0035] S4.3: Since a large amount of check-in trajectory data needs to be given in the prompt word when predicting human mobility, a longer prompt word context length is required. Therefore, the present invention applies FlashAttention-2 technology to enable Transformers to accept longer contexts.

[0036] S4.4: After obtaining the supervised fine-tuned large language model, the present invention uses prompt words without label data to perform predictions to obtain the final prediction results.

[0037] The second aspect of the present invention relates to a human mobility prediction device based on a large language model and a small model plug-in, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the human mobility prediction method based on a large language model and a small model plug-in of the present invention.

[0038] The third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the human mobility prediction method based on a large language model and a small model plug-in of the present invention.

[0039] The present invention constructs a user's whereabouts time series knowledge graph by analyzing the user's check-in data, aiming to realize a lightweight and efficient traditional model; extracts the user's historical and recent trajectories, and combines the plug-in traditional model and other data as the input prompt words of the large language model; the pre-trained large language model has a rich common sense knowledge base, which deeply understands the implicit knowledge in the context information that is difficult to mine with traditional methods; in addition, the large language model can explain the prediction results, which solves the shortcomings of traditional methods in explaining the prediction results; the plug-in model effectively solves the limitations of large language models in processing long contexts and mining non-common sense preferences. The present invention can provide more accurate prediction results.

[0040] Compared with the prior art, the main beneficial effects of the present invention are:

[0041] 1. The plug-in model constructed based on the proposed user whereabouts time series knowledge graph is a new time series knowledge graph reasoning model that can be used for individual human mobility prediction tasks. It uses a context-aware replication mechanism to identify users' habits and preferences, and learns from global user data through a simple model structure instead of complex graph convolution, effectively alleviating the data sparsity problem and solving the user cold start problem.

[0042] 2. The present invention proposes a technical architecture that aims to coordinate pre-trained large-scale language models and plug-in models to achieve the prediction of human mobility. This architecture uses the extensive common-sense knowledge contained in large-scale language models to provide many useful considerations for human mobility prediction. The present invention integrates small deep learning models constructed using traditional methods as plug-ins, and uses the prediction results generated by these small models as input prompts to activate the common-sense knowledge of large-scale language models. In addition, while predicting human mobility, the present invention can also use the natural language processing capabilities of large-scale language models to generate corresponding prediction reasons. In short, the present invention can not only effectively make up for the limitations of traditional methods, but also significantly improve the accuracy of human mobility prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a model structure diagram of a method for predicting human mobility based on a large language model in collaboration with a small model plug-in according to the present invention;

[0044] Figure 2 The figure is a flow chart of the method for predicting human mobility based on a large language model in collaboration with a small model plug-in according to the present invention.

[0045] Figure 3 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] It should be noted that, in the absence of conflict, the features in the following embodiments and implementations may be combined with each other.

[0048] like Figure 1 and Figure 2 As shown, the present invention proposes a method for predicting human mobility based on a large language model in collaboration with a small model plug-in, and the method specifically includes the following steps:

[0049] S1: Preprocess the original user check-in data and construct a temporal knowledge graph containing the global interaction relationship between users and places in each time period to represent the user's check-in behavior. At the same time, construct the user's long-term historical check-in trajectory and recent check-in trajectory;

[0050] S2: construct a user memory vector and train a plug-in model consisting of multiple encoders. The plug-in model is not limited to the form described in this application and can be freely replaced by other plug-in models;

[0051] S3: Extract some historical check-in trajectories and recent check-in trajectories of the user, and use the small model to generate the top K predicted labels, confidence scores, and benchmark truth values ​​for each preference to generate prompt words;

[0052] S4: Use historical data to supervise and fine-tune a large language model, predict the user's whereabouts based on prompts without ground truth, and obtain the final prediction result.

[0053] The above-mentioned method for predicting human mobility based on a large language model and a small model plug-in, S1 specifically includes the following steps:

[0054] The user's check-in is processed into a check-in quadruple (u, r, l, d), where u and l are the user and location entities, r is the time period relationship, and d represents the date. Specifically, the check-in quadruple reflects the check-in behavior of user u in a specific time period on a certain date. The present invention divides the knowledge graph into snapshots of different dates according to the user's check-in time, and divides it into multiple equally divided time periods according to the time of a snapshot, so as to obtain the time period relationship of the user's check-in. In addition, the previous check-in of each user's check-in is saved, and the human flow relationship between global locations is constructed based on this. In addition, the present invention needs to construct a historical check-in trajectory of length M and a recent check-in trajectory of length N for each user.

[0055] Preferably, the step S2 specifically includes:

[0056] S2.1: Construct the historical check-in frequency of all locations of the user in different time periods into a user memory vector. Specifically, for each user check-in record, the present invention traverses all historical check-in data before its timestamp t to calculate the user's check-in memory vector for each location As shown in formula (1).

[0057]

[0058] Among them, t i and t j represents the check-in record of user u and t i >t jtimestamp, r represents the user's check-in time period, and T represents the time information of the check-in quadruple, that is, the date number. Next, we use the following formula to convert Convert to

[0059]

[0060] S2.2: A learning strategy based on a replication mechanism is adopted to model the user's personal preferences from two aspects. One is the user's habits, and the other is the user's potential exploration needs for checking in to new places. Regarding the user's habitual preferences, existing studies have observed that users' daily lives have certain regularities, which means that researchers can capture the user's habitual preferences through the time information of the current query context and the user's historical check-in data. Therefore, the present invention connects the user entity embedding and the temporal relationship embedding as query features, and then uses it to generate a potential habit preference vector as follows:

[0061]

[0062] Where tanh is the activation function, and is a trainable parameter, L is the location embedding vector, represents the embedding splicing operation. It is worth noting that the present invention will Adding it to the habit preference vector divides the candidate space into two parts, making the habit preference vector pay more attention to the places that have been checked in before.

[0063] However, due to factors such as daily life needs, curiosity about new things, and the influence of popular events, users may also be interested in places they have never checked in before. Therefore, in contrast to the user's habitual preferences, the present invention also needs to capture the user's novel preferences. According to the same logic as modeling the user's habitual preferences, the user's novelty preference vector is obtained as follows:

[0064]

[0065] Contrary to conventional preference, the present invention will The negative number of is added to the early adopter preference vector, making the early adopter preference vector pay more attention to the places that the user has never checked in before.

[0066] S2.3: For each sign-in quadruple (u, r, l, d), the present invention saves the previous sign-in record, i.e. (u, r i ,l i ,d i ,r j ,l j ,d j). To obtain the time information between the sequential check-ins, the present invention uses the date difference between the two check-ins and the unit date vector d u Finally, the present invention combines the embedding of sequential check-ins to obtain the location flow transfer feature vector:

[0067] Δd=(d i -d j )×d u (5)

[0068]

[0069] S2.4: The various preference vectors obtained are activated by the Soft-max function to obtain the phased prediction results, as shown in the following formula:

[0070] P=σ(V) (7)

[0071] Preferably, in step S3, in order to construct the prompt words, the present invention selects M records from the historical check-in to form a historical check-in sequence, and selects N records from the recent check-in to form a recent check-in sequence. In addition, the present invention uses a large language model to generate a user profile summary of limited length based on the user's historical data. The profile summarizes the user's personal information, including key information such as gender, age, education level, economic status, and hobbies.

[0072] Then, the present invention performs a top-k accuracy analysis on each preference prediction in the plug-in model. We find that when the plug-in model predicts the top 30 most likely locations, it can achieve an accuracy of more than 80%, but when more prediction items are selected, the relative accuracy improvement is not obvious. Therefore, after comprehensive evaluation, the present invention selects the top 30 most likely prediction results as candidates to avoid generating too long prompt word contexts.

[0073] Finally, these data are used as prompt words to predict human mobility, as shown below:

[0074] <question>:Your task is to conduct a comprehensive analysis of User{UID}'s preferences and then consider which POI id the user will visit attime{time}?

[0075] Given the user’s history check-ins sequence:{history_sequence}(format:[(PID,Category,Time),...]).

[0076] Given the user’s recent check-ins sequence:{recent_sequence}(format:[(PID,Category,Time),...]).

[0077] Given the plugin model's predictions of the user's preferences attime{time}:{plugin_predictions}(format:{"Specific Preference”:[(PID,Category,Distance,Confidence),...],...})

[0078] Now let me explain the elements in the format."PID”refers to theunique ID of the POI,"Distance”represents the distance(in kilometers)betweenthe user's last visited address and the current POI,and"Category”representsthe semantic information of the POI."Confidence”represents the confidencelevel of the plugin model's predicted results.

[0079] <answer>:At time{time},user{UID} will visit POI{gt_PID}which is a / an{gt_Category}.

[0080] Preferably, the step S4 specifically includes:

[0081] S4.1: Fine-tuning a large language model can be costly, so the present invention applies parameter-efficient fine-tuning (PEFT) techniques during the fine-tuning phase. The present invention applies the Low-Rank Adaptation (LoRA) algorithm to freeze the dense layers in a large language model and update the weights with a rank decomposition matrix, so that only a small number of parameters receive gradient updates during training.

[0082] S4.2: To reduce excessive video memory overhead, the present invention adopts a double quantization method, that is, Normal Float 4-bit quantization is used for storage, and Brain Floating 16-bit quantization is used for forward and backward propagation.

[0083] S4.3: Since a large amount of check-in trajectory data needs to be given in the prompt word when predicting human mobility, a longer prompt word context length is required. Therefore, the present invention applies FlashAttention-2 technology to enable Transformers to accept longer contexts.

[0084] S4.4: After obtaining the supervised fine-tuned large language model, the present invention uses prompt words without label data to perform predictions to obtain the final prediction results.

[0085] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0086] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.< / answer> < / question>

Claims

1. A method for predicting human mobility based on a large language model and a small model plug-in, comprising the following steps: S1: Preprocess the original user check-in data and construct a temporal knowledge graph containing the global interaction relationship between users and places in each time period to represent the user's check-in behavior; at the same time, construct the user's long-term historical check-in trajectory and recent check-in trajectory; S2: construct a user memory vector and train a small plug-in model composed of multiple encoders; the small plug-in model is not limited to the form described in this application and can freely replace other small plug-in models; S3: Extract some historical check-in trajectories and recent check-in trajectories of the user, and use the small model to generate the top K predicted labels, confidence scores, and benchmark truth values ​​for each preference to generate prompt words; S4: Use historical data to supervise and fine-tune a large language model, predict the user's whereabouts based on prompts without ground truth, and obtain the final prediction result.

2. A method for predicting human mobility based on a large language model and a small model plug-in as claimed in claim 1, characterized in that: Step S1 specifically includes: The user's check-in is processed into a check-in quadruple (u, r, l, d), where u and l are the user and location entities, r is the time period relationship, and d represents the date. Specifically, the check-in quadruple reflects the check-in behavior of user u in a specific time period on a certain date. According to the user's check-in time, it is divided into knowledge graph snapshots of different dates, and divided into multiple equally divided time periods according to the time of a snapshot, so as to obtain the time period relationship of the user's check-in. In addition, the previous check-in of each user is saved, and the human flow relationship between global locations is constructed based on this. In addition, a historical check-in trajectory of length M and a recent check-in trajectory of length N need to be constructed for each user.

3. The method for predicting human mobility based on a large language model and a small model plug-in as claimed in claim 1, characterized in that: Step S2 specifically includes: S2.1: Construct the historical check-in frequency of all locations of the user in different time periods into a user memory vector; specifically, for each user check-in record, traverse all historical check-in data before its timestamp t to calculate the user's check-in memory vector for each location As shown in formula (1); Among them, t i and t j represents the check-in record of user u and t i >t j timestamp, r represents the user's check-in time period, and d represents the time information of the check-in quadruple, that is, the date number; Next, according to the following formula Convert to S2.2: A learning strategy based on replication mechanism is adopted to model the user's personal preferences from two aspects; one is the user's habits, and the other is the user's potential exploration needs to check in to new places; the user's habitual preferences are captured through the time information of the current query context and the user's historical check-in data; therefore, the user entity embedding and the temporal relationship embedding are concatenated as query features, and then used to generate the potential habit preference vector as follows: Where tanh is the activation function, and is a trainable parameter, L is the location embedding vector, represents the embedding concatenation operation; it is worth noting that Adding it to the habit preference vector divides the candidate space into two parts, making the habit preference vector pay more attention to the places that have been checked in before; Capture the user's novelty preference; follow the same logic as modeling the user's habitual preference to obtain the user's novelty preference vector as follows: Will The negative number of is added to the early adopter preference vector, making the early adopter preference vector pay more attention to the places that the user has never checked in before; S2.3: For each sign-in quadruple (u, r, l, d), the previous sign-in record (u, r i , l i , d i , r j , l j , d j ); In order to obtain the time information between the sequential check-ins, according to the date difference between the two check-ins and the unit date vector d u The time difference embedding is generated; finally, the embedding of the sequential check-in is combined to obtain the location flow transfer feature vector: Δd=(d i -d j )×d u (5) S2.4: The various preference vectors obtained are activated by the Soft-max function to obtain the phased prediction results, as shown in the following formula: P=σ(V) (7) 4. The method for predicting human mobility based on a large language model and a small model plug-in as claimed in claim 1, characterized in that: In step S3, in order to construct the prompt word, M records are selected from the historical check-in to form a historical check-in sequence, and N records are selected from the recent check-in to form a recent check-in sequence; in addition, a user profile summary of limited length is generated based on the user's historical data using a large language model; the profile summarizes the user's personal information, including key information such as gender, age, education level, economic status, and hobbies; Then, the accuracy of each preference prediction in the plug-in model is analyzed, and the top k prediction results are selected to avoid generating too long prompt word context; Ultimately, these data are used as cue words to predict human mobility.

5. The method for predicting human mobility based on a large language model and a small model plug-in as claimed in claim 1, characterized in that: Step S4 specifically includes: S4.1: Apply parameter-efficient fine-tuning PEFT techniques during fine-tuning; apply the low-rank adapter algorithm LoRA to freeze dense layers in large language models and update weights with rank-decomposed matrices so that only a small number of parameters receive gradient updates during training; S4.2: A dual quantization method is used, where Normal Float 4-bit quantization is used for storage, and Brain Floating 16-bit quantization is used for forward and backward propagation; S4.3: Apply FlashAttention-2 technology to enable Transformers to accept longer contexts; S4.4: After obtaining the supervised fine-tuned large language model, use the prompt words without labeled data to make predictions and obtain the final prediction results.

6. A human mobility prediction device based on a large language model and a small model plug-in, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the human mobility prediction method based on a large language model and a small model plug-in as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the method for predicting human mobility based on a large language model and a small model plug-in as described in any one of claims 1 to 5 is implemented.

Images