A location prediction method based on preference context and trajectory graph enhanced expression

Abstract

The present invention discloses a location prediction method based on preference context and trajectory graph enhancement expression. First, based on graph theory, the map is divided into blocks, and a vertex is established for each block. The POI data is vectorized and added to the attributes of the vertex; second, the dense location data is extracted from the research object through the set hyperparameters and rules, and edges are generated according to the time sequence of the location data, and global variables are generated according to the attributes of the research object itself; third, the multimodal preference data is vectorized and represented, and the generated preference vector is added to the attributes of the vertex; fourth, the vertices, edges and global vectors in the graph are updated based on the self-attention mechanism; fifth, based on the PAC theory, machine learning theory and recommendation system method, the self-attention mechanism is used in the generated research object trajectory graph to predict the possible location of the research object at the next moment and the possibility of the research object visiting the specified location.

Description

Technical Field

[0001] This invention is based on the theory of recommendation systems in machine learning, and innovates and applies algorithms based on the characteristics of data sources. It belongs to the field of recommendation systems, specifically involving multimodal preference context, trajectory representation, graph structure enhancement, and self-attention mechanism methods. Background Art

[0002] With the rapid development of artificial intelligence, machine learning algorithms have greatly boosted productivity in areas such as machine vision, natural language processing, and recommendation systems. Among them, location prediction algorithms play a crucial role in search and e-commerce.

[0003] The original purpose of recommendation algorithms was to address the problem of information overload. In today's information-exploding world, information overload is about extracting valuable information or information of interest to the research subject from the vast amount of information, thereby reducing the intensity of information processing required by the research subject and improving the efficiency of information utilization.

[0004] The mathematical foundations of recommendation algorithms are based on the law of large numbers and PAC theory. Specifically, location-based recommendation algorithms leverage the historical location data of research objects (specifically, objects that provide continuous location information, including but not limited to mobile phones, vehicle-mounted transmitters, GPS terminals, and Beidou terminals) to discover patterns and periodic characteristics within application scenarios. These patterns and periodicity are then used to predict demand at future points in time.

[0005] Because the activities of mobile subjects are based on three-dimensional locations, location-based prediction is a common scenario in e-commerce. The presence of a specific location in a subject's activity trajectory provides a relatively clear picture of the user's preferences. This has broad application value in commercial activities such as tourism and product recommendations.

[0006] A shortcoming of existing technologies is that algorithms built on sparse datasets often deal with data sparsity to varying degrees during their use. Most existing algorithms are based on modeling and calculations performed on sparse locations. Such algorithms inherently fail to accurately reflect the location preferences of location information devices. Data indicates that publicly published locations in the wild only account for 0.03% of the total number of locations visited by location information devices. Both theoretically and intuitively, location information device preferences derived from such a small data set cannot accurately reflect the true preferences of the research subjects. Furthermore, the sparsity of location data makes it difficult to uniformly embed other multimodal data with location information. The reason is obvious: location data submitted by a location information device rarely coincides with other related multimodal data. Furthermore, when a location information device experiences drift in interest, the time interval between two check-ins is uncertain due to data sparsity, making it impossible to distinguish between the location device's actual destination and a destination after the drift in interest.

[0007] Common sense introduction in the field:

[0008] POI data provides a structured representation of geographic data. This structured representation allows geographic features to be incorporated into the feature vectors required for prediction models. Based on the geographic features represented by POIs, geographic information can be applied in location-based prediction systems for a variety of scenarios, including tourism, dining, and shopping.

[0009] A graph is a data structure used to represent relationships between entities. Entities are represented by vertices (nodes), relationships between entities are represented by edges, and a global variable represents the overall properties of the graph. Graphs can not only represent simple data but also conveniently define relationships between entities. Therefore, graph structures are an excellent way to represent relationships between geographic locations and temporal sequences.

[0010] The Transformer model is an encoder-decoder model that uses a self-attention mechanism. Since its introduction, it has achieved remarkable results in natural language processing (NLP). Subsequent research has also seen its application in areas such as computer vision and speech coding, achieving excellent results. The Transformer model demonstrates the potential of a model to unify multimodal data. Summary of the Invention

[0011] The present invention discloses a location prediction method based on preference context and trajectory graph enhancement. This method integrates graph enhancement structure, multimodal preference context, self-attention mechanism, and object periodicity analysis for location prediction, achieving two goals: first, determining the likely location of the object at the next moment; and second, predicting the likelihood of the object visiting a given location. Specifically, "object" refers to an object that can provide continuous location information, including but not limited to mobile phones, vehicle-mounted transmitters, GPS terminals, and Beidou terminals.

[0012] This invention uses three types of data: first, point of interest (POI) data; second, dense trajectory data; and third, multimodal preference context. The algorithm first sets hyperparameters to determine input conditions; second, all data is fused to form an initial graph structure containing multiple features; third, the GAT model is used to dynamically aggregate vertices, edges, and global information in the initial graph; and fourth, the prediction of the next location is transformed into a ranking problem of associated predictions, predicting the possible locations of the research object. The probability of the research object appearing at a given location is transformed into a classification problem, determining whether the research object will be visited at a given location.

[0013] Technical solution of the present invention:

[0014] A location prediction method based on preference context and trajectory graph enhanced expression is characterized by: first, based on feature representation and graph theory, the geographic space is represented in the form of a graph structure, and then the POI data is vectorized and added to the attributes of the vertex; second, the multimodal preference context is integrated into the graph structure; third, the trajectory graph of the research object is extracted through the location data; fourth, the vertices, edges and global vectors in the graph are updated; fifth, the possible locations are predicted based on the trajectory graph enhanced expression. Further, based on feature representation and graph theory, the geographic space is represented in the form of a graph structure: by segmenting the two-dimensional plane of the geographic space, the vertices in the graph structure representing the spatial location are obtained. Further, the POI data is vectorized and added to the attributes of the vertex: after vectorizing the classification features of the POI, the attributes of the geographic part in each vertex attribute are obtained.

[0015] A location prediction method based on preference context and trajectory graph enhanced expression, characterized by the following specific method processes:

[0016] Step 10: Set multiple hyperparameters mainly based on time interval and spatial interval.

[0017] Step 20: Import the POI data of the specified area into the method and proceed to step 90.

[0018] Step 30: Input the dense trajectory data of the designated area and proceed to step 50.

[0019] Step 40: Input the multimodal data of the research subjects in the designated area and proceed to step 60;

[0020] Step 50: Generate a continuous partitioned map according to the spatial interval set in step 10, then generate vertices according to the slices, and proceed to step 70;

[0021] Step 60: Extract the regional preference information from the multimodal context and then embed it. Specifically, a decision fusion method based on feature fusion is used to extract the research subject preference feature vector from the multimodal data, and then proceed to step 110.

[0022] Step 70: Extract the designated research object to generate the object's stay points and the edges connecting the stay points, and proceed to step 80.

[0023] Step 80: Extract periodic information from the graph structure and proceed to step 100.

[0024] Step 90: Generate vertex position features based on the POI classification, denoted as f POI . POI Assign to V in the vertex feature vector location =f POI Part, providing step 120;

[0025] Step 100: Generate a graph enhancement structure, and generate an adjacency matrix of size n×n using the generated graph structure information, denoted as A=(a ij ),a ij =μ(v i ,v j ), where μ(v i ,v j ) is represented as a graph connection v i and v j The number of edges; provide step 120. The generated graph is denoted as G(V,E). It is vectorized using the attributes of the research object itself and denoted as U. The generated geographic location fragments, each of which is generated as a vertex. The set of all vertices is denoted as V = {v1,v2,…,v n}, where n is the number of vertices. Vertex feature vector, for a vertex, its feature is recorded as v i , where i is the vertex ID.

[0026] Step 110: Based on step 60, generate the multimodal preference context of the research subject, denoted as f preference . preference , providing step 120.

[0027] Step 120: Attach the trajectory features, periodic features, and preference context to the corresponding global, vertex, and edge in the generated graph enhancement structure. Finally, a matrix containing the trajectory features of the research object is formed. The attribute of the i-th vertex is recorded as v i , the kth edge is denoted as e k Among them, V location =f POI 、V preference =f preference , V location and V preference As shown in Table 4, the regional features and preference features in the vertex features are respectively represented.

[0028] Step 130: Aggregate and update the vertex, edge, and global variable features in the graph enhancement structure, and proceed to step 140 and step 150; two aggregations will be performed in the graph enhancement structure to facilitate message passing between vertices, edges, and global variables.

[0029] Vertex aggregation uses v ineighbor =∑ j a j v j +∑ k b k e k +cU to get vertex v i Auxiliary information of v j v i Neighbors with a distance of 1, e k v i The features of the connected edges, U is the global feature; a j 、b k , c are the weights obtained through self-attention mechanism training. is the v after convergence i The features of the dataset are used to obtain the training parameters through the transformer model.

[0030] Edge aggregation uses Get edge e i Auxiliary information of v j for e i The vertices at both ends, U feature is a global feature; a j , b weights obtained through self-attention mechanism training. After convergence, e iThe features of the dataset are used to obtain the training parameters through the transformer model.

[0031] Global variables aggregate using U agg =∑ j a j v j +∑ k b k e k Get U agg Auxiliary information of v j is the characteristic of all the stay vertices, e k is the characteristic of all edges; a j 、b k The weights obtained through self-attention mechanism training. U=Attention(U,U agg ) is the feature of U after aggregation, and the training parameters are obtained through the transformer model.

[0032] Step 140: Use the transformer model to predict the possible location of the research object at the next moment, and then go to step 160. i and the features v of other vertices in the graph j Combined together, the self-attention mechanism is used to give a regression prediction value for its association, pn ij =Attention nexttime (v i ,v j ). Then pn ijmax =argmax(pn ij ),i≠j,i,j∈{1,2,…,n}. Then the vertex corresponding to j is the most likely position at the next moment. ij Sort in descending order and take the top 3 positions as the most likely position list to recommend to the user.

[0033] Step 150: Use the transformer model to predict the possible location of the research object at the next moment, and then go to step 170. i Features Use the self-attention mechanism to give it a regression prediction value, pl i The value is the probability that the research subject will visit the specified location.

[0034] Step 160: Obtain a recommended list of possible locations; the Top-N accuracy formula is calculated as follows: TP (True Positive) makes a positive decision, and the decision is correct. FP (False Positive) makes a positive decision, but the decision is wrong. TN (True Negative) makes a negative decision, and the decision is correct. FN (False Negative) makes a negative decision, but the decision is wrong.

[0035] Step 170: Obtain a probability value of the research subject visiting the designated location.

[0036] In the above technical solution, the implementation process of step 60 is to fuse the multimodal data using a feature level fusion strategy and finally extract the preference features. Specifically, it includes the following steps:

[0037] Step 601: Input comment data.

[0038] Step 602: Input image data.

[0039] Step 603: Input rating data.

[0040] Step 604: Use the self-attention mechanism to encode the comment statement and output it as a preference feature vector of length 128; defined as f comment , f comment =Attention(Comments).

[0041] Step 605: First, unify the size of the image and output the preference feature vector. First, unify the size of the image to 512*512 pixels. The unification method uses proportional scaling. Divide 512*512 by the input image size (width, height), take the smaller ratio, and scale the length and width at the same time according to this ratio. Generate a background image with a size of 512*512 and RGB values ​​of (255, 255, 255). Then align the scaled image and the background image to form a single image. Secondly, slice the image into 16*16 pieces. Third, use the linear embedding sequence of these slices as the input of the transformer model. The final output is a preference feature vector of length 128. The image preference feature vector is defined as f image , f image =Attention(Image embedding ).

[0042] Step 606: The 3-level, 5-level, and 10-level ratings are input through three vectors with lengths of 3, 5, and 10, respectively, and the MLP model is output, the preference vector, and the rating preference feature is defined as frank .

[0043] Step 607: Based on the unified length preference features in steps 604, 605, and 606, the three types of preference features are combined into a preference matrix. Define the intermediate feature variable as f temp , f temp =concat(f comment ,f image ,f rank ). Use the self-attention mechanism to process it and output the preference feature vector. The final output multimodal context preference feature is defined as f preference , f preference =attention(f temp ).

[0044] In the above technical solution, step 70 is used to implement an algorithm for generating stay points and edges of the research object in the graph enhancement structure: using a sliding window method to extract stay points at a specified time interval. By default, the stay points of all historical data of the research object under the time interval model value are extracted. Specifically, the following steps are included:

[0045] Step 701: Determine whether a threshold speed is set; if a threshold speed is set, go to step 704; if not, go to step 702.

[0046] Step 702: Sum the distance traveled by the subject in the time interval preceding the integer multiple of the current moment.

[0047] Step 703: Calculate the average velocity in the sliding window of the research object.

[0048] Step 704: Based on steps 701 and 703, determine whether the speed in the current time interval is less than the input speed (threshold speed or average speed); if less than the input speed, go to step 706; if greater than or equal to the input speed, go to step 705.

[0049] Step 705: Based on step 704, determine whether the stay point to be generated and the previous stay point are in the same area; if they are in the same area, go to step 706; if they are not in the same area, go to step 707.

[0050] Step 706: Based on steps 704 and 705, determine whether the edge exists. If the edge exists, modify the attributes of the edge and go to step 708; if the edge does not exist, generate a new edge and go to step 709.

[0051] Step 707: Based on step 705, generate a stop point and its corresponding edge information, and save the relevant information. The information contained in the stop point and edge is shown in Tables 4 and 5. For stop points without preference features, the preference distribution is estimated using a method based on the principle of maximum likelihood estimation, and then the expected distribution of the preference feature is assigned to the stop point.

[0052] Step 708: Based on step 706, modify the attributes of the existing edge.

[0053] Step 709: Based on step 709, a new edge is generated.

[0054] Step 7010: Determine whether the sliding window has reached the current time. If it has reached the current time, then store all the stay point information. If it has not reached the current time, then go to step 701.

[0055] In the above technical solution, step 80 is used to implement an algorithm for extracting periodic features of the research object: according to the periodic features in the dense trajectory data and the established dwell vertex and edge statistics, specifically comprising the following steps:

[0056] Step 801: According to the stop vertex corresponding to the dense trajectory data, the periodic information of the stop vertex is counted; including: V duration_maximum is the maximum duration of a stay point within time T, V duration_average is the average duration of stay at the point T, V total_visit_count is the total number of visits to the stop point within time T, V cycle_visit_count is the number of visits to the stop point within Δt.

[0057] Step 802: Count the periodicity information of the edge along which the dense trajectory data passes; duration is the average time consumed on the edge, E total_visit_count is the total number of times the edge is passed in time T, E cycle_visit_count is the number of times the edge is passed within Δt;

[0058] Step 803: quantize the features in step 801;

[0059] Step 804: quantize the features in step 802;

[0060] Step 805: Assign the feature vectors obtained in steps 803 and 804 to the corresponding attributes of the corresponding stop vertices and edges respectively.

[0061] The present invention uses the self-attention mechanism in multimodal preference context fusion, graph enhancement structure and prediction, and defines the self-attention mechanism as follows:

[0062] Q=ReLu(Input W Q )

[0063] K=ReLu(Input W K )

[0064] V=ReLu(Input W V )

[0065] Where, Input=[v1,v2,…,v n ] T Represents an input matrix composed of multiple vectors.

[0066] is the model weight. model d k d v are the model dimension, the dimensions of Q and K, and the dimension of V respectively. MultiHead(Q,K,V)=concat(head1,…,head h )W o , where head i =Attention head (QW i Q ,KW i K ,VW i V ), is the multi-head attention weight. The relationship between the input and output of the self-attention mechanism used in the model is:

[0067] BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0069] Figure 1 Inventing the overall data flow diagram

[0070] Figure 2 Multimodal preference context fusion graph

[0071] Figure 3 Add stop vertices and edges to the graph to generate a flow chart

[0072] Figure 4 Add a flowchart for extracting periodic features in the structure of the graph

[0073] Figure 5 Table 1 Time interval and space interval adjustment correspondence table

[0074] Figure 6 The data in Table 2 includes a feature description table

[0075] Figure 7 Table 3 Geographical location classification table

[0076] Figure 8 The features of the stay points in Table 4

[0077] Figure 9 The features included in Table 5 DETAILED DESCRIPTION

[0078] The following will describe the embodiments of the present invention in detail with reference to the accompanying drawings and examples, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0079] The location prediction method based on preference context and trajectory map enhanced expression includes the following specific steps: Figure 1 shown.

[0080] Step 10: Set multiple hyperparameters based on time intervals and spatial intervals. The hyperparameters that need to be set in the present invention include at least 5 hyperparameters that need to be set in advance. Specifically: First, specify the research area. Generally speaking, a city can be used as a research area. For those that cannot be divided evenly, the length and width of the research area can be appropriately enlarged to facilitate segmentation. At the same time, the classification of all POIs in the geographic location segment is used to generate a feature vector using a statistical method. And the POI classification feature vector is assigned to V in the vertex feature vector. locationPart 1: Second, specify a prediction research object. The prediction research object must have historical trajectory data for at least 3 months, as described in detail in step 30. Third, set the time interval. Set the fixed time interval as the minimum time division granularity for the operation, denoted as Δt, in minutes, with a default value of 60 minutes. The six fixed time intervals are: 30 minutes, hours (default value), days, weeks, months, and 3 months. It should be emphasized that all feature extraction, modeling, and prediction results are based on the same time interval. By default, the prediction results use the same time granularity as feature extraction, modeling, and prediction. You can also choose a time granularity larger than the feature extraction and modeling time granularity, but you cannot choose a time granularity smaller than the previous step. When the time granularity is adjusted, the corresponding spatial interval will also be adjusted accordingly. The corresponding relationship of the adjustment is shown in Table 1. Fourth, set the geographic space interval division. This invention is based on two-dimensional planar geographic location data. The location data does not contain altitude information. The spatial interval is denoted as Δl, in meters, with a default value of 1000 meters. The specified area is divided into regions according to five defined spatial intervals: 500 meters, 1000 meters (default value), 10,000 meters, 50,000 meters, and customized positive integer multiples of 50,000 meters; fifth, the geographic POI data update interval is set, and the default value of this value is 1 year.

[0081] Step 20: Import POI data for the specified region into the method, and proceed to step 90. POI data is not limited to including the following features: location name, detailed classification, postal address, administrative district (in China, this must be at least as detailed as the prefecture-level city level), and longitude and latitude. Specifically, the detailed description of the included data is shown in Tables 2 and 3. POI data, i.e., Point of Interest data. Specifically, in terms of data content, POI data must have at least the following features: location name, detailed classification, postal address, administrative district (in China, this must be at least as detailed as the prefecture-level city level), and longitude and latitude.

[0082] The primary use of POI data is to label geographic locations. This labeling indicates the location's classification. This classification facilitates vectorization of POIs and facilitates computation with research subject preferences. Table 2 illustrates the features of POI data, and Table 3 describes the specific categories of "detailed classification."

[0083] The integrated POI features generate a globally unique identifier for each specific POI, designated as POI_ID. POI data is categorized into 14 major categories, 81 subcategories, and 524 minor categories. These 14 major categories include catering services, shopping services, science, education, and culture services, scenic spots, public facilities, companies and enterprises, transportation facilities and services, finance and insurance services, commercial and residential services, lifestyle services, sports and leisure services, healthcare services, government agencies and social groups, and accommodation services. This covers virtually all facility types. Another reference classification method is GB50137-2011. A category identifier is generated for each POI_ID using one-hot encoding.

[0084] For newly generated POI data, a fully connected Neural Network model is used to generate new classification labels based on existing features such as POI name and postal address, using the existing data as training data for supervised learning. If there is a large amount of newly generated POI data, manual sampling and verification of the generated labels are required.

[0085] According to the detailed classification, the reference classification method is shown in Table 3.

[0086] The classification of all POIs in the geographic location segment is used to generate a feature vector using statistical methods. The POI classification feature is recorded as f POI .

[0087] Step 30: Input the dense trajectory data of the specified area and proceed to step 50. Dense trajectory data is longitude and latitude data that conforms to the WGS 1984 standard and is collected at fixed time intervals. The characteristics of dense trajectory data are as follows: First, the time interval between each position data in the trajectory is much smaller than Δt. Second, the trajectory data of the research object is obtained for a long time. The total duration of the historical data obtained is recorded as T, and T ≥ 2160 Δt ≈ 3 months is required. Third, the data format provided by each position data can be, but is not limited to, longitude and latitude, distance and direction relative to the initial position, etc. This form is not expressed in a form similar to "Tongji University, Jiading District, Shanghai" or other descriptions containing geographical attribute features. In other words, this data format only expresses spatial information. Fourth, the final dense location data of a specific research object is represented as a set of location points arranged in chronological order, recorded as L.

[0088] Step 40: Input multimodal data for the research object in the specified region, and proceed to step 60. The multimodal preference data of the present invention includes three types: ratings, text, and images. The multimodal preference context has the following characteristics: first, a rating description of the location; second, a comment (text) description of the location; and third, images taken at the location.

[0089] Step 50: Generate a continuous divided map according to the spatial interval set in step 10, then generate vertices according to the slices, and go to step 70.

[0090] Step 60: Extract the regional preference information from the multimodal context and then embed it. Specifically, a decision fusion method based on feature fusion is used to extract the research subject preference feature vector from the multimodal data, and then proceed to step 110.

[0091] Step 70: Extract the designated research object to generate the object's stay points and the edges connecting the stay points, and proceed to step 80;

[0092] Step 80: Extract periodic information from the graph structure and proceed to step 100.

[0093] Step 90: Generate vertex position features based on the POI classification, and convert f POI Assign to V in the vertex feature vector location =f POI Part, providing step 120.

[0094] Step 100: Generate a graph enhancement structure, and generate an adjacency matrix of size n×n using the generated graph structure information, denoted as A=(a ij ),a ij =μ(v i ,v j ), where μ(v i ,v j ) is represented as a graph connection v i and v j The number of edges. Because it is a simple undirected graph, the adjacency matrix is ​​a symmetric matrix with all elements being 0 and 1 and all elements on the main diagonal being 0. Provide step 120; the generated graph structure is a simple undirected graph that does not contain cycles and parallel edges. Denoted as G(V,E). Vectorize the object of study into a vector of length 256 using its own attributes. Denoted as U. The generated geographic location fragments, each fragment is generated as a vertex. The set of all vertices is denoted as V = {v1,v2,…,v n}, where n is the number of vertices. The length of the vertex feature vector is 256, and the feature of a vertex is recorded as v i, where i is the vertex ID. Using a sliding window, the dense location data and generated vertices are used to extract dwell vertices at a specified time interval. By default, dwell vertices for all historical data of the research object under the time interval model value are extracted. The features of the dwell vertices are shown in Table 4. The length of the dwell vertex feature vector is 256.

[0095] The difference between a stop vertex and a normal vertex during the graph enhancement structure generation phase is:

[0096] 1) The feature vector of the vertex that stays contains more information; specifically, the features contained in all vertices include V identity 、V location ; The remaining features will be assigned values ​​only when they are stop vertices.

[0097] 2) There are edges connecting the stay vertex to other stay vertices, while ordinary vertices do not have edges connecting to other vertices. Based on the dense trajectory data, an edge is generated between two stay vertices. The features contained in the edge are shown in Table 5. The length of the feature vector of the jth edge is 256, denoted as E j .

[0098] By processing the historical data of the research object, the time-related features V duration_maximum 、V duration_average 、V total_visit_count 、V cycle_visit_count 、E duration 、E total_visit_count 、E cycle_visit_count For details, see the periodic feature extraction implementation step 80.

[0099] Step 110: Based on step 60, generate the multimodal preference context of the research subject and obtain f preference , providing step 120.

[0100] Step 120: Attach the trajectory features, periodic features, and preference context to the corresponding global, vertex, and edge in the generated graph enhancement structure. Finally, a matrix containing the trajectory features of the research object is formed. The attribute of the i-th vertex is recorded as v i , the kth edge is denoted as e k As shown in Table 4 and Table 5, V location =f POI 、V preference =f preference , proceed to step 130.

[0101] Step 130: Aggregate and update the vertex, edge, and global variable features in the graph enhancement structure, and proceed to step 140 and step 150; two aggregations will be performed in the graph enhancement structure to facilitate message passing between vertices, edges, and global variables. Get vertex v i Auxiliary information of v j v i Neighbors with a distance of 1, e k v i The features of the connected edges, U is the global feature; a j 、b k , c are the weights obtained through self-attention mechanism training. is the v after convergence i The features of the dataset are used to obtain the training parameters through the transformer model.

[0102] Edge aggregation uses Get edge e i Auxiliary information of v j for e i The vertices at both ends, U feature is a global feature; a j , b weights obtained through self-attention mechanism training. After convergence, e i The features of the dataset are used to obtain the training parameters through the transformer model.

[0103] Global variables aggregate using U agg =∑ j a j v j +∑ k b k e k Get U agg Auxiliary information of v j is the characteristic of all the stay vertices, e k is the characteristic of all edges; a j 、b k The weights obtained through self-attention mechanism training. U=Attention(U,U agg ) is the feature of U after aggregation, and the training parameters are obtained through the transformer model.

[0104] Step 140: Use the transformer model to predict the possible location of the research object at the next moment, and then go to step 160; all predictions are made at the next time interval, so if you need to make predictions at different time intervals, you need to train models at different time intervals. i and the features v of other vertices in the graph jCombined together, the self-attention mechanism is used to give a regression prediction value for its association, pn ij =Attention nexttime (v i ,v j ). Then Then the vertex corresponding to j is the most likely position at the next moment. ij Sort in descending order and take the top 3 positions as the most likely position list to recommend to the user.

[0105] Step 150: Use the transformer model to predict the possible location of the research object at the next moment, and then go to step 170; all predictions are made at the next time interval, so if you need to make predictions at different time intervals, you need to train models at different time intervals. i Features Use the self-attention mechanism to give it a regression prediction value, pl i The value is the probability that the research subject will visit the specified location.

[0106] Step 160: Obtain a recommended list of possible locations; the Top-N accuracy formula is calculated as follows: TP (True Positive) makes a positive decision, which is correct. FP (False Positive) makes a positive decision, but is incorrect. TN (True Negative) makes a negative decision, which is correct. FN (False Negative) makes a negative decision, but is incorrect. Step 170: Determine the probability value of the research subject visiting the specified location.

[0107] In the above technical solution, the implementation process of step 60 is to fuse the multimodal data using the feature level fusion strategy and finally extract the preference features. Figure 2 Specifically, the following steps are included:

[0108] Step 601: Input comment data.

[0109] Step 602: Input image data.

[0110] Step 603: Input rating data.

[0111] Step 604: Use the self-attention mechanism to encode the comment statement and output it as a preference feature vector of length 128; defined as f comment , f comment =Attention(Comments).

[0112] Step 605: First, unify the size of the image to 512*512 pixels. The unification method uses proportional scaling. Divide 512*512 by the input image size (width, height), take the smaller ratio, and scale the length and width at the same time according to this ratio. Generate a background image with a size of 512*512 and RGB values ​​of (255, 255, 255). Then align the scaled image and the background image to form a single image. Secondly, slice the image into 16*16 slices. Third, use the linear embedding sequence of these slices as the input of the transformer model. The final output is a preference feature vector of length 128. Defined as f image , f image =Attention(Image embedding ).

[0113] Step 606: The 3-level, 5-level, and 10-level ratings are uniformly encoded into a preference vector of length 128 through an MLP model with three input vectors of length 3, 5, and 10, respectively, and an output length of 128. Defined as f rank .

[0114] Step 607: Based on the unified length preference features in steps 604, 605, and 606, the three types of preference features are combined into a preference matrix f with a dimension of 3*128. temp =concat(f comment ,f image ,f rank ). Use the self-attention mechanism to process it and output a preference feature vector with a length of 128. preference =attention(f temp ).

[0115] In the above technical solution, step 70 is used to implement the algorithm for generating the stay points and edges of the research object in the graph enhancement structure: using a sliding window to extract the stay points of a specified time interval. By default, the stay points of all historical data of the research object under the time interval model value are extracted, such as Figure 3 Specifically, the following steps are included:

[0116] Step 701: Determine whether a threshold speed is set; if a threshold speed is set, go to step 704; if not, go to step 702.

[0117] Step 702: Sum the distance traveled by the subject in the time interval preceding the integer multiple of the current moment.

[0118] Step 703: Calculate the average velocity in the sliding window of the research object.

[0119] Step 704: Based on steps 701 and 703, determine whether the speed in the current time interval is less than the input speed (threshold speed or average speed); if less than the input speed, go to step 706; if greater than or equal to the input speed, go to step 705.

[0120] Step 705: Based on step 704, determine whether the stay point to be generated and the previous stay point are in the same area; if they are in the same area, go to step 706; if they are not in the same area, go to step 707.

[0121] Step 706: Based on steps 704 and 705, determine whether the edge exists. If the edge exists, modify the attributes of the edge and go to step 708; if the edge does not exist, generate a new edge and go to step 709.

[0122] Step 707: Based on step 705, generate a stop point and its corresponding edge information, and save the relevant information. The information contained in the stop point and edge is shown in Tables 4 and 5. For stop points without preference features, the preference distribution is estimated using a method based on the principle of maximum likelihood estimation, and then the expected distribution of the preference feature is assigned to the stop point.

[0123] Step 708: Based on step 706, modify the attributes of the existing edge.

[0124] Step 709: Based on step 709, a new edge is generated.

[0125] Step 7010: Determine whether the sliding window has reached the current time. If it has reached the current time, then store all the stay point information. If it has not reached the current time, then go to step 701.

[0126] In the above technical solution, the step 80 is used to implement an algorithm for extracting periodic features of the research object: according to the dense trajectory data and the established periodic features in the dwell vertex and edge statistics, such as Figure 4 Specifically, the following steps are included:

[0127] Step 801: According to the stop vertex corresponding to the dense trajectory data, the periodic information of the stop vertex is counted; including: V duration_maximum is the maximum duration of a stay point within time T, V duration_average is the average duration of stay at the point T, V total_visit_count is the total number of visits to the stop point within time T, V cycle_visit_count is the number of visits to the stop point within Δt.

[0128] Step 802: Count the periodicity information of the edge along which the dense trajectory data passes; duration is the average time consumed on the edge, E total_visit_count is the total number of times the edge is passed in time T, E cycle_visit_count is the number of times the edge is passed within Δt;

[0129] Step 803: quantize the features in step 801;

[0130] Step 804: quantize the features in step 802;

[0131] Step 805: Assign the feature vectors obtained in steps 803 and 804 to the corresponding attributes of the corresponding stop vertices and edges respectively.

[0132] The innovations of the present invention lie in the following three points:

[0133] The present invention primarily overcomes the shortcomings of current POI location prediction methods, which cannot effectively utilize multiple subject preference contexts and can only use implicit features to represent subject information and POI features. Furthermore, when training the model, the hyperparameter settings cannot provide convincing explanations to fully explain why subjects prefer specific locations and what causes them to visit a certain location. Furthermore, when modeling subject features, it is impossible to effectively capture the individual characteristics of the subjects, making it impossible to fully represent the subjects.

[0134] One of the innovations: Effectively organize preference context and spatial characteristics.

[0135] Traditional POI-based location prediction models cannot effectively align a user's multimodal context with their trajectory due to the sparsity of location data. This invention effectively aligns multimodal preference context with their trajectory, thereby organically integrating preference features with spatial features.

[0136] The second innovation: integrating user temporal and spatial features based on enhanced graph expression.

[0137] By integrating preference context and spatial features, this paper uses a graph structure to embed user trajectories in a representable manner, thereby organically integrating temporal and spatial features in an interpretable manner. All parameters in the graph are highly interpretable. At the same time, the individual characteristics of the research subjects are used as global variables in the graph enhancement structure, fully leveraging the graph enhancement structure's ability to express nonlinear data. This approach expands existing model construction methods for POI location prediction and can improve prediction results.

[0138] Innovation #3: Feature Extraction Based on Self-Attention. This paper fully utilizes the self-attention mechanism, applying it to the aggregation, feature fusion, and position prediction stages of the graph enhancement structure. This effectively focuses on the more heavily weighted features. This approach effectively leverages the multimodal representation potential of self-attention, making the model simpler and easier to use, making it suitable for industrial applications.

Claims

1. A location prediction method based on preference context and trajectory graph enhanced expression, characterized in that: First, based on feature representation and graph theory, the geographic space is represented as a graph structure. POI data is then vectorized and added to vertex attributes. Second, multimodal preference context is integrated into the graph structure. Third, the trajectory graph of the research object is extracted from the location data. Fourth, the vertices, edges, and global vectors in the graph are updated. Fifth, the possible locations are predicted based on the enhanced expression of the trajectory map; further, based on the feature representation and graph theory, the geographic space is represented in the form of a graph structure: by segmenting the two-dimensional plane of the geographic space, the vertices in the graph structure representing the spatial location are obtained; further, the POI data is vectorized and added to the attributes of the vertex: after vectorization through the classification features of the POI, the attributes of the geographic part in the attributes of each vertex are obtained.

2. The method according to claim 1, wherein: The specific method process is: Step 10: Set multiple hyperparameters based on time interval and spatial interval; Step 20: Import the POI data of the specified area into the method and proceed to step 90; Step 30: Input the dense trajectory data of the designated area and proceed to step 50; Step 40: Input the multimodal data of the research subjects in the designated area and proceed to step 60; Step 50: Generate a continuous partitioned map according to the spatial interval set in step 10, then generate vertices according to the slices, and proceed to step 70; Step 60: extracting the regional preference information from the multimodal context and then embedding it; specifically, using a decision fusion method based on feature fusion for the multimodal context to extract the research subject preference feature vector from the multimodal data, and then proceeding to step 110; Step 70: Extract the designated research object to generate the object's stay points and the edges connecting the stay points, and proceed to step 80; Step 80: extract periodic information from the graph structure and proceed to step 100; Step 90: Generate vertex position features based on the POI classification, recorded as ;Will Assign to the vertex feature vector Part, providing step 120; Step 100: Generate a graph enhancement structure, using the generated graph structure information to generate a size of The adjacency matrix of ,in Represented as a graph connection and The number of edges; Providing step 120; The generated graph is denoted as ; Use the properties of the research object itself to vectorize it, and record it as ; Each geographic location fragment generated is generated as a vertex; the set of all vertices is recorded as , where n is the number of vertices; Vertex feature vector, for a vertex its feature is recorded as , where i is the vertex ID; Step 110: Based on step 60, generate the multimodal preference context of the research subject, which is recorded as ;Will , providing step 120; Step 120: attaching the trajectory features, periodic features, and preference context to the corresponding global, vertex, and edge in the generated graph enhancement structure; ultimately forming a matrix containing the trajectory features of the research object; The attribute of the i-th vertex is recorded as , the kth edge is recorded as ;in, 、 , and As shown in Table 4, the regional features and preference features in the vertex features are respectively represented; proceed to step 130; Step 130: Aggregate and update the vertex, edge, and global variable features in the graph enhancement structure, and proceed to step 140 and step 150; two aggregations will be performed in the graph enhancement structure to facilitate message passing between vertices, edges, and global variables; Vertex aggregation uses Get Vertex Auxiliary information of for Neighbors with a distance of 1, for The characteristics of the connected edges, is a global feature; Weights trained through the self-attention mechanism; After the convergence The features of are used to obtain training parameters through the transformer model; Edge aggregation uses Get the edge Auxiliary information of for The vertices at both ends, is a global feature; Weights trained through the self-attention mechanism; After the convergence The features of are used to obtain training parameters through the transformer model; Global variable aggregation use get Auxiliary information of are the characteristics of all the stop vertices, is the characteristic of all edges; Weights trained through the self-attention mechanism; After the convergence The features of are used to obtain training parameters through the transformer model; Step 140: Use the transformer model to predict the possible location of the research object at the next moment, and then go to step 160; the feature of the i-th vertex where the research object is currently located Features of other vertices in the graph Combined together, the self-attention mechanism is used to give a regression prediction value for its association. ; then ; then the vertex corresponding to j is the most likely position at the next moment; Sort in descending order, and take the top three positions as the most likely position list to recommend to the user; Step 150: Use the transformer model to predict the possible location of the research object at the next moment, and then go to step 170; Features Use the self-attention mechanism to give it a regression prediction value, ; The value is the probability that the research subject will visit the specified location; Step 160: Obtain a recommended list of possible locations; the Top-N accuracy formula is calculated as follows: TP makes a positive decision, and the decision is correct; FP makes a positive decision, but the decision is wrong; TN makes a negative decision, and the decision is correct; FN makes a negative decision, but the decision is wrong; Step 170: Obtain a probability value of the research subject visiting the designated location.

3. The method according to claim 2, wherein: The implementation process of step 60 is to fuse the multimodal data using a feature level fusion strategy and finally extract the preference features; Specifically, it includes the following steps: Step 601: Enter comment data; Step 602: Input image data; Step 603: input rating data; Step 604: Use the self-attention mechanism to encode the comment statement into a preference feature vector of length 128; defined as , ; Step 605: First, unify the size of the image and output the preference feature vector; first, unify the size of the image to 512*512 pixels; the unification method adopts proportional scaling; divide 512*512 by the input image size, take the smaller ratio, and scale the length and width at the same ratio; generate a background image of size 512*512 and RGB values ​​255, 255, 255; then align the scaled image and the background image to form a single image; secondly, slice the image into 16*16 pieces; thirdly, use the linear embedding sequence of these slices as the input of the transformer model; finally, output a preference feature vector of length 128; the image preference feature vector is defined as , ; Step 606: The 3-level, 5-level, and 10-level ratings are passed through three input vectors with lengths of 3, 5, and 10, respectively, and the MLP model is output, the preference vector, and the rating preference feature is defined as ; Step 607: Based on the unified length preference features in steps 604, 605, and 606, the three types of preference features are combined into a preference matrix; definition The intermediate characteristic variable is , ; Use the self-attention mechanism to process it and output the preference feature vector; the final output multimodal context preference feature is defined as , .

4. The method according to claim 2, wherein: The step 70 is used to implement an algorithm for generating the stay points and edges of the research object in the graph enhancement structure. Specifically, it includes the following steps: Step 701: Determine whether a threshold speed is set; if a threshold speed is set, go to step 704; if not, go to step 702; Step 702: Sum the distance traveled by the subject in the time interval preceding the integer multiple of the current moment; Step 703: Calculate the average velocity in the sliding window of the research object; Step 704: Based on steps 701 and 703, determine whether the speed in the current time interval is less than the input speed; If it is less than the input speed, go to step 706; if it is greater than or equal to the input speed, go to step 705; Step 705: Based on step 704, determine whether the stay point to be generated is in the same area as the previous stay point; If they are in the same region, go to step 706; if they are not in the same region, go to step 707; Step 706: Based on steps 704 and 705, determine whether the edge exists. If the edge exists, modify the edge attributes and go to step 708. If the edge does not exist, generate a new edge and go to step 709. Step 707: Based on step 705, generate a stay point and corresponding edge information, and save the relevant information of the stay point and edge; for stay points without preference features, use the maximum likelihood estimation method to estimate the distribution of preferences, and then assign the expected value of the preference feature distribution to them; Step 708: Based on step 706, modify the attributes of the existing edge; Step 709: Based on step 709, a new edge is generated; Step 7010: Determine whether the sliding window has reached the current time. If it has reached the current time, then store all the stay point information. If it has not reached the current time, then go to step 701.

5. The method according to claim 2, wherein: The step 80 is used to implement an algorithm for extracting periodic features of the research object: according to the periodic features in the dense trajectory data and the established dwell vertex and edge statistics, specifically comprising the following steps: Step 801: Count the periodic information of the dwelling vertices corresponding to the dense trajectory data, including: is the maximum duration of a stopover point within time T, is the average duration of stay at the point T, is the total number of visits to the stop point within time T, For The number of visits to the stopover point within the period; Step 802: Count the periodicity information of the edge along which the dense trajectory data passes; is the average time consumed on the edge, is the total number of times the edge is passed in time T, For The number of times the inner edge is passed; Step 803: quantize the features in step 801; Step 804: quantize the features in step 802; Step 805: Assign the feature vectors obtained in steps 803 and 804 to the corresponding attributes of the corresponding stop vertices and edges respectively.

6. The method according to any one of claims 1 to 5, characterized in that: The self-attention mechanism is used in multimodal preference context fusion, graph enhancement structure and prediction, and the self-attention mechanism is defined as follows: in, Represents an input matrix composed of multiple vectors; , , is the model weight; 、 、 They are the model dimension, the dimensions of Q and K, and the dimension of V; ,in , is the multi-head attention weight; the relationship between the input and output of the self-attention mechanism used in the model is: .

Images