City interest point representation learning semantic enhancement system based on large language model

Abstract

The application discloses a city interest point representation learning semantic enhancement system based on a large language model, wherein three different types of prompt words are designed for the interest point representation learning task in the system, and each type of prompt word is composed of three different parts, each part providing different aspects of information for the large language model to improve the quality of the generated results of the large language model; in addition, the last hidden layer output of the large language model is taken as the feature vector of the output, and the cross-attention mechanism is used to align the information extracted from different types of prompt words, then the aligned feature vectors are fused by using a multi-layer perception and a weighted sum, and finally the fusion result is used to enhance the representation of the interest point based on the cross-attention mechanism. Through comparative learning from three angles, a representation vector with higher quality than the original interest point representation can be finally obtained, so that the purpose of enhancing the representation of the interest point is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and intelligent transportation systems, and more particularly to a semantic enhancement system for learning and representing urban points of interest based on a large language model. Background Art

[0002] With the widespread use of smart electronic devices, the amount of location-based mobile data, such as user check-in data at points of interest and user trajectory data, has surged. This data has high commercial value, and downstream tasks based on this data, such as user destination prediction and traffic flow prediction, have also gained widespread application. As one of the fundamental supports for these tasks, point of interest representations largely determine the performance of these downstream tasks. Developing better point of interest representations is a current research focus in this field.

[0003] Large language models have been a research focus in the field of artificial intelligence in recent years. Large language models trained on massive amounts of internet text data not only demonstrate impressive capabilities in natural language processing but also possess significant potential in spatiotemporal domains. However, to date, no work has applied large language models to the field of POI representation. Learning POI representations with large language models has the potential to further improve the performance of various downstream tasks, but currently, this technical approach still faces the following challenges:

[0004] First, it's difficult to effectively extract geographical knowledge highly relevant to POIs from large language models. While numerous studies have explored how to design prompts to extract geographical knowledge from large language models, current large language models are limited by their poor understanding of numbers and inability to accurately process them. Consequently, they are unable to effectively capture geographical information about POIs, including precise numerical data such as GPS coordinates.

[0005] Second, it's difficult to integrate the textual information output by large language models into systems for learning POI representations. Many existing studies have used a technique that uses one-hot encoding of POI category features and directly concatenates this encoding with the POI representation vector to generate the final result. However, using one-hot encoding degrades the semantic information contained in the vector, while the concatenation operation results in abrupt transitions between different parts of the vector, making downstream tasks more difficult. Summary of the Invention

[0006] In view of this, the present invention addresses the needs and shortcomings of current technological development and provides a semantic enhancement system for urban POI representation learning based on a large language model. It can make full use of the text generated by the large language model based on geographical knowledge to enhance the POI representation, thereby improving the performance of downstream tasks related to POI.

[0007] To achieve the above object, the present application adopts the following technical solutions:

[0008] The embodiment of the present application provides a city interest point representation learning semantic enhancement system based on a large language model, which enhances the representation of an interest point based on geographical knowledge generated by a large language model, and the system comprises a prompt word generation and feature extraction module, an interest point representation enhancement module and a multi-view contrast learning module, wherein:

[0009] The prompt word generation and feature extraction module comprises three types of prompt words, namely interest point access mode, interest point address and interest point surrounding, each type of prompt word is composed of role introduction, interest point information and question, and provides different aspects of information for the large language model; in the feature extraction process, the three types of prompt words are sequentially transmitted into the large language model, and the final output text is ignored, and a feature vector is extracted from the last hidden layer of the large language model;

[0010] The interest point representation enhancement module uses a cross-attention mechanism to align the feature vectors extracted by different types of prompt words, then uses a multi-layer perception and a weighted sum to fuse the aligned feature vectors, and finally uses the fusion result to enhance the representation of the interest point based on the cross-attention mechanism;

[0011] The multi-view contrast learning module selects positive samples by using different views, and generates negative samples by random sampling; and uses a loss function to perform contrast learning, reduces the distance between representation vectors of the same type in the hidden space, and increases the distance between representation vectors of different types.

[0012] Further, in the role introduction part, the prompt word instructs the large language model to act as a role familiar with geographical information; in the interest point information part, according to the different types of prompt words, the corresponding interest point information is transmitted into the large language model in the form of natural language; and in the question part, different questions are designed for different types of prompt words.

[0013] Further, the multi-view contrast learning module selects positive samples by using three different views, and the three different views comprise a check-in record sequence, a geographical angle and a functional angle.

[0014] Further, a check-in record is represented by a triple r=(u,p,t), which indicates that a user u visited an interest point p at time t, and the behavior of the user in a period of time is composed of a sequence R={r1,r2,…,r L} of check-in records, wherein each check-in record r1,…,r L is arranged in chronological order, and L represents the length of the check-in record sequence.

[0015] Further, the loss function used by the multi-view contrastive learning module includes an InfoNCE loss function and a cosine similarity-based loss function, and the expression is:

[0016]

[0017] In the formula, represents the InfoNCE loss function, represents the cosine similarity-based loss function; p i , respectively represent the original sample, the positive sample and the negative sample in the contrastive learning, sim is a similarity function, gamma is a temperature parameter, m is the total number of interest points in the training batch, and cos is a cosine similarity function; E FUSE represents the enhanced interest point representation, and E POI represents the original interest point representation vector before enhancement.

[0018] Compared with the prior art, the technical solution has at least the following technical advantages:

[0019] The application provides a city interest point representation learning semantic enhancement system based on a large language model, and in view of the problem that it is difficult to extract geographic information from the large language model, three different types of prompt words are specially designed for the interest point representation learning task, and each type of prompt word is composed of three different parts, each part provides different aspects of information for the large language model to improve the quality of the generated results of the large language model; in view of the problem that it is difficult to integrate the output of the large language model and the interest point representation vector, the last hidden layer output of the large language model is used as the feature vector of the output, and a cross-attention mechanism is used to align the information extracted from different types of prompt words, then a multi-layer perception and a weighted sum are used to fuse the aligned feature vectors, and finally the fusion result is used to enhance the representation of the interest point based on the cross-attention mechanism. By carrying out contrastive learning from three angles, the system of the application can finally obtain a representation vector with higher quality than the original interest point representation, thereby achieving the purpose of enhancing the interest point representation; using the system framework of the application can significantly improve the performance of the interest point representation method in various downstream tasks.

[0020] Other features and advantages of the application will be set forth in the following description of the application, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and the appended drawings.

[0021] The technical solutions of the application will be further described in detail below with reference to the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced as follows. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0023] The accompanying drawings are used to provide a further understanding of the present application, and constitute a part of the specification, and are used to explain the present application together with embodiments of the present application, and do not constitute a limitation on the present application.

[0024] Figure 1 The overall architecture schematic diagram of the city interest point representation learning semantic enhancement system based on a large language model provided by the embodiments of the present application.

[0025] Figure 2 The ablation experiment result schematic diagram under different downstream tasks provided by the embodiments of the present application.

[0026] Figure 3 The influence schematic diagram of different model layers on overall performance provided by the embodiments of the present application.

[0027] Figure 4 The performance schematic diagram of the POI clustering task on different data sets provided by the embodiments of the present application.

[0028] Figure 5 The electronic device structure schematic diagram provided by the embodiments of the present application. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0030] In the description of the present application, it should be noted that in some processes described in the specification and the accompanying drawings, a plurality of operations appear in a specific order, but it should be clearly understood that these operations can be executed or performed in parallel or in the order in which they appear in this text. In addition, various serial numbers and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0031] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0032] Referring to Figure 1 As shown, the present application provides a POI-Enhancer framework (Point of Interest, POI), that is, a city interest point representation learning semantic enhancement system based on a large language model. This system framework can fully utilize the text generated by the large language model based on geographical knowledge to enhance the representation of the interest point, thereby improving the performance of numerous downstream tasks related to the interest point.

[0033] The problem studied by the present application can be represented by the following several definitions:

[0034] Definition 1 (Point of Interest): A point of interest is a geographic location represented by a five-tuple (id, pn, c, lon, lat), where id represents the node number, pn represents the name of the point of interest, c represents the category to which the point of interest belongs, and lon and lat represent the longitude and latitude of the point, respectively. In addition to the five-tuple, each point of interest can carry certain additional attributes, such as access patterns, addresses of points of interest, and environments around points of interest. A typical point of interest in a city data set is shown in Table 1.

[0035] Table 1 Information contained in a typical point of interest in the New York data set

[0036] Field Value Point of Interest Number 22337 Name New York Stock Exchange Longitude 74.011154 Latitude 40.706806 Point of Interest Category Stock Exchange Street Name Wall Street House Number 11 Postal Code 10005 Point of Interest Surroundings Office, Building and Road Access Pattern Between 6 am and 9 am, Weekday

[0037] Definition 2 (Check-in Record): A check-in record is represented by a three-tuple r = (u, p, t), which means that user u visited point of interest p at time t. The behavior of a user over a period of time can be represented by a sequence of check-in records R = {r1, r2, …, r L} where each check-in record r1, …, r L is arranged in chronological order, and L represents the length of the sequence of check-in records. The present application additionally defines the set of all users' check-in record sequences as S.

[0038] Definition 3 (Point of Interest Representation): For the set of all points of interest P = {p1, p2, …, p N}, the present application maps each point of interest p i to a fixed vector This vector i.e. the representation of the point of interest p i , where N is the total number of points of interest.

[0039] Problem statement: for any point of interest representation mapping function Point of interest set P = {p1, p2, …, p N}, check-in sequence S and other related data, the system of the present application learns a function g that can enhance the point of interest representation mapping function so that the enhanced representation vector can achieve better performance on downstream tasks.

[0040] The architecture, working principle and specific implementation of the system of the present application will be described in detail below in conjunction with Figures 1-4 .

[0041] I. System framework

[0042] In one specific embodiment, the framework of the system proposed by the present application can be divided into three module parts: prompt word generation and feature extraction, point of interest representation enhancement and multi-view contrastive learning, corresponding to the three parts (a), (b) and (c) in Figure 1 . Among them:

[0043] In the prompt word generation and feature extraction part, as shown in part (a) of Figure 1 , the present application designs three categories of prompt words for the task of enhancing point of interest representation using geographic knowledge in large language models: point of interest access mode, point of interest address, and point of interest surrounding area. Each category of prompt word is composed of three parts: role introduction, point of interest information, and question. In the role introduction part, the prompt word instructs the large language model to play the role of a resident familiar with local geographic information. In the point of interest information part, according to the different types of prompt words, the present application transmits the corresponding point of interest information into the large language model in the form of natural language. In the question part, the present application designs different questions for different types of prompt words to fully mobilize the geographic knowledge inside the large language and improve the quality of the output of the large language model. In the feature extraction process, the present application transmits the three categories of prompt words into the large language model in turn, ignores the final output of the text, extracts the vector from the last hidden layer of the large language model, and further processes it. Let the three categories of prompt words be , then the present application can obtain three corresponding feature vectors as follows:

[0044]

[0045] Among them, is the large language model ignoring all structures after the last hidden layer.

[0046] In the embodiment of the present application, the interest point representation enhancement part includes three step modules: double feature alignment, semantic feature fusion and cross attention fusion. The double feature alignment part involves aligning feature vectors with certain relationships, thereby further improving the quality of the feature vectors. In this part, the three feature vectors obtained above will be converted into higher-dimensional hidden vectors

[0047]

[0048] wherein, d is the dimension of the hidden space, W V′ is a learnable parameter, A is a learnable parameter, S All three matrices can be learned. After completing the conversion, for this pair of vectors, the present application uses a cross attention mechanism to pair and combine them into a vector. First, the present application defines

[0049]

[0050] wherein, is a learnable parameter, φ is a Softmax function, d h represents the dimension size of each head in the multi-head attention mechanism, || is the concatenation operation of vectors, is also a learnable weight parameter, and H represents the number of heads in the multi-head attention mechanism. Let the output of the first layer of this multi-layer network be Z'1, then

[0051]

[0052] Z'1 = LN(Z + FFN(Z))

[0053] wherein, FFN is a basic feedforward neural network. After obtaining the output Z'1 of the first layer network, this vector will be input into the next layer network together with E A as a new input value until the L1 layer network is calculated. The present application records the output of the last layer network as E A-V ; the calculation process of E A-S is similar to that of E A-V , which will not be described here.

[0054] In the semantic feature fusion part, the present application combines the two feature vectors E A-V and E A-S obtained before through a weighted sum. First, the present application calculates

[0055] θ A-V = W2·LeakyReLU([W1E A-V || W1EA-S ])

[0056] θ A-S =W2·LeakyReLU([w1E A-S ||W1E A-V ])

[0057] Among them, θ A-V ,θ A-S For E A-V ,E A-S The attention score, The role of is to project the feature vector into the latent space or convert it into an attention score.

[0058] After calculating the score, the present invention can calculate the weight in the weighted sum

[0059] [ω A-V ,ω A-S ]=φ([θ A-V ,θ A-S ])

[0060] Among them, ω A-V ,ω A-S Respectively represent E A-V ,E A-S The corresponding weighted combination weight.

[0061] And calculate the final output

[0062] E LLM =ω A-V ·E A-V +ω A-S ·E A-S

[0063] in,

[0064] In the last step of cross attention fusion, the present invention uses the cross attention mechanism to obtain E LLM To represent the output E of the interest point representation model POI Enhance. Let the output of the first layer of this multi-layer model be X'1, then

[0065] X=LN(E POI +MQA(E LLM ,E POI ))

[0066] X′1=LN(X+FFN(X))

[0067] This process is repeated until the L2 layer network completes the calculation, and the output of the last layer is recorded in the present invention. For E FUSE .

[0068] In one specific embodiment, in the multi-view contrastive learning part, the application preferably uses three different views to select positive samples. Specifically, from the perspective of the user's check-in record sequence, the various points of interest visited by the user on the same day are likely to have similarities, so the application stipulates that a point of interest visited by the user on the same day and the two points of interest visited before and after it are positive samples of each other. From the perspective of geography, points of interest close in geographic location should have similarities, so for any point of interest, the application constructs a fixed-size square region on the map centered on it, and stipulates that all points of interest in the region are positive samples of the point of interest. From the perspective of function, points of interest belonging to the same point of interest category and having similar access patterns should be semantically similar, so the application sets points of interest that meet these two conditions as positive samples; and negative samples are generated by random sampling.

[0069] In one specific embodiment, the loss function of the contrastive learning process of the application is composed of two parts. The first part is the InfoNCE loss function, the expression of which is:

[0070]

[0071] where p i , represent the original sample, the positive sample and the negative sample in the contrastive learning respectively, sim is the similarity function, γ is the temperature parameter, and m is the total number of points of interest in the training batch; and in order to ensure that the similarity between the point of interest representations before and after reinforcement is not lost, the application adds another loss function based on cosine similarity, the expression of which is:

[0072]

[0073] where cos is the cosine similarity function;

[0074] The loss function used by the application in the training process is the sum of the two parts, i.e.

[0075] II. Experimental verification

[0076] In the system evaluation link, the application uses Foursquare-NY, Foursquare-SG, FourSquare-TKY three user check-in sequence data sets collected in New York, Singapore and Tokyo respectively. In order to reduce the noise in the data, the application removes the interest points with less than 5 check-ins and the user check-in sequence with a length less than 10. The statistical information of the processed data set is shown in Table 2. In all experiments, the application shuffles the data in the data set and divides the training set, validation set and test set according to the ratio of 7:1:2.

[0077] Table 2 Statistical information of the preprocessed data set

[0078] Dataset Number of Users Number of Points of Interest Number of Check-in Records FourSquare-NY 15,171 24,118 641,005 FourSquare-SG 10,909 20,154 696,306 FourSquare-TKY 2,293 15,164 496,459

[0079] In order to fully demonstrate the ability of the system proposed by the application, the application selects six different interest point representation models Skip-Gram, POI2Vec, Geo-Teaser, TALE, Hier and CTLE to conduct experiments, and evaluates them on three different downstream tasks of interest point recommendation, check-in sequence classification and interest point access flow prediction. For the interest point recommendation task, the application uses Hit@k, that is, the frequency of the first k interest points recommended by the model as the evaluation index; for the check-in sequence classification task, the application uses accuracy (ACC) and Macro-F1 as the evaluation index; for the interest point access flow prediction, the application uses mean absolute error (MSE) and root mean square error (RMSE) as the evaluation index. In this experiment, the application uses Llama-2-7B as the backbone large model, sets the hidden space size d to 256, L1 and L2 to 4 and 2 respectively, the temperature parameter γ to 0.1, trains 100 epochs, and uses the optimizer AdamW with the learning rate of 0.001 and the weight decay coefficient of 0.001.

[0080] The results of this experiment are shown in Table 3. On the POI recommendation task, the model of the present application, POI2VEC, improved the performance of POI2VEC on the New York dataset by more than 20%, which is likely due to the fact that the POI2Vec model focuses on capturing the relationships between each piece of check-in information within the sequence, while ignoring information from other modalities. On the check-in data sequence classification task, the POI2Vec model also achieved a very high improvement, while the Skip-Gram algorithm improved less, which is likely because the Skip-Gram model focuses more on modeling user trajectories, and its optimization space is less than other algorithms. On the POI visit traffic prediction, the CTLE, which is the SOTA model, showed a large improvement in performance after reinforcement, which further demonstrates the effectiveness of the model of the present application in reinforcing POI representation. At the same time, the present application has improved in almost all datasets, models and downstream tasks, which also demonstrates the feasibility and effectiveness of the method of extracting additional geographic knowledge from large language models and fusing it into POI representation.

[0081] Table 3: POI-Enhancer framework improvement on each model, dataset and downstream task

[0082]

[0083]

[0084] Continued Table

[0085]

[0086] Continued Table

[0087]

[0088] Further, in addition to the main experiment, the present application also conducted additional ablation experiments, parameter analysis and representation quality analysis. The experimental results of the ablation experiment are shown in Table 4. Figure 2where / P represents that the system removes the specially designed role introduction and question part in the prompt, / D represents that the system removes the double feature alignment and semantic feature fusion module, / F represents that the system uses direct splicing instead of the cross-attention fusion module, and / C represents that the system only selects positive samples from the geographical perspective when performing contrastive learning. The experimental results show that when the system model is reduced according to the above steps, the quality of the generated point of interest representation will decline; this shows that the special prompt design of the system can better extract information from the large language model, the double feature alignment and semantic fusion can ensure the quality of the obtained feature vector, the cross-attention fusion module can better combine the information in the large language model into the existing point of interest representation, and the contrastive learning under multiple views can bring more useful information to the model than single view.

[0089] The experimental results of parameter analysis are shown in Figure 3 The experiment was completed using the Hier model on the New York dataset. It can be found that under the condition that other conditions remain unchanged, as L1 or L2 increases, the performance of the model first increases and then decreases, and the performance of the experiment is best when L1 = 4 and L2 = 2. This is because when the number of layers of the model of the application is too low, it cannot correctly complete the feature alignment and feature fusion task, and when the number of layers of the model is too high, overfitting and other problems will cause the model to decline in the test set. Therefore, in the main experiment, L1 and L2 are set to 4 and 2, respectively.

[0090] The experimental results of quality analysis are shown in Figure 4 To further measure the quality of the enhanced point of interest representation, the application uses the K-means algorithm to test the clustering task on three datasets. In this process, the application uses the K-means algorithm to cluster the point of interest representation vectors before and after enhancement, and uses the normalized mutual information (NMI) index to evaluate the clustering results. From the experimental results, it can be found that compared with the results before enhancement, the quality of the point of interest representation after enhancement has been significantly improved. This shows the following three points: (1) the application successfully extracts high-quality features from the large language model, and this feature plays a key role in improving the quality of the point of interest representation; (2) the application efficiently fuses the extracted features into the unenhanced point of interest representation, improving its various properties; (3) the multi-view contrastive learning method used in the application can reduce the distance between the same type of representation vectors in the hidden space, while increasing the distance between different types of representation vectors.

[0091] From the description of the above embodiments, those skilled in the art can know that: the present application innovatively puts forward a system framework named POI-Enhancer which can enhance the representation of points of interest by using large language models. In order to achieve this goal, on the one hand, the present application overcomes the challenge that large language models are not sensitive to numbers and are difficult to extract information from large language models by designing three special prompt words; on the other hand, in order to better align and fuse the obtained information, the present application designs three steps of double feature alignment, feature fusion and cross attention fusion to replace the relatively simple splicing method. In addition, the present application also designs a multi-view contrast learning training method for the training of this system framework. Experiments prove that the system framework of the present application can significantly improve the performance of almost all points of interest representation methods in various downstream tasks.

[0092] Further, referring to Figure 5 The electronic device can include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and can further include a computer program stored in the memory 11 and executable on the processor 10, and the processor executes the computer program to implement the above-mentioned large language model-based city point of interest representation learning semantic enhancement system, which fully utilizes the text generated by the large language model based on geographical knowledge to enhance the representation of points of interest, thereby improving the performance of numerous downstream tasks related to points of interest.

[0093] In some embodiments, the processor 10 can be composed of integrated circuits, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple packaged integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and combinations of various control chips, etc. The processor 10 is the control core of the electronic device, which connects all components of the electronic device through various interfaces and lines, and executes or runs programs or modules stored in the memory 11 and calls data stored in the memory 11 to perform various functions and process data of the electronic device.

[0094] Those skilled in the art should understand that the embodiments of the present application can be provided as a computer program product or an electronic device. Therefore, the present application can be in the form of a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0095] It is noted that the word "comprising" does not exclude not present components or steps not listed in a claim. The word "a" or "an" preceding the

[0096] The various embodiments in the specification are described in progressive order with reference to each embodiment, and each embodiment emphasizes different aspects of the other embodiments. The same or similar parts between the various embodiments are referred to each other.

[0097] The above description of disclosed embodiments is intended to enable the person skilled in the art to implement or use the invention. Various modifications to these embodiments will be apparent to a person skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to these embodiments shown herein, but is to accord with the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A city point of interest representation learning and semantic enhancement system based on a large language model, characterized by: The system uses a large language model to generate text based on geographic knowledge to enhance the representation of points of interest. The system includes: a prompt word generation and feature extraction module, a point of interest representation enhancement module, and a multi-view comparative learning module, where: The prompt word generation and feature extraction module includes three types of prompt words: POI access mode, POI address, and POI surroundings. Each type of prompt word consists of three parts: character introduction, POI information, and question, providing different aspects of information for the large language model. During the feature extraction process, the three types of prompt words are sequentially input into the large language model, and the final output text is ignored. The feature vector is extracted from the last hidden layer of the large language model. The interest point representation enhancement module uses a cross-attention mechanism to align the feature vectors extracted from different types of prompt words, then fuses the aligned feature vectors using a multi-layer perceptron and weighted sum, and finally uses the fusion result to enhance the interest point representation based on the cross-attention mechanism; The multi-view contrastive learning module uses different views to select positive samples and randomly samples to generate negative samples; and uses the loss function to perform contrastive learning, reducing the distance between representation vectors of the same type in the latent space while increasing the distance between representation vectors of different types; The multi-view comparative learning module uses three different views to select positive samples. The three different views include: check-in record sequence, geographical perspective, and functional perspective. Specifically: Based on the user's check-in record sequence, the various points of interest visited by the user on the same day may have similarities. Therefore, it is stipulated that a point of interest visited by the user on the same day and the two points of interest visited before and after the user are mutually positive samples; From a geographical perspective, points of interest with close geographical locations should be similar. Therefore, for any point of interest, a square area of ​​fixed size is constructed on the map with it as the center, and all points of interest in the area are defined as positive samples of the point of interest. From a functional perspective, points of interest that belong to the same point of interest category and have similar access patterns should be semantically similar, so points of interest that meet these two conditions are set as positive samples of each other.

2. The city point of interest representation learning semantic enhancement system based on a large language model according to claim 1 is characterized in that: In the role import part, the prompt word instructs the large language model to act as a role familiar with geographic information; in the point of interest information part, the corresponding point of interest information is passed into the large language model in the form of natural language according to the type of prompt word; in the question part, different questions are designed for different types of prompt words.

3. The city point of interest representation learning semantic enhancement system based on a large language model according to claim 1 is characterized in that: A check-in record is represented by a triplet r = (u, p, t), which means that user u visited the point of interest p at time t. The user's behavior over a period of time is represented by a sequence of check-in records R = {r1, r2, ..., r L }, where each sign-in record r1,…,r L Arranged in chronological order, L represents the length of this sign-in record sequence.

4. The city point of interest representation learning semantic enhancement system based on a large language model according to claim 1 is characterized in that: The loss function used by the multi-view contrast learning module includes: InfoNCE loss function and cosine similarity-based loss function, which are expressed as: Where, represents the InfoNCE loss function, represents the loss function based on cosine similarity; p i , They represent the original sample, positive sample and negative sample in contrastive learning respectively, sim is the similarity function, γ is the temperature parameter, m is the total number of interest points in the training batch; cos is the cosine similarity function; E FUSE represents the enhanced interest point representation, E POI Represents the original interest point representation vector before enhancement.

5. An electronic device, characterized in that: The system comprises a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement a city interest point representation learning semantic enhancement system based on a large language model as described in any one of claims 1 to 4.

Images